This past January, a strange phenomenon briefly overtook Twitter, Instagram, and Facebook: Post after post showed selfies of friends and celebrities juxtaposed with artistic doppelgängers. Behind these matches was Google’s Arts & Culture app, which let users take playful advantage of machine learning tools to find which framed masterpiece or sculpture resembled them best. Some of the comparisons were spot on. Others were, well, very rough approximations.
Although the virality of art selfies faded after just a few days of fame, it’s clear that Google took note of their popularity. The company is back with a different — and, arguably, more fun — version of the art selfie, this time called Move Mirror. Developed by another team at Google, Move Mirror is meant to show how machine learning can “be used in more playful ways”.
AdvertisementADVERTISEMENT
"We wondered, what fun experiences could we create with people moving in front of their webcam?" Jane Friedhoff and Irene Alvarado, both creative technologists at Google's Creative Lab, told Refinery 29 over email. "How could we find weird, serendipitous connections across the breadth of human movement — from martial arts to cooking to skiing to babies taking their first steps?"
As its name implies, Move Mirror does not use a static selfie. Instead, it works to match your dance movements with those in a library of over 80,000 images, pulled from videos licensed from YouTube and Shutterstock in addition to original footage.
Here’s how it works: Open Chrome and head to the Move Mirror site. Turn on your computer’s web cam and make sure you’re the only person in the frame. (Google says that no webcam images on Move Mirror are ever sent to a server — everything remains on-device.) Stand further back to get more of your body in the shot. When you’re ready to get your groove on, tap "start recording" and a five-second countdown will begin. You’ll see the photo matches appear alongside your dance video, and can even choose to make the resulting compilation into a share-able Gif. The experience is an addictive one: You will want to record yourself over and over again to see new matches.
According to Google’s Blog, Move Mirror is powered by PoseNet, a machine learning model that can figure out where “key body joints” are in real-time and use that information to find companion images. You might see yourself next to a yoga instructor or a ballerina in another country.
AdvertisementADVERTISEMENT
"We wanted to work with a dataset that is representative of the diversity that exists all over the world," said Friedhoff and Alvarado. "To be clear, the pose estimation technology we’re using, PoseNet, does not take into account race, gender, height, body type, physical ability, or any other individual characteristic."
Depending on what moves you bust out, the result can feel a bit like your own take on comedian Judson Laipply's viral YouTube video, Evolution of Dance. Move Mirror hasn't made the social rounds yet, but it's only a matter of time before it's as a big as a photo of you next to a crotchety old lady from the sixteenth century.