Facebook’s New Colocation And Image Recognition Patents Tease The Future Of Sharing |
Posted: June 9, 2013 |
Facebook’s empire was built on photo tags and sharing, but it’s a grueling process many neglect. Luckily, new Facebook patents give it tech to continuously capture video whenever your camera is open, rank and surface the best images, and auto-tag them with people, places, and businesses. They tease a future where pattern, facial, and audio recognition identify what you’re seeing for easy sharing. The patents are for Automatic Photo CaptureBased on Social Components and Identity Recognition (’80), Preferred images from captured video sequence (’00), and Image selection from captured video sequence based on social components (’65). The were filed for in October 2011 and granted over the last two months to Facebook and its employeesAndrew “Boz” Bosworth, David Garcia, and Soleio Cuervo (who now works at Dropbox). The patents cover some colocation technologies similar to that of failed startup Color, who came out of stealth in March 2011 a few months before Facebook filed for the patents. That may be no coincidence, and Color’s ideas for using every available sensor on a phone to tell who someone is with may have inspired Facebook to brainstorm in the space. Soon after Color emerged from stealth,I called on Facebook to develop its own colocation technology to help it forge an “implicit social graph” of who you spend time with. What it came up with could redefine the way we share. As we look at what the patents include, I’ll be referencing the last two digits of the patents number and their PDF page numbers so you can follow along. SHOOTING VIDEO WHILE YOU TAKE PHOTOSThe foundation of the patents is the idea that Facebook can capture video of everything you see in the viewfinder while you use the camera in its main smartphone apps or its standalone Camera app. It’s like an infinite “Zoe” video that some Android cameras take surrounding a photo:
Basically, Facebook could let you take traditional photos while dicing up continuously recorded video into still images. As camera lenses, storage capacity, and wireless connections improve, these images will increase in quality. KNOWING WHAT YOU SEEWhat’s special is what Facebook could do with these videos and images. The patents describe the ability to scan the frames for important things like public figures via facial recognition, brands or products via image matching, and landmarks or businesses via pattern and text character recognition plus location:
You might not want to formally tag these things, but the tags would be pre-filled for easy sharing. Whether or not you display the tags, recognition of he presence of these objects and locations can tell Facebook what the most important frames of your video are, exactly where you are, and what types of businesses might want to reach you. COLOCATING YOU AND YOUR FRIENDSFacebook’s patents also give it exciting ways to figure out who you’re with. Instead of manually tagging friends, or even using facial recognition through the Face.com technology it acquired, Facebook could put to work all the sensors in the phones of you and those around you:
That last capability, matching the audio recorded by the microphones of two users to identify that they’re right next to each other, was one of Color’s most exciting features. IDENTIFYING WHAT’S IMPORTANTSo Facebook knows who and what’s around around you. Then it wants to rank which frames of your video are the most interesting to you. To do that, it looks at your affinity to these objects (which friends, locations or brands you interact with most), and how popular they are to the public. It combines this data with extra audio and image cues of importance:
Honestly, the idea that an app could hear you say “This is beautiful” and know the photos and video you take at that time are important is sci-fi brilliant. CHOOSING YOUR BEST MOMENTSAt this point, Facebook can make well-educated guesses about what you want to share. It imagines splaying them out in what it calls a “media wheel” and showing the best frames as thumbnails when you share to the news feed:
Essentially, Facebook would rank all the frames of your video, show you the best ones, and you could select your favorites to represent your video when you share it. When people want to watch the video, they can click on one of the thumbnails, and instantly view the video starting 10 seconds before that frame. As quality of the still images ripped from video improve, Facebook could likely even allow you to share the frames as photos. ASSISTED SHARINGThese patents could redefine how we share. You wouldn’t need to search for people, locations, or things to tag. They’ll just be there waiting for your approval. That’s a big win on mobile where you want to share and get back to your life. Your content will contain so much structured data that Facebook could better route it to the people who’ll find it most interesting. And when we consume video, an opaque medium that’s classically tougher to skim than photos, there’ll be anchors and highlights pointing us to the most important moments. These could all encourage sharing and expose us to more enjoyable content. Facebook could even use colocation to create collaborative photo albums and carry on Color’s mission to let you “Take photos together”. There’s also huge implications for Facebook’s business. Likes are a terribly inaccurate graph of what businesses and places you care about. Today, recommendations of what to Like are scattershot, and it’s a chore to key them in. By recognizing businesses and brands in photos and videos, it can add them to its treasure trove of information about your preferences. That will help it fill out Graph Search, and target you with increasingly accurate ads and ecommerce opportunities. Imagine if Coca-Cola could target ads to people frequently seen with cans of Pepsi in their photos — high potential customers who specifically don’t like them yet. A restaurant could show push real-time ads to people shooting videos of a landmark next door. Facebook could open up ads APIs to surface characteristics like “currently with three or more friends” that a bar could take advantage of to advertise to nearby groups. The war for ad dollars will be won with data. Facebook already sees over 300 million photos uploaded each day. These patents could be a diamond drill, allowing Facebook to mine much more business information out of every piece of user generated content it imports. The world’s premier social network is 9 years old now. In some ways, that’s actually a disadvantage. People have forged connections with too many people and things they don’t actually care about. Facebook doesn’t know much about your offline life, or whether someone is a close friend or a distant acquaintance in that realm. That leads to a boring news feed — a huge danger to Facebook’s engagement-based business model. Facebook has spent years trying to get people to put in work to explicitly prune and classify their relationships with Friend Lists and news feed filters, but most still don’t. Implicit colocation and business identification could bring new richness and detail to its social graph, so your meatspace experience enhances your world of ones and zeros.
|
||||||||||||||||||||||||||||||||||||||||||||
|