Not too long after the first rumors surfaced, Apple has given its usual non-confirmation that it has acquired Faceshift, the company behind the technology Star Wars used to animate the faces of CGI characters. It’s not an obvious fit for Apple, so what could be the thinking behind the purchase?
Like Apple’s patents, it is sometimes easy, I think, to read too much into some of the company’s acquisitions. Sure, it doesn’t go around acquiring companies randomly, but it may not always be after the complete package. It may well be that there is some small element of the company’s technology that Apple wants, or it may be an acquihire – where it’s the engineers rather than the specific tech the company wants.
But in this particular case, there is reason to suspect that Apple does have an interest in the broad brush-strokes of what Faceshift does …
That reason is that this is not a single acquisition in isolation. As we noted earlier, Apple bought PrimeSense back in 2013, the company behind the technology used in Microsoft’s Kinect sensor, and facial recognition company Polar Rose way back in 2010. So it’s clear that Apple has significant interest somewhere in the facial recognition/motion-sensing field, the question is: why? I can see six possibilities.
Some years back, we thought we had a pretty good idea. In the days when Apple was believed to be working on its own television – rather than just the Apple TV box – there were long-standing rumors that Apple was planning some kind of gesture-based UI. Perhaps we’d use a hand gesture to scroll through content and point to what we wanted, for example.
Apple reportedly abandoned its full TV set plans when it couldn’t see a way to make enough money from it. It was suggested at the time that there were two reasons for that: TVs typically have low margins, and people keep them for a long time, so the upgrade cycle is long.
I don’t necessarily buy the low-margin argument – you could say exactly the same about PCs or smartphones, both areas in which Apple still manages to make a lot of money. But I do think the lengthy upgrade cycle is likely to have given Apple pause: it’s something that has already impacted iPad sales.
But this doesn’t rule out TV use, of course. Apple does already make a television, it just asks customers to buy their own displays. The company has already introduced one new UI in the form of Siri, and there’s no reason to think that it isn’t working on other options for future models. So television remains one possibility.
Given Apple’s historic tie-ins with Pixar and Disney, it’s not beyond the bounds of possibility that it might want to create its own animated shows – where the Faceshift tech could obviously be used as-is.
Another possibility is gaming. While Apple may be unlikely to develop its own games, it could potentially offer a framework that would allow developers to incorporate Faceshift-style features into their own games. Imagine a game where your on-screen character mirrors your own facial expressions.
Couple this to Kinect-style motion-sensing, and you could have games where your character effectively becomes a virtual version of you, copying both your movements and expressions.
These are things that haven’t made too much sense for iOS games to date, where you are playing them on a portable device, but could make a great deal of sense now that Apple TV is aiming to bring mobile gaming into the living-room.
On the subject of entertainment, the tech could be a lot of fun in FaceTime chats with kids. Grandparents probably want to spend more time chatting with their grandchildren than vice-versa, but if the kids were looking at animated characters, their attention span might lengthen considerably.
Automatic tagging of people in photos and videos is a more mundane possibility – but one with enormous potential. Apple currently offers developers an API that can spot faces in photos and videos, but the code can’t yet identify who the people are.
This is an area where Google Photos is ahead of Apple, allowing you to search your photos for specific objects, and even particular people. Google’s tech is still pretty crude when it comes to recognizing people, though, so there is a definite opportunity for Apple to take the lead. The Mac Photos app currently has limited capacity to suggest similar faces, but it’s again very crude.
Imagine, for example, building facial recognition right into the Camera app in iOS, so that photos and videos are automatically tagged with the people in them. You’d then be able to search for those people directly on your iPhone, and the tags would of course carry over into the Photos app on your Mac so you could do the same there.
Perhaps give you the option of auto-including the tags when uploading photos to Facebook and other social media sites. This kind of stuff could be huge.
Finally, it could be that Apple simply wants to develop facial recognition tech good enough to use to unlock devices. Face-unlock has existed on Android for ages, but it’s way too crude to be trusted – it was originally fooled by photos and, when blinking was added, by a pair of photos. It has also been easily fooled by people who vaguely resemble the owner of the device, while failing to consistently recognize the real owner – the worst of both worlds.
Apple could be taking the same approach here as it did with fingerprint sensors, which first appeared on competitor phones many years ago. Apple waited until it could do it properly, with tech that is both fast and reliable.
If it could make face-recognition as reliable as Touch ID, that would be even more convenient: unlock your device just by picking it up (or opening the lid of a MacBook). Authorize iTunes purchases without doing a thing, simply because you’re already looking at the screen and it knows who you are. No need to hold your iPhone awkwardly when making Apple Pay purchases – so long as the camera can see your face, it authorizes them automatically.
Which possibilities appeal to you most? Take our poll to let us know, and share your thoughts – and any other ideas you may have – in the comments.
Photos: Main & FaceTime photos Faceshift; Pixar characters Pixar; Disney Infinity characters Disney; all other photos Apple.
