The augmented reality acquisition space is hot — Facebook, Snap, Apple and others are throwing money at teams and technologies that promise to increase user..
The augmented reality acquisition space is hot — Facebook, Snap, Apple and others are throwing money at teams and technologies that promise to increase user engagement. Spektral, a Danish startup, is the latest venture-backed visual effects company setting its sights on the massive space. Spektral is announcing a $2.8 million round today from Litecap and Amp Ventures to continue development of its machine learning-powered, real-time, green screen technology.
Spektral, unlike 99 percent of venture-backed startups doesn’ t have a product or at least not in the traditional tech sense. Instead of spending time on a go-to-market, Spektral has been investing heavily in research and development. After first pursuing still frames under the name CloudCutout, the team is moving into real time video — combining machine learning with spectral graph theory to separate people and objects from their original backgrounds and overlay them in a new stream.
It’s quite easy to imagine this technology being implemented into Snapchat or Messenger, but just because it’s obvious doesn’ t make it statistically likely. This is probably why Spektral is making the effort to show how its technology could be useful in other use cases, like production and advertising.
Other research groups have been exploring how machine learning could open up new design possibilities for separating objects from their backgrounds. Adobe, the Beckman Institute for Advanced Science and Technology and the University of Illinois at Urbana-Champaign worked together to publish research on automating that process. That said, video and pictures are two entirely different monsters.
Hair strokes have long been a key criteria for judging cut outs. Toke Jansen, founder and CTO of Spektral, explained to me just how easy it is to underestimate the difficulty of cutting around hair. Equipped with scissors, a human can cut around complex shapes without thinking about it. But even with the latest deep learning models trained on over a million images, machines struggle.
Spektral, as the name implies, is experimenting with spectral clustering for image segmentation within a video frame. This additional information can be added in as a prior to augment more traditional models. In the future, this technology could pave the way for more complex video editing. The team alluded to object manipulation, perhaps moving your friends hand wave with your own, as a natural next step.
To get the company and its technology to the next phase, it’s brining on a number of domain experts. Most notably, Danny Lange, the head of machine learning at Unity, is joining the startup’s Board of Directors. Lange was formerly the leader of machine learning efforts at Uber.
