Google is testing a new AI green screen tool for YouTube called Mobile Real-time Video Segmentation. We’ve become used to every chat tool worth its salt offering AR overlays and masks, but this technology makes it possible to change backgrounds — in real time.
Google is testing a new AI green screen tool for YouTube called Mobile Real-time Video Segmentation. We’ve become used to every chat tool worth its salt offering AR overlays and masks, but this technology makes it possible to change backgrounds — in real time.
The technique makes use of neural networks to determine which parts of a scene are foreground to be left alone and which are background to be replaced. No special equipment is needed, and it has the potential to be used for serious videos as well as the more frivolous things masks tend to be used for at the moment.
Before you get too excited about this, it hard to say when — or, indeed, if — this is something that will makes its way to everyone any time soon. It’s currently only being made available to a limited number of YouTube creators. It’s still early days for the technology, and Google makes it sound as though it doesn’t currently cope well with fast-moving scenes.
Writing about the technology in a post on its research blog, Google says:
“Today, we are excited to bring precise, real-time, on-device mobile video segmentation to the YouTube app by integrating this technology into stories. Currently in limited beta, stories is YouTube’s new lightweight video format, designed specifically for YouTube creators. Our new segmentation technology allows creators to replace and modify the background, effortlessly increasing videos’ production value without specialized equipment.”
The company goes on to say: “Our immediate goal is to use the limited rollout in YouTube stories to test our technology on this first set of effects. As we improve and expand our segmentation technology to more labels, we plan to integrate it into Google’s broader Augmented Reality services.”
There’s a lot more information about how the technology works over on the Google Research blog, including details about the data the company used in training.
