A view into your desktop
Adding eyes to artificial intelligence is always a tricky thing. Do you want it to see everything you’re doing all the time? Certainly not, but I think most of us agree that an AI visual assist when you need it could come in quite handy. Microsoft’s new Copilot Vision may be one of the most promising applications of AI-based visual capabilities I’ve seen yet.
Microsoft unveiled the Copilot Vision update for its Windows App and mobile apps (you can point your camera at things, and Vision can identify them for you) during a splashy, combined Copilot and Microsoft 50th Anniversary event.
Copilot all but got a brain transplant, using both homegrown (Microsoft AI or MAI) and OpenAI GPT generative models to deliver updates across memory, search, personalization, and vision capabilities.
Now that I’ve seen Copilot Vision in action, I can tell you it’s one of the most exciting and important updates of the bunch – even if it is coming in two stages.
In the version you can access for your supporting Windows Desktop app right now, Copilot Vision can see the apps you’re running on the desktop. When you open Copilot – by selecting the icon or pressing your Copilot key on your keyboard – you can now select the new eyeglasses icon.
This lets you see a list of open apps; in our case, we had two running: Blender 3D and Clipchamp.
