Microsoft’s Fall Creators Update of Windows 10 includes a beta of an eye-tracking technology, called Eye Control. What could be a solid eye tracker falls short due to an apparent bug.
Microsoft’s Eye Control for the Windows 10 Fall Creators Update offers up a noble goal: controlling your PC with your eyes, using a feature called Eye Control in the Windows 10 Fall Creators Update. And it works, mostly.
Eye tracking has quietly emerged as an alternative to the mouse, led by companies like Tobii. The concept is simple enough: Sensors within an eye tracker bar you mount on your monitor „watch“ your gaze, guessing pretty accurately what you’re looking at. Though Tobii sees eye tracking as a tool for productivity and gaming, Microsoft currently sees it as a purely assistive feature, rather than a productivity tool.
We’ve reviewed two generations of eye-tracking hardware from Tobii—Eye Control was specifically designed for its latest eye tracker, the Tobii 4C —which Tobii positioned first as a productivity assistant, then as a gaming peripheral. However, Microsoft designed Eye Control as a virtual replacement for both a traditional mouse and keyboard, specifically for those with ALS (Amyotrophic Lateral Sclerosis) or other degenerative diseases.
To enable eye tracking currently, you need to own a Tobii 4C, which mounts beneath a desktop monitor. In real life, the LED light output is dialed down considerably.
Besides the traditional mouse, keyboard, and PC, Windows 10’s Fall Creators Update doesn’t require any specialized hardware. Eye Control does. You’ll need a Tobii Eye Control 4C to enable it. Eventually, Eye Control will support the Tobii Dynavox PCEye Mini, PCEyePlus, EyeMobile Plus, and I-series, according to Microsoft. (Tobii hardware is also being built into some gaming notebooks like the MSI GT72S G Tobii-805, eliminating the need for a mounting strip.) In addition to tracking your eyes, an eye tracker also serves as a way to enable Windows Hello on PCs that don’t already support it, like desktops.
Essentially, the 4C works the same way as the earlier Tobii EyeX: You mount the long, wand-shaped eye tracker on your monitor, plug it into an available USB port for power, and complete the short setup process.
Tobii’s eye tracker can quickly move your cursor around your screen, or alternatively can be used to switch between apps or virtual desktops.
Eye tracking is buried in Microsoft’s ease-of-use assistive settings.
Once Eye Control is turned on, a small box appears at the top of your screen, with four icons. The first, a pair of up and down arrows, simply relocates this navigational box to the top or bottom of your screen. The others—a keyboard, a dialog box, and some sort of a mouse/directional pointer—represent the Eye Control functions: a traditional keyboard, “shape writing,” and the mouse functionality.
Enable eye tracking within Windows, and you’ll see this overlay.
When I first tried out Eye Control, the app failed, and failed hard. Tobii’s own „Gaze Trace“ tool places a small bubble to represent your gaze as it moves about the screen. If you let your gaze linger on the „mouse“ icon on the Windows overlay, that function will launch. Windows, too, uses a circular target as a mouse „cursor“ of sorts.
Originally, when compared against Microsoft’s own Eye Control „mouse,“ I could measure the discrepancy between the two in inches. As it happened, a bug didn’t account for my multi-monitor setup and the differences in resolution. That, plus a driver update, resolved my issues.
(As our earlier testing shows, the size, shape, and color of your eye matters significantly; Tobii’s eye tracker simply didn’t interpret narrow eyes as accurately as wide-eyed people, and glasses affected accuracy, too. As a blue-eyed white male, however, my eyes were among the most accurately tracked of the ten different users we picked.)
As your gaze drifts across the screen in mouse mode, letting it linger over a spot will bring up the „target“ overlay for the mouse, below. Here, you can fine-tune your spot, then „click“ using one of the buttons. Once you’ve „clicked,“ you’ll need to bring your gaze up to the mouse icon again to once again enter mouse mode.
If your gaze lingers on any one spot for a second or two, the Windows’ assistive eye-tracking mouse function will pop up this overlay, which you can “fine tune” by clicking the arrow keys, then the mouse. Once an action has been taken, you’ll need to click the mouse or keyboard icon to begin again.
Text entry is performed via the keyboard icon, again within the overlay. Within the keyboard, “hovering” your gaze over a letter enters it on the screen, while a spacebar, Function, Control and a backspace key help you navigate about the screen. It’s a laborious process.
Microsoft’s assistive, eye-tracking keyboard.
What Microsoft has referred to as “shape writing” attempts to use prediction to facilitate the process, especially in a scenario where the user is using eye tracking to communicate in real time. Like a Windows phone keyboard, shape writing allows users to let their gaze drift over the letters of a word, only requiring them to linger or “dwell” on the first and last letters. Microsoft’s predictive technology then tries to guess the word. Here, Microsoft’s assistive technology is used to enter words into a text field that can be “played” or read aloud. Users also have the option to pick from a few preconfigured words (such as “hello,” or “please”) at the top of the screen.
Microsoft’s “shape writing” keyboard uses predictive technologies and text to speech to facilitate communication.
In promoting Microsoft CEO Satya Nadella’s new book, Hit Refresh, Nadella and his wife revealed that they have two special-needs children, one with cerebral palsy. Over the past few years, Microsoft has been instrumental in adding and promoting new modalities for interacting with PCs: touch, pen, voice, and, now, gaze. Though Microsoft designs its products for billions of consumers worldwide, Nadella’s public disclosures paint Microsoft’s admirable work in assistive technologies in a new light.
