Gestural Computer Interface

Soon enough, we may be searching through data on our computer in the same way Tom Cruise searches through memories/video for evidence of a crime about to be committed in Minority Report. Since the film's 2002 release, many companies have tried to develop similar gestural computer interfaces in the real world. In fact, the science and technology adviser for the film, John Underkoffler, was given a significant amount of money to research the subject for the film, and recently debuted a prototype at TED talks this year. However, such interfaces require an entire room for setup, not to mention expensive screens, gloves and other accessories, rendering them impractical at the moment. Touch screen technology on iPhones and other devices allow for similar zoom and scroll capabilities, but they can be cumbersome and the technology has only been applied to simple apps.

But the future may be closer than we think. This year, at Music Hack Day Boston, a team of hackers created a gestural interface over the course of a weekend that can manipulate MIDI data. The program is called Toscanini, named after an Italian conductor famous for his wild and exuberant gestures. It only requires simple software that can be downloaded for free, and a Texas Instruments watch that sends signals to a receiver that attaches to your computer via USB. This second device only costs $50. While the program is in its alpha stage and has plenty of flaws, it is designed to improved upon and customized by other users. The intended users, in fact, are musicians, dancers, and other artists who will be able to have more intimate and personal control over their work on their computers through movement.

The potential for gestural computer interfaces cannot be overstated. In the next year, a dance troupe wearing the TI watch and Toscanini could create a visual program on the computer that could project a visual display behind them while they dance, responding to their actual movements. Musicians are no longer limited to clicking and dragging, no longer inhibited by the mouse as they work with Logic, ProTools and a myriad of other programs. Later on, it seems that Toscanini and more complex programs like Underkoffler's will meet somewhere in the middle, improving their ability to deal with complex programs while meeting practical needs and budgets in the real world. When that happens, users will be able to organize, render, and view data in the way that is easiest for them, i.e. scanning through photos, videos and music from the visual perspective that they choose -- and there will be an infinite number of ways to choose from. It will make computer use more action, like the Wii has done for video games, and will allow more than one person to work on the same computer at the same time, once they are no longer confined by the mouse. And the gestural computer interface is far from reaching its peak, as it is not widely available. But it is likely the future of human-computer interaction -- if you ask John Underkoffler, all computers will use such interfaces within 5 years.

Minority Report clip:
http://www.youtube.com/watch?v=NwVBzx0LMNQ

John Underkoffler @ TED talks:
http://www.ted.com/talks/john_underkoffler_drive_3d_data_with_a_gesture.html

The Toscanini website:
http://toscaniniinterfacing.com/