OmniTouch

OmniTouch is a wearable computer, depth-sensing camera and projection system that enables interactive multitouch interfaces on everyday surface.

For example, the present shoulder-worn implementation allows users to manipulate interfaces projected onto the environment (e.g., walls, tables), held objects (e.g., notepads, books), and their own bodies (e.g., hands, lap).

On such surfaces - without any calibration - OmniTouch provides capabilities similar to that of a touchscreen: X and Y location in 2D interfaces and whether fingers are “clicked” or hovering.

This is approaching the accuracy of capacitive touchscreens, like those found in smart phones, but on arbitrary surfaces.

A central contribution of the work was a novel depth-driven, fuzzy template matching approach to finger tracking and click registration.