Samsung Smart TV

Bimanual, Multi-fidelity Interaction

Users can scroll through a main document with one hand while using the other to sift through a 3D visual pile of documents, searching for relevant text. They can then drag this text into the main document for precise placement. This setup allows users to effortlessly alternate between detailed, precise actions and broader, quick document browsing—or even do both simultaneously.

Hardware Design

The objective for the hardware design was to create something that looks as close to a conventional desktop computer as possible, so that users can feel familiar. Through multiple iterations, I ended up creating a structure with adjustable hinge that can move in the range of tilting angles within which most of the users can find a comfortable spot.

Sensing and Display

Users can interact with UIs floating on the screen and in 3D space, thanks to a transparent screen and dynamic perspective adjustment. A depth camera facing the user enables motion parallax by tracking head movements, while another camera monitors hand gestures and touch positions in the interaction space. I adapted lightweight algorithms for touch and pinch detection, originally developed by Andy Wilson, for smooth transitions between interaction modes.


US PATENT 20140098085, Jinha Lee, Cati Boulanger, Steven Bathiche, Transparent Display


Jinha Lee, Alex Olwal, Hiroshi Ishii, and Cati Boulanger. 2013. SpaceTop: Integrating 2D and Spatial 3D Interaction in See-through Desktop Environment. In ProceedingS of CHI 2013. ACM, New York, NY, USA.

Jinha Lee and Cati Boulanger. 2012. Direct, spatial, and dexterous interaction with see-through 3D desktop. In ACM SIGGRAPH 2012 Posters

Selected Press

WIRED - Amazing 3D Desktop was born at Microsoft

The Verge- SpaceTop 3D interface lets you reach inside your computer screen

Cnet- Microsoft's 3D computer offers a world for your hands

BGR - See-through 3D computer with gesture controls gives us a glimpse of the future

Mashable - Microsoft Demonstrates Mind-Blowing 3D Desktop

Wired UK - Microsoft demos Kinect-powered transparent 3D desktop