SpaceTop

As augmented reality evolves, direct 3D interaction is emerging alongside traditional 2D interfaces, offering unique advantages for tasks like management, creativity, and collaboration. The aim is not to replace 2D UIs but to integrate 3D into desktop interfaces. SpaceTop merges 2D and 3D interactions within one workspace for seamless transitions and enhanced workflows across various tasks.

Bimanual, Multi-fidelity Interaction

Users can scroll through a main document with one hand while using the other to sift through a 3D visual pile of documents, searching for relevant text. They can then drag this text into the main document for precise placement. This setup allows users to effortlessly alternate between detailed, precise actions and broader, quick document browsing—or even do both simultaneously.

Hardware Design

The objective for the hardware design was to create something that looks as close to a conventional desktop computer as possible, so that users can feel familiar. Through multiple iterations, I ended up creating a structure with adjustable hinge that can move in the range of tilting angles within which most of the users can find a comfortable spot.

Sensing and Display

Users can interact with UIs floating on the screen and in 3D space, thanks to a transparent screen and dynamic perspective adjustment. A depth camera facing the user enables motion parallax by tracking head movements, while another camera monitors hand gestures and touch positions in the interaction space. I adapted lightweight algorithms for touch and pinch detection, originally developed by Andy Wilson, for smooth transitions between interaction modes.

Publications

Jinha Lee, Alex Olwal, Hiroshi Ishii, and Cati Boulanger. 2013. SpaceTop: Integrating 2D and Spatial 3D Interaction in See-through Desktop Environment. In ProceedingS of CHI 2013. ACM, New York, NY, USA.

Jinha Lee and Cati Boulanger. 2012. Direct, spatial, and dexterous interaction with see-through 3D desktop. In ACM SIGGRAPH 2012 Posters

Selected Press

WIRED - Amazing 3D Desktop was born at Microsoft

The Verge- SpaceTop 3D interface lets you reach inside your computer screen

Cnet- Microsoft's 3D computer offers a world for your hands

BGR - See-through 3D computer with gesture controls gives us a glimpse of the future

Mashable - Microsoft Demonstrates Mind-Blowing 3D Desktop