UIST '16

SketchingWithHands: 3D sketching handheld products with first-person hand posture

Yongkwan Kim

Seok-Hyung Bae


We present SketchingWithHands, a 3D sketching system that incorporates a hand-tracking sensor. The system enables product designers to easily capture desired hand postures from a first-person point of view at any time and to use the captured hand information to explore handheld product concepts by 3D sketching while keeping the proper scale and usage of the products. Based on the analysis of design practices and drawing skills in the art and design literature, we suggest novel ideas for efficiently acquiring hand postures (palm-pinning widget, front and center mirrors, responsive spangles), for quickly creating and easily adjusting sketch planes (modified tick-triggered, orientable and shiftable sketch planes), for appropriately starting 3D sketching products with hand information (hand skeleton, grip axis), and for practically increasing user throughput (intensifier, rough and precise erasers)—all of which are coherently and consistently integrated in our system. A user test by ten industrial design students and an in-depth discussion show that our system is both useful and usable in designing handheld products.

Keywords: Product design; hand tracking; 3D sketching

A 30-second preview

A 5-minute walkthrough

VR controller sketched in our system with contact visualization (first row) and a physical mockup made using 3D modelling and rapid prototyping based on the 3D point curve data from the concept model (second row). Note: all the figures represent the same product held with the same hand posture but are seen from different viewpoints.

Walking through an example handheld product (a smart hand drill) design (from the first row to the second row).

Example 3D sketch: a handheld noise detector

Example 3D sketch: a camcorder

Example 3D sketch: a dental curing light

Example 3D sketch: a robotic exoskeleton brace

Example 3D sketch: a drone controller

Example 3D sketch: a VR controller


Yongkwan Kim


Seok-Hyung Bae

Associate professor


🏆 Deputy Prime Minister of South Korea

BK21 Plus Excellent Researcher Award

Yongkwan Kim

Department of Industrial Design, KAIST
© SketchLab All Rights Reserved.