Download as PDF, PPTX
Uploaded byMark Billinghurst
COMP 4026 Lecture 6 Wearable Computing
The document discusses the evolution and specifications of wearable computing, highlighting major advancements and prototypes from the 1960s to present-day devices like Google Glass and smartwatches. It outlines key features of ideal wearables, their historical development, and various applications including augmented reality and collaboration tools. It also presents insights on user demographics and preferences for wearable devices, input methods, and design prototyping resources.
More Related Content
Similar to COMP 4026 Lecture 6 Wearable Computing
COMP 4026 Lecture 6 Wearable Computing
- 1. LECTURE 6: WEARABLE COMPUTING COMP4026 – Advanced HCI Semester 5 - 2016 Mark Billinghurst University of South Australia September 1st 2016
- 5. Major changes incomputing
- 8. Living Heads Upvs.Heads Down
- 9. What is aWearable Computer ? ▪ A computer that is: ▪ Portable while operational ▪ Enables hands-free/hands-limited use ▪ Able to get the user’s attention ▪ Is always on, acting on behalf of the user ▪ Able to sense the user’s current context Rhodes, B. J. (1997). The wearable remembrance agent: A system for augmented memory. Personal Technologies, 1(4), 218-224.
- 10. Wearable Computing ▪ Computeron the body that is: ▪ Always on ▪ Always accessible ▪ Always connected ▪ Other attributes ▪ Augmenting user actions ▪ Aware of user and surroundings
- 11. The Ideal Wearable ▪ Persists and Provides Constant Access: Designed for everyday and continuous user over a lifetime. ▪ Senses and Models Context: Models the users environment, mental state, it’s own state. ▪ Augments and Mediates: Information support for the user in both the physical and virtual realities. ▪ Interacts Seamlessly: Adapts its input and output modalities to those most appropriate at the time. Starner, T. E. (1999). Wearable computing and contextual awareness (Doctoral dissertation, Massachusetts Institute of Technology).
- 12. History of Wearables ▪ 1960-90: Early Exploration ▪ Custom build devices ▪ 1990 - 2000: Academic, Military Research ▪ MIT, CMU, Georgia Tech, EPFL, etc ▪ 1997: ISWC conference starts ▪ 1995 – 2005+: First Commercial Uses ▪ Niche industry applications, Military ▪ 2010 - : Second Wave of Wearables ▪ Consumer applications, Head Worn
- 13. Thorp and Shannon(1961) • Wearable timing device for roulette prediction • Audio feedback, four button input Ed Thorp
- 14. KeithTaft (1972) • Wearablecomputer for blackjack card counting • Toe input, LED in Glasses for feedback Belt computer Shoe Input Glasses Display
- 15. Steve Mann (1980s- ) http://wearcomp.org/
- 16.
- 17. Enabling Technologies (1989+) ▪ Private Eye Display (Reflection Technologies) ▪ 720 x 280 dipslay ▪ Red LED ▪ Vibrating mirror ▪ Twiddler (Handykey) ▪ Chording keypad ▪ Mouse emulation
- 18. MIT Tin Lizzy(1993) ▪ General Purpose Wearable ▪ Doug Platt, Thad Starner ▪ 150 MHz Pentium CPU ▪ 32-64 Mb RAM ▪ 6 Gb hard disk ▪ VGA display ▪ 2 PCMCIA slots ▪ Cellular modem http://www.media.mit.edu/wearables/lizzy/lizzy/index.html
- 19.
- 20. US Military Wearables(1989- ) ▪ Early experimentation ▪ 386 computer, VGA display ▪ GPS, mapping software ▪ Land Warrior (1991-) ▪ Integrated wearable system ▪ Camera, colour display, radio ▪ Navigation, reports, photos Zieniewicz, M. J., Johnson, D. C., Wong, C., & Flatt, J. D. (2002). The evolution of army wearable computers. IEEE Pervasive Computing, 1(4), 30-40.
- 21. Wearables at CMU(1991–2000) ▪ Industry focused wearables ▪ Maintenance, repair ▪ Custom designed interface ▪ Dial/button input ▪ Rapid prototyping approach ▪ Industrial designed, ergonomic http://www.cs.cmu.edu/afs/cs/project/vuman/www/frontpage.html
- 23. Early Commercial Systems ▪ Xybernaut (1996 - 2007) ▪ Belt worn, HMD, 200 MHz ▪ ViA (1996 – 2001) ▪ Belt worn, Audio Interface ▪ 700 MHz Crusoe • Symbol (1998 – 2006) • Wrist worn computer • Finger scanner
- 24. Prototype Applications ▪ RemembranceAgent ▪ Rhodes (97) ▪ Augmented Reality ▪ Feiner (97), Thomas (98) ▪ Remote Collaboration ▪ Garner (97), Kraut (96) • Maintenance • Feiner (93), Caudell (92) ▪ Factory Work ▪ Thompson (97)
- 25. Mobile AR: TouringMachine (1997) ▪ University of Columbia ▪ Feiner, MacIntyre, Höllerer, Webster ▪ Combines ▪ See through head mounted display ▪ GPS tracking ▪ Orientation sensor ▪ Backpack PC (custom) ▪ Tablet input Feiner, S., MacIntyre, B., Höllerer, T., & Webster, A. (1997). A touring machine: Prototyping 3D mobile augmented reality systems for exploring the urban environment. Personal Technologies, 1(4), 208-217.
- 26. MARS View ▪ Virtualtags overlaid on the real world ▪ “Information in place”
- 27. Backpack/Wearable Systems 1997 BackpackWearables ▪ Feiner’s Touring Machine ▪ AR Quake (Thomas) ▪ Tinmith (Piekarski) ▪ MCAR (Reitmayr) ▪ Bulky, HMD based Piekarski, W., & Thomas, B. (2002). ARQuake: the outdoor augmented reality gaming system. Communications of the ACM, 45(1), 36-38.
- 28. 2008: Location AwarePhones Nokia NavigatorMotorola Droid
- 29. 2009 - Layar(www.layar.com) • Location based data – GPS + compass location – Map + camera view • AR Layers on real world – Customized data – Audio, 3D, 2D content • Easy authoring • Android, iPhone
- 30.
- 31. Second Gen. Systems • Recon(2010 - ) • Head worn displays for sports • Ski goggle display • Investment from Intel (2013) • Google (2011 - ) • Google Glass • Consumer focus
- 32.
- 33.
- 35. • Hardware • CPU TI OMAP4430 – 1 Ghz • 16 GB SanDisk Flash,1 GB Ram • 570mAh Battery • Input • 5 mp camera, 720p recording, microphone • GPS, InvenSense MPU-9150 inertial sensor • Output • Bone conducting speaker • 640x360 micro-projector display
- 36. ViewThrough Google Glass Alwaysavailable peripheral information display Combining computing, communications and content capture
- 38.
- 39. Smart Watches • EgMotorola Moto 360, Apple Watch • Measure heart rate, movement • Provide notifications, runs apps, messages, etc
- 40. Moto 360 SmartWatch https://www.youtube.com/watch?v=Q0prQniEZTA
- 41. Activity Monitor • Fitbit, MicrosoftBand • Measures footsteps, heart rate, calories, sleep
- 42.
- 43.
- 44. Number of DevicesShipped
- 45. Summary Wearables are anew class of computing Intimate, persistent, aware, accessible, connected Evolution over 50 year history Backpack to head worn Custom developed to consumer ready device Enables new applications Collaboration, memory, AR, industry, etc Many head worn wearables are coming Android based, sensor package, micro-display
- 46.
- 47.
- 48. Some Key Aspects ▪ Display Technologies ▪ Input Devices ▪ Interaction Metaphors ▪ Perceptual Factors ▪ Ergonomics ▪ Cognitive Aspects
- 49. Types of HeadMounted Displays Occluded See-thru Multiplexed
- 50. Multiplexed Displays ▪ Aboveor below line of sight ▪ Strengths ▪ User has unobstructed view of real world ▪ Simple optics/cheap ▪ Weaknesses ▪ Direct information overlay difficult • Display/camera offset from eyeline ▪ Wide FOV difficult
- 51. Vuzix M-100 ▪ Monocularmultiplexed display ($1000) ■ 852 x 480 LCD display, 15 deg. FOV ■ 5 MP camera, HD video ■ GPS, gyro, accelerometer
- 52. Display Types ▪ CurvedMirror ▪ off-axis projection ▪ curved mirrors in front of eye ▪ high distortion, small eye-box ▪ Waveguide ▪ use internal reflection ▪ unobstructed view of world ▪ large eye-box
- 53. See-through thin displays ▪ Waveguide techniques for thin see-through displays ▪ Wider FOV, enable AR applications ▪ Social acceptability Opinvent Ora
- 54. Waveguide Methods See: http://optinvent.com/HUD-HMD-benchmark#benchmarkTable Holographic Hologramdiffracts light Limited FOV Colour bleeding Diffractive Slanted gratings Total internal reflection Costly, small FOV
- 55. Waveguide Methods See: http://optinvent.com/HUD-HMD-benchmark#benchmarkTable Clear-VuReflective Several reflective elements Thinner light guide Large FOV, eye-box Reflective Simple reflective elements Lower cost Size is function of FOV
- 56. Input Options ▪ PhysicalDevices ▪ Keyboard ▪ Pointer ▪ Stylus ▪ Natural Input ▪ Speech ▪ Gesture ▪ Other ▪ Physiological
- 57. Twiddler Input ▪ Chordingor multi-tap input ▪ Possible to achieve 40 - 60 wpm after 30+ hours ▪ Chording input about 50% faster than multi-tap ▪ cf 20 wpm on T9, or 60+ wpm for QWERTY Lyons, K., Starner, T., Plaisted, D., Fusia, J., Lyons, A., Drew, A., & Looney, E. W. (2004, April). Twiddler typing: One-handed chording text entry for mobile phones. In Proceedings of the SIGCHI conference on Human factors in computing systems (pp. 671-678). ACM.
- 58. Virtual Keyboards ▪ Inair text input ▪ Virtual QWERTY keyboard up to 20 wpm - On real keyboard around 45-60+ wpm ▪ Word Gesture up to 28 wpm - On tablet/phone Word Gesture up to 47 wpm ▪ Handwriting around 20-30 wpm A. Markussen, et. al. Vulture: A Mid-Air Word-Gesture Keyboard (CHI 2014)
- 59. Unobtrusive Input Devices ▪ GestureWrist ▪ Capacitive sensing ▪ Change signal depending on hand shape Rekimoto, J. (2001). Gesturewrist and gesturepad: Unobtrusive wearable interaction devices. In Wearable Computers, 2001. Proceedings. Fifth International Symposium on (pp. 21-27). IEEE.
- 60. Unobtrusive Input Devices ▪ GesturePad ▪ Capacitive multilayered touchpads ▪ Supports interactive clothing
- 61. Interaction on theGo ▪ Fitt’s law still applies while interacting on the go ▪ Eg: Tapping while walking reduces speed by > 35% ▪ Increased errors while walking Lin, M., Goldman, R., Price, K. J., Sears, A., & Jacko, J. (2007). How do people tap when walking? An empirical investigation of nomadic data entry.International Journal of Human-Computer Studies, 65(9), 759-769.
- 62. Where to putWearables? ▪ Places for unobtrusive wearable technology Gemperle, F., Kasabach, C., Stivoric, J., Bauer, M., & Martin, R. (1998, October). Design for wearability. In Wearable Computers, 1998. Digest of Papers. Second International Symposium on (pp. 116-122). IEEE.
- 63. Where to PlaceTrackpad? ▪ User study 25 people different postures ▪ Front of thigh most preferred, torso/upper arm worst Thomas, Bruce, et al. "Determination of placement of a body-attached mouse as a pointing input device for wearable computers." 2012 16th International Symposium on Wearable Computers. IEEE Computer Society, 1999.
- 64. Where do users want Wearables? 29%on clothing 28% on wrist 12% on Glasses
- 65.
- 66. How do youDesign for this?
- 67. Typical Development Steps ▪ Sketching ▪ Storyboards ▪ UI Mockups ▪ Interaction Flows ▪ Video Prototypes ▪ Interactive Prototypes ▪ Final Native Application Increased Fidelity & Interactivity
- 68. Sketched Interfaces ▪ Sketch+ Powerpoint/Photoshop/Illustrator
- 69. GlassSim – http://glasssim.com/ ▪ Simulate the view through Google Glass ▪ Multiple card templates
- 70. GlassSim Card Builder ▪ Use HTML for card details ▪ Multiple templates ▪ Change background ▪ Own image ▪ Camera view
- 71.
- 72. Glass UI Templates ▪ Google Glass Photoshop Templates ▪ http://glass-ui.com/ ▪ http://dsky9.com/glassfaq/the-google-glass-psd-template/
- 73.
- 74. ToolKit for Designers ▪ Vectoform Google Glass Toolkit for Designers ▪ http://blog.vectorform.com/2013/09/16/google-glass- toolkit-for-designers-2/ ▪ Sample cards, app flows, icons, etc
- 75. Glassware Flow Designer • Visual design tool for Glass interaction flows • Collaboratively design apps using common patterns and layouts. • https://glassware-flow-designer.appspot.com/
- 76.
- 77. ▪ Series of stillphotos in a movie format. ▪ Demonstrates the experience of the product ▪ Discover where concept needs fleshing out. ▪ Communicate experience and interface ▪ You can use whatever tools, from Flash to iMovie. Video Sketching
- 78. See https://vine.co/v/bgIaLHIpFTB Example: VideoSketch of Vine UI
- 79. Interactive Wireframing ▪ Developinginteractive interfaces/wireframes ▪ Transitions, user feedback, interface design ▪ Web based tools ▪ UXpin - http://www.uxpin.com/ ▪ proto.io - http://www.proto.io/ ▪ Native tools ▪ Justinmind - http://www.justinmind.com/ ▪ Axure - http://www.axure.com/
- 80. UXpin - www.uxpin.com ▪ Web based wireframing tool ▪ Mobile/Desktop applications ▪ Glass templates, run in browser https://www.youtube.com/watch?v=0XtS5YP8HcM
- 81.
- 82. Viewing on Glass • Use Android Design Preview • Push desktop onto Android screen • https://github.com/romannurik/AndroidDesignPreview
- 83. Android Design PreviewDemo • https://www.youtube.com/watch?v=WvQrP1szEzg
- 84. Processing and Glass ▪ One of the easiest ways to build rich interactive wearable applications ▪ focus on interactivity, not coding ▪ Collects all sensor input ▪ camera, accelerometer, touch ▪ Can build native Android .apk files ▪ Side load onto Glass
- 85. HelloWorld //called initially atthe start of the Processing sketch! void setup() {! size(640, 360);! background(0);! } ! ! //called every frame to draw output! void draw() {! background(0);! //draw a white text string showing Hello World! fill(255);! text("Hello World", 50, 50);! }!
- 86.
- 87. HelloWorld Image PImage img;// Create an image variable! ! void setup() {! size(640, 360);! //load the ok glass home screen image! img = loadImage("okGlass.jpg"); // Load the image into the program ! }! ! void draw() {! // Displays the image at its actual size at point (0,0)! image(img, 0, 0);! }!
- 88.
- 89. Touch Pad Input • Tap recognized as DPAD input !void keyPressed() {! !if (key == CODED){! ! !if (keyCode == DPAD) {! !// Do something ..! • Java code to capture rich motion events • import android.view.MotionEvent;!
- 90. Motion Event //Glass TouchEvents - reads from touch pad! public boolean dispatchGenericMotionEvent(MotionEvent event) {! float x = event.getX(); // get x/y coords ! float y = event.getY();! int action = event.getActionMasked(); // get code for action! ! switch (action) { // let us know which action code shows up! !case MotionEvent.ACTION_DOWN:! ! !touchEvent = "DOWN";! ! !fingerTouch = 1;! !break; ! !case MotionEvent.ACTION_MOVE:! ! !touchEvent = "MOVE";! ! !xpos = myScreenWidth-x*touchPadScaleX;! ! !ypos = y*touchPadScaleY;! !break;!
- 91.
- 92. Sensors • Ketai Libraryfor Processing • https://code.google.com/p/ketai/ • Support all phone sensors • GPS, Compass, Light, Camera, etc • Include Ketai Library • import ketai.sensors.*;! • KetaiSensor sensor;!
- 93. Using Sensors • Setupin Setup( ) function • sensor = new KetaiSensor(this);! • sensor.start();! • sensor.list(); • Event based sensor reading void onAccelerometerEvent(…)! {! accelerometer.set(x, y, z);! }!
- 94.
- 95. Sensors ▪ Ketai Libraryfor Processing ▪ https://code.google.com/p/ketai/ ▪ Support all phone sensors ▪ GPS, Compass, Light, Camera, etc ▪ Include Ketai Library ▪ import ketai.sensors.*; ▪ KetaiSensor sensor;
- 96. Using the Camera ▪ Import camera library ▪ import ketai.camera.*; ▪ KetaiCamera cam; ▪ Setup in Setup( ) function ▪ cam = new KetaiCamera(this, 640, 480, 15); ▪ Draw camera image void draw() { //draw the camera image image(cam, width/2, height/2); }
- 97.
- 98.
- 99. Online Wearables Exhibit Onlineat http://wcc.gatech.edu/exhibition
- 100. Glass Resources ▪ MainDeveloper Website ▪ https://developers.google.com/glass/ ▪ Glass Apps Developer Site ▪ http://glass-apps.org/glass-developer ▪ Google Design Guidelines Site ▪ https://developers.google.com/glass/design/ index?utm_source=tuicool ▪ Google Glass Emulator ▪ http://glass-apps.org/google-glass-emulator
- 101. Other Resources ▪ ARfor Glass Website ▪ http://www.arforglass.org/ ▪ Vandrico Database of wearable devices ▪ http://vandrico.com/database
- 102. Books ▪ Programming GoogleGlass ▪ Eric Redmond ▪ Rapid Android Development: Build Rich, Sensor-Based Applications with Processing ▪ Daniel Sauter
- 103.