Wearable Computing
Atlanta — Thad Starner has been wearing his computer since 1993. What began as a short-term experiment became a life-long project, some would even say a mission.
Starner is a guru of sorts when it comes to wearable computing. Several of his graduate students in the College of Computing at Georgia Tech also use wearable computers on a daily basis. For these researchers, however, a wearable computer is not just the newest gadget or high-tech status symbol, but a driving force in their computer research - how to build computer applications that make the user smarter and more efficient in completing day-to-day tasks.
"The exciting thing about wearable computers is the fact that they're with you everywhere and they have access to the same sort of sensory information that you do," says Starner. "The display in your eyeglasses might also integrate a camera so the computer can see as you see. If you use a headphone for listening to music or for cellular phone calls, that headphone could also incorporate a microphone, so the computer can hear as you hear. Suddenly, for the first time, our computers have the ability to see and hear the world from our perspective. Instead of being deaf, dumb, and blind sitting on our desks or in our pockets, our computers might be able to observe what we do all day, understand what is important to us, and act as a virtual assistant who helps us on a second-by-second basis."
Thad Starner
Starner envisions wearable computers and their applications progressing to the point where everyone has a virtual personal assistant, similar to a corporate executive's assistant. The computer would remind the user of appointments, automatically schedule meetings, and open relevant notes and documents as the user talks about different topics.
"I first decided I wanted a wearable computer back in 1989. I saw the interface that the Terminator had in the movie 'Terminator 2,'" said Starner. "I thought the ability to overlay computer graphics and text on the real world as you're interacting with the real world is really powerful, so I wanted to design that interface. I finally came up with a system that I could use in 1993. Since then I've found the system really useful for taking notes in class or taking notes during conversations. Basically, the system is designed to allow rapid storage and retrieval of information in any situation. Often you are not close to a desktop computer, and it is too inconvenient or abrupt to pull out a piece of paper and pencil. The machine has proved to be very successful in getting down notes on what I want to remember at the time that I need to remember it."
Thad Starner
Starner developed his first effective wearable computer in 1993, and that first experience was so compelling that he became convinced of the power of wearable computing. Over the years his wearable's hard disk, computer display, battery power, wireless access, and other features have changed as the technology has improved. Around 1995-96, he was using a 100 MHz 486 computer with 1-gigabyte hard disk - providing him with computing power and flexibility. Around this time, Starner expanded the use of his wearable from an advanced PDA (Personal Digital Assistant) to a replacement for his desktop computer. In fact, he wrote his 250-page Ph.D. thesis on his wearable computer, and currently, he writes most documents on his wearable. Today, his wearable computer with 266 Mhz Pentium MMX and 10-gigabyte hard disk is more powerful than his desktop.
Thad Starner
"Wearable computers are good for getting to pieces of information in two seconds," says Starner. "Anything longer than two seconds becomes an undue burden on the user, and he won't bother using it. Being able to get to your keyboard or screen so quickly that it's so easy to take down a memory or a quotation is incredibly valuable. Another thing wearable computers are very good for is when you need to use your computer while doing something else, for example, if I'm in the middle of a conversation. The point is to discuss information or to be social. The computer is there as an assistant. Similarly, if you're doing something like fixing a car, you can use the wearable computer to help you see which bolt needs to be tightened next or which test you need to perform. Basically you have a manual right in front of your eyes as you're working with the real-world device."
Thad Starner
"Wearables also allow a new means for communication. By combining a head-up display with services like an Instant Messenger, I can ask for a colleague's help while I'm in a group discussion at a conference or work site. This is a great advantage when I'm on a panel session and want to appear smarter than I am! Another trick is to use wearable computers for augmented reality. You can attach a hypertext link to a location or object. For example, an annotation at the copier might say 'Joe— I've already made the copies for the meeting — meet me downstairs.' Or, similarly, the link might be a video showing how to replace the toner cartridge," said Starner.
Limitations to Wearable Computing
Starner obviously feels passionate about the potential of wearable computers and feels the future is bright for their becoming mainstream. However, he does recognize their limitations.
"There are four problems in wearable computing - power, networking, privacy, and interface," says Starner. "Adding more features to a computer - a faster CPU, bigger disk, wireless network connection, etc. - requires more power, which in turn implies larger batteries and more weight. Along with this, heat is a problem. A system that consumes a lot of power in a small form factor concentrates a lot of heat in a small space and can result in uncomfortable temperatures. You have to design systems that take little power and little space and last a long time."
Starner's current wearable gets thirteen hours of power using two standard camcorder batters, a vast improvement over his earlier systems.
"Another problem is networking, off your body to the Internet and networking on the body between the computer's components. For example, I might have the computer in my belt or pocket, the display is on my eyeglasses, and the keyboard is in my pocket or integrated into my clothing on a belt. These different components need to talk to each other, so an on-body wireless bus (an internal electrical pathway along which signals are sent from one part of the computer to another) is an area of research. Getting that data from the body to the Internet is another point of research. You can never expect networking to be ubiquitous; there will always be places where access to the Internet won't be available."
Currently, Starner's wearable components are connected with cables, and today his hard disk is large enough that he stores frequently used Web sites on his hard drive to ensure that he can access them even if he doesn't have an Internet connection.
"Another problem is privacy. Wearable computers allow you to have access to information that you normally wouldn't have. You can record conversations if you want, and keep your personal notes, schedule, diary, medical record, and home movies on your wearable if you want. When you get that type of concentration of information in one place, that means somebody who got hold of that information would know a lot about you, say your medical records. So you have to use a combination of security measures like encryption, physical measures like keeping the machine on you, and guarding it like a purse to protect it. In many senses, wearable computers give you the ability to keep the information on your body; however, the sensitive issues are still there. You have to figure out how you're going to protect your privacy," says Starner.
"However, I find that the most interesting problem is interface. How do we communicate with the computer and how does it communicate to us?"
Thad Starner
Working Toward a Virtual Assistant
Starner's research group, the Contextual Computing Group, focuses on projects to develop applications and interfaces for the computer to be aware of what the user is doing and to assist the user as appropriate. Several current projects at the research stage are envisioned to work together to assist a user in routine tasks such as automatically scheduling an appointment, re-directing an urgent phone call appropriately based on the user's schedule and current activity, and recognizing that the user is engaged in conversation and would prefer to take the phone call later.
GPS Schedule Prediction
Daniel Ashbrook, a recent Tech graduate with a B.S. in computer science, developed an application using GPS (Global Positioning System) location data to predict a user's actions. A user carries a GPS receiver over a month or so, and the system develops an understanding of the user's travel routine such as commuting to campus, walking to class, returning to the lab, and returning home. From this information, the system can determine where the user is most likely to be going at any given time. Used in concert with other applications, the wearable computer can use this information to let callers know when the user is unavailable and may next be available. For example, the computer might predict that the user is going to be in class shortly and may tell the user to call back in an hour when class is over.
Mobile Capture System
Ph.D. student Kent Lyons developed the mobile capture system to capture the interactions of wearable computer users with their machines and the environment. The captured data (video and audio) can then be analyzed to see how users employ their computers as they interact with the real world. This data is helpful in improving computer interfaces and developing new applications.
Social Engagement Recognition System
A wearable computer-based intelligent agent should know when it is appropriate to interrupt a conversation. Such social gracefulness requires that the agent understand what the user is doing at a given time. Ph.D. student Brad Singletary is developing a computer vision system to add a perceptual layer for wearable computers that can detect when the wearer is engaged in face-to-face conversation. Understanding a wearer's social context allows the wearable to act in terms of the detected context. For instance, conversational distracters, such as cell phones, might be suppressed during engaged discussion. The featured system uses Hidden Markov Models (HMMs) to estimate what state the conversation is in through observation of conversants in the wearer's field of view. This type of system begins to recover the contextual information a wearable assistant needs to interact in the human world.
Ben Wong
Conversational Agent
Doctoral student Ben Wong's Conversational Agent application uses a microphone and voice recognition software on a wearable system to capture the words a user speaks during everyday conversations. This makes it easier for the user to control the wearable computer with few explicit commands. For example, if the user mentions a date or time while scheduling an appointment, the computer can immediately open a calendar to the appropriate place.
Assistive Technology
Some interesting assistive technology projects have grown out of Starner's wearable computing research.
Helene Brashear
Mobile Sign Language Translator
This project, led by Ph.D. student Helene Brashear, is an extension of Starner's earlier graduate work in American Sign Language applications. In these systems, a wearable computer system is outfitted with cameras to track users' hand movements as they sign in American Sign Language. The application then translates the signs into English text either on the user's computer display or on a cell phone to help them communicate with non-Sign users. In the future, this text may be synthesized into speech to help the user communicate with non-signers. Brashear is investigating various ways of making the system work in a variety of lighting conditions, including outside daylight. Currently, she is investigating stereoscopic cameras, infrared illumination, and structured laser light in her quest.
Gesture Pendant
A related assistive technology project is the Gesture Pendant, a wearable system that recognizes and translates simple hand gestures into commands for home appliances such as a stereo, thermostat, or lights. The Gesture Pendant is a light-weight, wireless camera system that is worn as a piece of jewelry and uses infrared light to track hand movements. In addition, this system tracks tremors in the hands, which may aid in the monitoring of health conditions such as Parkinson's disease, or may assist in watching for signs of overmedication.
Amy Hurst
Monitoring Tremor from Parkinson's Disease
An extension to the Gesture Pendant project is an application used on a pressure-sensitive computer tablet to monitor tremor in patients with Parkinson's disease. This project was developed with Dr. Michael Okun at Emory University. The application is based on common paper-based tests that physicians currently use to monitor patients' manual dexterity.
Undergraduate student Amy Hurst converted several of these traditional paper-based tests to a computer tablet-based one. One such test has the patient trace a spiral presented on the tablet's display with a stylus that is able to gather pressure data. Since Parkinson's patients exhibit a very specific tremor, the goal of the system is to be able to analyze the user's data and recognize if the patient exhibits a Parkinsonian tremor. The physician can then examine the results over time to see how the tremor and pressure of their spiral changes over time. Since the software does not require a human administrator, the test can be taken more frequently and outside of the doctor's office. Additionally, the system saves the physician's time and may provide more quantifiable data as to the degree of change in the patient's speed, tremor, and amount of pressure applied.
Thad Starner
The Future of Wearables
Starner is very enthusiastic about the real-world applications of projects such as the Gesture Pendant and the Parkinson's Tremor Monitoring system. He sees wearable computers becoming more common in the future and points to the popularity of MP3 music players and "computerized" cell phones such as the one used by NTT DoCoMo as examples of recent commercial successes. Starner predicts that MP3 players will continue to grow in functionality and may eventually merge into a general-purpose device worn on the body. Starner notes a couple of commercial wearable systems that have successfully been marketed to industry. One audio system helps workers quickly fill mail orders in a warehouse operation. Another system involves a scanner designed for workers to wear like a ring enabling them to easily track packages as they are moved.
Related Links
Other researchers at Georgia Tech are doing exciting research in wearable computing as well. More information can be found by visiting the following Web sites.
Contextual Computing Group
http://www.cc.gatech.edu/ccg/
Smart Shirt Commercialized
http://www.gatech.edu/news-room/archive/news_releases/sensatex.html
For more information contact:
Megan McRainey, Communications & Marketing
Contact Megan McRaineymegan.mcrainey@comm.gatech.edu
404-894-6016