Post-Break Update

There have been quite a few developments since Winter Break. First, on the administrative side, the proposal was submitted, including a draft of an abstract and title:

Using smartphones for gesture-based control of robotic systems
Robotic control systems are becoming more common, especially in the military. With military applications, there are lives at stake, so having the most efficient, intuitive control system can make a large difference in the success of a mission and the safety of the soldiers involved. Arm and hand gestures are typical human forms of communication, so applying that to a robotic control system can yield a more intuitive system. Varcholik et. al. describe a gesture based control system that uses the Nintendo Wiimote to determine arm/hand gestures to control a robot. In this thesis, I propose the use of smartphones for gesture-based control of robotic systems. The proposed controller will be evaluated by performing a set of carefully designed human factors experiments and computing a set of metrics (e.g. time taken to complete tasks) to measure the efficacy of the gesture-based control system.

The thesis committee will consist of Prof. Badler, Prof. Lane, and Prof. Daniilidis. I also took Winter Break as an opportunity to purchase a Roomba 560, RooTooth dongle, XBOX 360 controller, and a Samsung Galaxy II for use with the project.

Samsung Galaxy II

Roomba 560

Next, the robotics side! After several days of working through the Roomba Open Interface Specification, the Roomba finally accepts commands through the RooTooth Bluetooth dongle. It can now be controlled through the XBOX controller, keyboard arrow keys and command line, using scripts that I wrote with the RoombaSCI python library. I also downloaded the Cellbots Android app so there are several more phone-based direct control schemes that are working out of the box (see first part of the video).

Cellbots for Android. I plan to use the D-Pad and Tilt controls from Part 1.
Cellbots provides an API for connecting Android phones to robots like the Roomba via Bluetooth, so I will almost certainly be using it to write the gesture-based app. I will also look into using this framework, which provides gesture recording and recognition as an Android service.

Once the "hand waving" app is complete, I can start to develop experiments involving several control methods to determine which scheme people would rather use.