iLearn: The App that Learns

Rahul Rajan, Ted Selker

Considerate Systems Group, Fall 2010

Abstract: In this paper, we describe iLearn, an application that embodies the Program-by-Demonstration paradigm on the Android platform. The objective here is to be able to "demonstrate" an activity to the phone that attempts to learn it, and recognize every subsequent occurrence of this particular activity. The paper will begin with the steps involved in feature engineering the sensor data from the microphone and the accelerometer. It will then cover how a sequence of actions can be modeled using HMMs, including state discovery. It also discusses how employing user feedback in the learning process can provide for a more provocative user experience.