THE MOZART GLOVE

Project team members: Laurin Schwab and Collin Christner

We wanted to create a simulation of a piano - without the piano. The result? A left-hand glove studded with force-sensitive resistors and vibration motors that can essentially re-create the experience of piano-playing on any surface. More specifically, our goal was to have The Mozart Glove function by outputting the sounds of an A minor scale based on user force input (e.g. tapping the pinky finger softly generates an "A" while pressing harder generates a "B"). We were successful with all of our goals, and it ended up working as intended; the Mozart Glove (as of version 3.2) plays an entire 8-note scale using 4 fingers, outputting low-frequency vibrations for individual notes and high-frequency vibrations for tritones (a B and an F played simultaneously). All haptic sensations and audio output work on command. In the future, this project could be expanded to two gloves, and modified to return haptic sensations more similar to a real piano. A streamlined version, sold commercially, could significantly reduce the costs of early piano learning by providing a cheaper, avant-garde alternative to the purchase of an otherwise exorbitant instrument.

Introduction

Our motivation was to create a light, portable, and ergonomic device that could simulate the sensation of piano-playing sans the piano. A haptic device was necessary because only haptics provides the tools that enable such a simulation. Because piano playing is an inherently touch-based activity (e.g. you have to consistently apply a certain force, and you know you've correctly done so when you feel the string's vibration), touch-based technology was required. While the force-sensitive resistors simulated the act of pressing the keys, the motors simulated the vibration of the strings.

Background

There is much relevant work in this field. For example, there are commercially sold piano gloves that simulate piano playing based on force input, such as the one electronic "Thanko" Piano Glove by OEM dimensions (demonstrated in the link in reference number 3). Using two gloves, the user can play a major scale by pressing the fingertips on a surface. This design differs from ours, however, in that the notes do not change based on force input; rather there is one note per finger.

Design

Hardware design

Items needed:

• 1 golfer glove
• 4 vibration motors
• 4 small foam rectangular prisms (for vibration motors)
• 4 FSR sensors
• 4 small foam circles (on the shape of the FSRs)
• 1 bread board
• 1 Hapkit Arduino (used from the Hapkit - see reference number 2 to the Hapkit page)
• 4 resistors
• 4 short wires

First, we attached 4 small foam pieces to the tops of the fingers using hot glue. Next, we glued the 4 vibration motors onto those pieces (the metal part touching the foam) to elevate them so there was no interference from the fabric when they vibrated. As for the FSRs, we cut out small foam circles to glue onto the fingertips before supergluing the FSRs themselves. This allowed the user to get a sense for the surface area of the FSR using the foam when pressing on a surface. We used superglue as opposed to hot glue here because the hot glue would interfere with the haptic sensation.

Before attaching the FSRs, it is important to note that we soldered the ends onto wires, which we then snaked down the glove and tied with a twisty tie to allow the user to move more freely away from the Arduino board and circuitry. We also soldered the very ends of the wires to facilitate their connection into the bread board, since the wires kept popping out.

As for the circuitry, each FSR was connected to an individual circuit that fed into its own analog pin. Each motor was hooked up in an individual circuit to a digital output pin.

Software design

We used two programs: Arduino and Processing. The Arduino board prints the force values from each finger to the serial monitor, which is read by Processing. Processing determines which finger is being pressed, and acts similarly to the Arduino code. If the value is above zero for a certain finger, it plays a note associated with that finger (the exact note is determined by the degree of force that the user uses).

Functionality

The Mozart Glove senses forces given by the user on the fingertips, and using input from those sensors, outputs a sensation and an audio clip.

To play the Mozart Glove, the user presses with any or all of his fingers. Any finger that has been pressed will receive a short vibration, signaling to the user haptically that they have pressed a "key". The degree with which the user presses determines the note that is played. It works as we intended it, with some limitations that we expected.

HOW DOES IT WORK?

Essentially, each finger operates on a binary system. If the force value for that finger is above 0, then the system knows it is being pressed, and outputs the vibration. Three of the fingers- the pinky, ring, and middle- have two notes programmed into them, that is, users may play different notes based on the degree of force that they use. In that case, the computer also checks for the force value having passed a certain threshold, and outputs a second vibration.

It is important to note that the force value thresholds are not the same for each finger. Since some fingers are inherently weaker than others, we made sure to tweak the force value to better fit the natural anatomy of the hand. The pinky, for example, requires less force input than, say, the index finger.

LIMITATIONS OF THE GLOVE as of v.3.2

Each finger (except the index) allows you to play two notes. But to play the higher of the two notes on each finger, you have to jump from low note to higher note; you can't go straight to the higher one. During prototyping we were unable to implement a solution. Another limitation is that while the device outputs a vibration, it's not necessarily an accurate haptic sensation in terms of the way a piano truly feels. If we were to continue the project, we would try to create a better simulation of the feeling of pressing a piano key.

The idea of expanding the glove to a device used for early piano learning is also problematic because the position of the hand, as of version 3.2, does not accurately simulate the position of a piano hand. The fingers of a pianist are more bent at the knuckles, so that the pianist may apply more force and achieve a crisper note. What with the way the FSRs are positioned on The Mozart Glove, however, only a flat-handed position can work the device.

This is Laurin demonstrating the range of the Haptic glove, as well as the A-minor chord.

Acknowledgments

Big thanks to Allison and Tania for all their help acquiring parts and helping us figure out different functionality issues. Shoutout to Alex in the PRL who was very patient with our inability to work Illustrator and the laser cutter. Also thanks to Derin and Vivian for giving a few tips on how Processing works.

Files

Files to our code are attached here. These are for educational use only.

• Attach:HapticGlove3_2.ino.zip

Our Arduino code (handling the haptic sensations).

• Attach:HapticGlove3_2.pde.zip

Our Processing code (handling the audio component)

References

Used this code as a starter package for Processing. http://arduino.cc/en/tutorial/Graph

The Hapkit project. http://hapkit.stanford.edu

The Electronic Glove as sold on Amazon: http://www.amazon.com/Hiyadeal-Electronic-Exercise-Instrument-Keyboard/dp/B00ATA1YCY/ref=pd_sim_sbs_misc_5