Jaime Moses

Mutating Pin Array

Project team member(s): Jaime Korman and Moses Swai

So far, the Mutating Pin Array consists of two small rectangular boxes called pins. Each pin can change color and can be propelled upwards or downwards within a range of force. The device can also detect the position of each pin at any time. The goal of the project was to display the dynamic capabilities of such a design. This goal was achieved by the successful implementation of a piano key program, which was the result of several interwoven haptic models.


The Mutating
Pin Array prototype.

Introduction

The Mutating Pin Array that was implemented only consisted of two "pins." While the resulting project simulated the compelling haptic experience of piano keys, it was just a window into what could be accomplished through the same underlying design principles. That is, a board with not just two pins, but thousands of thinner pins could be utilized to create many different and varying physical tools and haptic experiences. A desk for instance, made of such pins could render an entire piano, a functioning keyboard, or even a chess board.

Background

Our prototype was constructed with many pieces from the Stanford Hapkit. Namely, we used the Hapkit's 3-D printed motor pulley and its Arduino board. Such recycling of parts made for a much more streamlined implementation process, as few key parts needed to be designed from scratch.

Hardware design

The physical design of the Mutating Pin Array can be divided into three distinct categories: pin, actuator, and support. For more specific information, please refer to the attached Adobe Illustrator file.

Pin

Each pin is made of individually lasercut pieces of semi-transparent 1/8" acrylic. An RGB LED is fastened inside the pin box. A hole on the bottom face of the pin is used both to snuggly secure a small lasercut plank made out of 1/8" duron and to release the wiring of the LED. The LED wires and the plank, where the former is significantly longer than the latter, are glued to a thin strip of neoprene whereby the wiring is sandwiched between the other two materials.

Actuator

Each pin is actuated by a friction drive, so that a motor shaft rubs against the neoprene of the pin. To increase such friction, the motor shaft is fitted with a 3-D printed pulley which is also covered in a thin piece of neoprene. For position sensing, a magnet is glued to the end of the motor pulley, such that the magnet rotates with the pulley. This magnets rotation can be precisely measured by a proximity electromagnetic sensor. The entire actuation system is contained in boxes that are also lasercut out of 1/8" duron slabs.

Support

After the two actuators were created, they were both placed on a tray within a larger lasercut duron support box. The tray has holes for the pin plank to freely move up and down and has other holes for the LED wiring to loop back to the top of the tray. The wiring of each pin's LED, proximity electromagnetic sensor, and motor are connected to an Arduino board on the tray (there being two Arduino boards in all). The top of the support box is covered by an opaque white acrylic panel, with two square holes for the pins to freely move up and down. A hole is cut on a side of the box to retrieve USB cables that connected to the Arduino boards.

Software design

The software for the project is divided into two parts: One is programmed by the Arduino which controls the pin state, movement and the lighting of the LED. The second part is programmed in Processing which plays the sound and displays the note that has been played.

Arduino

This is the program that controls the functioning of the pin. The setup method initiates the Serial communication between the Arduino board and the computer. Then it sets the pulse width modulation frequency for the motor pin. It then defines the modes of the pins that would be used to connect the motor and the LED. It then initializes the motor and the position variables.

The program has a loop that runs after the setup is done. The loop is divided into two main parts. The first raises the pin to make it ready for use and the second part runs after the pin has been raised. These two parts are distinguished by a condition statement that checks the integer variable called "pinState" that is set to zero in the beginning to show that the pin is not raised yet. When pinState is zero it applies a force to the motor that pushes the pin up and switches the pinState to one to make the program run the second part of the loop. The program then sets the current position as the top position of the pin. When the user presses the pin down by one centimeter the program creates a wall by making the motor generate a force that is a function of the distance below the wall. This sets the pin to a pressed state that is determined by the integer variable pressState. This makes the program run a part of the loop that restore the pin to the original unpressed state by applying a force to it.

The program has methods that calculate the position of the rod and the force to be generated by the motor. The method to calculate the position uses the magneto-resistive sensor attached to the Arduino board which measures the rotation of the magnet attached to the motor rod. This method calculated the position in count and then has a calibration formula that converts the counts into centimeters. The method to calculate the force determined the direction of rotation of the motor depending on the sign on the variable called "force". The method also transforms the force magnitude into torques by multiplying it with the radius of the motor pulley. The torque is then converted into a duty cycle between 0 and 255. The output from the duty cycle is sent to the motor through the motor pin.

Processing

This is part of the software design receives the serial output from the Arduino board and plays the sound and displays the picture that describes the note. The software imports a class called soundcipher that generates the notes played. The setup of the program determines the serial port, the background and loads the image files. The program contains a SerialEvent method that is called whenever the computer receives a serial communication from the defined port (which is done always). The serial input then determines whether the note should be played. When the input is 1 the program plays a lower C note and displays an image of the letter C in the middle of the frame. If the input is zero the program displays the original background.

Functionality

The pin simulates a piano key. It starts by extending the key to a top position and then lets the user play a note. When the user presses the key to a certain depth the motor generates a force that resembles the end of a key press on a piano. The board then plays a music note on the computer and shows the user what note it was. The key is then restored to its top position by a force generated by the motor. The key has a rainbow LED inside it that changes color depending on the state of the key. When the key is ready to be used the LED lights a green color. When the key is pressed the LED lights a red color. The red light stays on until the key is restored to its top position where its witches back to green.

Most of the users at the "haptics open house" said that the key recoil was very compelling and resembled a real piano key. One big challenge was trying to find a code that would render that kind of kinesthetic feedback without causing a slip on the friction drive.

The device functionality could have been improved by using a voice coil motor as the actuator instead of a motor. This would have prevented friction drive slips that result to inaccurate position sensing. This improvement could also allow the user to feel a much bigger force feedback because the force on the friction drive is limited by the limiting friction of the neoprene used.

How it's used:

Files

References

http://arduino.cc/en/Reference/HomePage (Arduino code)
https://www.processing.org/reference/ (Processing code)
http://explodingart.com/soundcipher/tutorials.html (The sound library used in processing)