Intuitive Vibrotactile Feedback Educational Feedback Devices

Project team member(s): Niels van Galen Last, Zhan Fan Quek

The goal of the project is to come up with an affordable yet generic vibrotactile feedback device to be used for educational purposes. A chemistry learning environment is developed to demonstrate a specific application of our device, in which users can manipulate ions and feel the corresponding electromagnetic attraction and repulsion forces through the vibration feedback. The entire project went well and people who used our device agreed on its fun yet educational purpose.

Introduction

Many educational haptic feedback devices use kinesthetic force feedback devices such as the Novint Falcon and the Phantom Omni. Although these devices are relatively cheap compared to other high end haptic feedback devices, they are still expensive for educational purposes. Our team has therefore decided to develop an affordable vibrotactile haptic device that can be used as an education tool for augmenting teaching of various disciplines through the use of haptics. The device must be generic enough such that it can be used in many different educational situations.

Our team's intention to use vibrotactile feedback is motivated by the fact that vibrotactile feedback devices had been used in real-world applications such as medical catheter teleoperator, gaming controller, and mobile phones with considerable success. In addition, vibrotactile feedback devices are generally less complex and cheaper compared to kinesthetic feedback devices, which makes it suitable for use in the education field.

Background

Many haptics applications have been designed for teaching scientific concepts in the educational field. For example, [1] and [2] use the Phantom Omni haptic device together with detailed molecular simulation in order to convey chemical bonding information to the user. Similarly, [3] has used the Novint Falcon to allow user to the feel the molecules and their atomic interaction forces. [4] has used haptics to teach mathematical functions to people with vision disabilities, by allowing them to feel the curve and shape of the mathematical functions.

Many of the above educational haptics applications rely on commercial products such as the Novint Falcon and the Phantom Omni, Though this product are cheap compared to other high end haptic devices, they still cost from $300 for the Novint Falcon and $1000 for the Phantom Omni. Given that schools typically requires tens or more haptic devices, such high cost will prevent them from investing in using haptics as a tool for their education work.

Vibrotactile haptic feedback devices has been used in many industries with varying degrees of success. Vibration feedback has long been used in pagers and cell phones to inform user of impending phone calls, messages, or emails. In the gaming sector, vibration feedback has been used to add realism to the gaming experience [5], in which gamers feel vibrations whenever they are being attacked or when they bumped into a wall. Recently, vibration feedback had been used in medical catheter teleoperation systems to convey obstacle information to the surgeon. Given such rich uses of vibrotactile feedback in many other fields, it is a wonder why vibrotactile haptic feedback devices has not been more pervasive in the education field. Our literature study informs us that there are very few devices that uses vibration feedback for use in education or teaching. This therefore motivates us to come up with a cheap yet generic vibrotactile feedback device for use in educational settings.

Methods

For our overall design, our team has decided to come up with a vibration feedback device that is self contained, i.e., no external power input or wiring cable needs to be attached to the device. This is to allow complete freedom of motion when using our device. In order to realize this objective, the device will be battery powered, and that wireless communication will be used to communicate between the device and an external computer.

Mechanical design

Fig. 1: Solidworks drawing of the haptic sphere together with the electronics and battery pack.
Fig. 2: Silicone rubber pads used for isolating the vibration of the motor from the rest of the haptic sphere structure

Our team has decided to come up with a spherical shaped haptic device with finger indentations. This design allows us to standardize the manner in which user will handle the haptic device. In addition, vibration feedback can be provided to the user's individual fingers through the finger indentations. By placing the vibration feedback under the user's finger and isolating the vibration of the motor from the haptic sphere structure, user will be able to better discriminate the vibration between the different motors, which will allow complex patterning to be used in order to convey different information to the user. Such patterning will also expand the different type of information that can be conveyed to the user using limited number of vibration motors. With regards to the number of vibration motors, our team has decided to provide feedback to the index, middle, and ring finger, resulting in 3 vibration motors. The thumb and the baby finger are mainly used for providing a firm grip of the device and therefore we decided not to provide any vibration feedback information to these fingers.

To isolate the vibration of each individual vibration motor, our team has decided to mount the vibration motor onto a silicone gel pad before inserting them into the spherical structure. The silicone gel, due to its rubbery nature, damped out most of the vibration and therefore decreases the amount of vibration felt on the haptic sphere structure.

In addition, holes are cut out from within the haptic sphere in order to contain the electronics. The main components of the electronics are the Arduino Uno board, the 2-inch square PCB board, and the battery pack.

Electrical design

For our electrical design, we uses the arduino microcontroller to control the vibration motors as well as to sense input from the Inertial Measurement Unit (IMU), which contains a 3-axis accelerometer, a 3-axis gyroscopes, and a 3-axis magnetometer. The IMU allow us to sense the orientation of the device. MOSFETs are used to enable the microcontroller to switch on/off the vibration motor. An input switch is also incorporated in our design to allow user input. The bill of material for our electronics is shown in the table below:

A diagram of our electronic circuit is shown as follows:

Fig. 3: Electronics circuit schematic for the haptic sphere

Arduino code and Computer serial interface

For our device, the Arduino microcontroller acts as a communication medium between the computer and the IMU / vibration motors. The Arduino microcontroller is coded with instructions on the motor vibration pattern and frequencies when specific signals is received from the computer through the serial interface. In addition, the accelerometer, gyroscope, and magnetometer data are also read through the Arduino microcontroller and send to the computer through the serial interface. Such architecture allow us to minimize any calculations done on the microcontroller, as well as to perform complex calculations or render complex virtual environment using the computer.

Fig. 4: Diagram illustrating an overview of the entire system

Chemistry application demonstration

For our Haptic Chemistry application, our team has decided to use CHAI3D (www.chai3d.org), an Application Programming Interface (API) for use in haptic rendering. The API provide us with the basic functionality to display and render a virtual environment easily. In this chemistry application, our idea is to allow user to manipulate ions using our devices. Based on the interaction forces between the different ions, user will be able to feel the interaction forces through the different vibration patterns, as well as the frequency of vibration. A diagram illustrating our haptic chemistry virtual environment is shown in the figure below:

Fig. 5: Diagram illustrating the virtual environment for the haptic chemistry application

Results

Our team had managed to build the device and integrate the device with the CHAI3D virtual environment. Through the responses obtained from the Haptics Open House, we realized that some users had issues using the device orientation to move the cursor around in the virtual environment. These users mentioned that they still have a tendency to translate the device in order to the move the cursor. However, these users also mentioned that with sufficient training and instructions, they are able to navigate the environment sufficiently well.

From the Haptics Open House, users also mentioned that they are able to feel the change in vibration frequency, but is not able to distinguish the different vibration patterns. We hypothesize that this may be due to the way in which users grip the haptic sphere. Our team agreed that further improvement to the placement of the vibration motors can be made to allow users to better distinguish the different vibration patterns.

Future Work

Future work could include optimizing the location in which the vibration motors and damping pads should be located. User study can be performed to determine the optimum placement point for the vibration motors and damping pads so as to maximize the discrimination ability of users for different vibration patterns.

In addition, the current design of the haptic sphere contains an Arduino Uno module, which results in a large sphere design as space is needed to contain the Arduino Uno. The design can be improved further by replacing the Arduino Uno with a microcontroller. This will reduce the space needed to contain the electronics, which can result in a smaller and lighter haptic sphere.

The haptic chemistry virtual environment can also be improved further by allowing users to mix and combine compatible ions to form molecules. This will improve the realism of the simulation, as well as to allow student to learn advance chemistry knowledge other than electromagnetic attractions and repulsions.

Acknowledgments

We will like to thank Professor Allison Okamura for allowing us to use the CHARM Lab's 3D printer to print our haptic sphere. In addition, we will like to thank Sam Schorr for taking time to help to fabricate the 3D printed haptic sphere.

Files

Haptics Sphere Mechanical Design (Solidworks): Attach:HSMechanical.zip
Haptics Sphere Arduino Code: Attach:HapticSphere.txt
Serial Port Interface and Orientation Determination: Attach:Orientation.txt
Chemistry Application code (for use together with CHAI3D): Attach:Chem_App.txt
Haptics Sphere Components and cost: Attach:HSComponents.xls

References

[1] S. Comai and D. Mazza. A Haptic-Based Framework for Chemistry Education: Experiencing Molecular Interactions with Touch. Technology Enhanced Learning. Quality of Teaching and Educational Reform, pages 338- 344, 2010.

[2] Konrad H Marti and Markus Reiher. Haptic quantum chemistry. Journal of Computational Chemistry, 30(13):2010{2020, October 2009.

[3] R Andrew Davies, James S Maskery, and Nigel W John. Chemical education using feelable molecules. In Web3D '09: Proceedings of the 14th International Conference on 3D Web Technology. ACM Request Permissions, June 2009.

[4] F. Van Scoy, T. Kawai, M. Darrah, and C. Rash. Haptic display of mathematical functions for teaching mathematics to students with vision disabilities: design and proof of concept. Haptic Human-Computer Interaction, pages 31 - 40, 2001.

[5] D.H. Kim and S.Y. Ki. Immersive game with vibrotactile and thermal feedback. pages 903-906, 2010.

[6] J. Minogue and M.G. Jones. Haptics in education: exploring an untapped sensory modality. Review of Educational Research, 76(3):pages 317-348, 2006.

[7] W. Bareld. The Use of Haptic Display Technology in Education. Themes in Science and Technology Education, 2(1-2):pages 11-30, 2010.

Appendix: Project Costs

Appendix: Project Checkpoints

Checkpoint 1

// could be removed.. The purpose of the project is to come up with a vibrotactile haptic device that can be used as an education tool for augmenting teaching of various disciplines through the use of haptics. Various haptic devices have been used for education purposes. However, based on our literature search, many of the educational haptic devices uses kinesthetic force feedback haptic devices. There was very little devices that uses tactile feedback such as vibrotactile elements. Given that vibrotactile tactile feedback haptic devices had been successful in many other industries, such as medical, gaming, and mobile phones with considerable success, our team had decided to come up with a vibrotactile feedback haptic device for educational purposes.

Our design idea is come up with 2 sphere-shaped devices in which users can hold on to. There will be finger indentation on the sphere to mark the location where user should placed their fingers, and vibration motors will be placed at these finger locations. Orientation of the device will be sensed using an inertial measurement unit (combination of accelerometer, gyroscope, and magnetometer).

The checklist for the first checkpoint are as follows:
1) Determine method for orientation sensing & order part
2) Specify the electronics, determine battery requirements
3) Rough prototype that allows you to play with vibration feedback on a plastic ball
4) Mechanical design first draft complete (sketches + initial CAD)

For the first item in the checklist, we have ordered an IMU through ebay that contains a 3 axis accelerometer, 3 axis gyroscope, and a 3 axis magnetometer. Interface with the IMU has been done, and we are now able to determine the orientation of the IMU with relative accuracy.

For the second item in our checklist (the electronics), our team will require the following items:

The electronic circuit for our project is shown as follows:

A preliminary calculation was also performed to determine the feasibility of having the entire device being battery powered and wireless. The preliminary calculations are shown as follows:

For our fourth checklist, the initial CAD that we come up with is shown as follows:

The CAD consist of a sphere with flat bottoms to allow the device to rest on the table at the correct orientation and without problem of rolling around. Finger size indentation are made around the sphere to mark the locations in which user should place their fingers on. The electronics are housed within the sphere, consisting of the Arduino UNO board and a custom made PCB board containing the necessary electronics to actuate the motor and sense the device orientatino. A 9V battery holder is also incorporated into our design for easy swapping of batteries.

Checkpoint 2

The last two weeks we have made steady progress with our haptic ball. Our objectives for this week were:

1) First prototype (one ball) built and be able to turn on and off vibration motors
2) Read orientation data from sensor
3) Consult students from LDT program (or Paulo Blikstein)
At this moment we have completed number 1 and 2 of the checklist.

Figure 1. The first haptic ball.


Figure 2. Everything fits inside the ball.


Figure 3. Damping pads.

Making the haptic ball. The 3D printing of the ball took about 48 hours from start until we were able to hold it and start assembling the ball (Figure 1). Not everything fitted as smoothly as we expected so we had to sand the Arduino and XBee on the sides to fit them in the slots. The holes for the vibration motors were a little too small so we used the dremel to increase the diameter of the slots. We also found that the slot for our little finger was a bit misoriented. Figure 2 shows the ball with all electronics inside.

We applied a small current to the vibration motors inside the ball and not completely unexpected found that the entire ball was heavily vibrating. It was even impossible to tell which of the three motors was vibrating with your fingers placed directly on the motors. To solve this problem we decided to make the holes bigger and place the motors in small silicon pads to damp the vibrations, shown in Figure 3. The mold for the pads was created at the TLTL using a laser cutting machine.

The second edition (right handed) is being printed at the moment of writing, with several changes made to the design:

1) Removed the motor under the thumb, because there is not enough space for the wires to control it inside the ball.
2) Bigger slots for the pads and Arduino / XBee.
3) Smaller slot for the power switch.
4) Added some fillets to remove the sharp edges.
Once we have the complete prototype working we are going to talk to students in the LTD program, so that point will earn its checkmark next week.

Checkpoint 3

For the past week, we have built the second haptics ball based on the lesson learned from our previous design. Pictures of the 2 haptics balls are shown below:

With regards to the application, our team has decided to use our tool to teach about attraction and repulsion in atomic chemistry. Basically, users will be able to use our two haptics sphere to interact with a virtual environment containing ions of different elements. They will be able to feel the attraction / repulsion of the ions based on the vibration cues on the haptics ball. A preliminary look at our virtual environment is shown in the diagram below: