RAMESASI Hockey!

Project team member(s): Veronica Sanchez, Brandon Racca, Esteban Mejia, Thomas Froger Silva

We are creating an air hockey simulator with haptic and visual feedback. We used the graphkit design from the Charm Lab to make the pantographs for our virtual hockey game controllers. We will implement controls and force feedback to simulate the response to a collision between a puck and the handle and to activate the different special features players can use against their opponent. in this first iteration of our project we created a one - player version of the game due to last minute hardware issues. Our project was demonstrated during the class' haptics open house and tested by class members and other visitors.

Introduction

The gaming industry is expanding: as eSports becomes more popular and the virtual/augmented reality spaces become more developed, the desire to have force feedback in the user experience will be in increasingly high demand. We have chosen this project because entertainment and gaming are popular areas where haptic feedback is implemented, and the opportunity presented by virtual and augmented reality could expand haptic use cases in these industries. Our project takes a popular game that you find in an arcade and allows the virtual version of this game to be more representative of the real game without needing the physical space for it. Air hockey is a beloved pastime for kids and adults alike. Having been a staple in the arcade world since the 70s, the air hockey table is recognizable immediately. However, we all know how bulky these tables are. What if we could bring that experience to your kitchen table? This is what our project is meant to explore.

While we set out to use two devices to create a head-to-head challenge between two human players, because of electronic hardware issues and time constraints, we were unable to fully implement the envisioned system. We then had to pivot to a intermediate game that tests the functionality of a single player in air hockey and their interactions with the puck. This game also implements a power up feature. The software for the game has been specifically written so that in the future, more players and devices can be added easily, given functioning hardware.

Background

In this section we will look at other devices that involve games with haptic feedback. Each of these devices serve different purposes, like entertainment and education (or both) and present interesting features that could potentially be used in our device.

Similar to our device, Haptic Pong is a haptic device that allows two players to face off on a virtual game using haptic feedback.(2) Contrary to our device, this device only has one degree of freedom for both players. Using a pantograph will allow us to render a similar game as Haptic Pong but with two degrees of freedom for both players. This device was conceived mainly for educational purposes to teach physics and controls concepts. Our device could also be used for education, allowing students to render different haptic feedback using power ups in a larger workspace than the Haptic Pong device.

Another haptic device described in Visuo-Haptic Collaborative Augmented Reality Ping-Pong also simulates a game with haptic feedback.(3) This device also renders graphics in 3D using virtual reality. The main feedback response from this device is when the ball hits the virtual ping pong racket. This possibility of combining virtual reality and haptics could go a long way for the gaming industry. However this device has its limitation. Its high cost could make it inaccessible for educational purposes. Furthermore, the device only renders one form of haptic feedback, ball collision. Our device is inexpensive and allows the users to feel different forms of haptic feedback.

Sony developed an augmented reality air hockey game where up to 4 players can play at the same time.(4) The advantages of this device are the great graphics giving an enjoyable user experience and the high speed “vision sensor” to track the position of the players on the field at a very high rate and very accurately allowing for almost instantaneous haptic feedback to the player when there is a collision with the puck. However, this device is very expensive and might not be a viable option for educational purposes like our device and more geared purely towards entertainment. Moreover, the handles are not grounded and don't possess a mechanism like in Harold's Purple Crayon allowing it to use force feedback without grounding.(1) This prevents it from rendering force feedback to the players and the only haptic feedback it gives the player are vibrations.

All three devices showed different ways to use haptic feedback for entertainment and education. Our devices tries to find a balance between these devices, using more degrees of freedom than Haptic Pong, more force feedback then Sony’s air hockey game and cheaper than the augmented reality ping-pong.

Methods

Provide a detailed description of your project, such that another student from the class could generally re-create your project/experiment from the report if necessary. (You don't need to document every screw, but the design should be clear.) Add images and videos as needed to support the description. You can refer to downloadable drawings and code in the "Files" section (later). You should divide this section into subsections, which can vary depending on your particular project. Here is an example set of subsections:

Hardware design and implementation

For this project, we used the framework of the Hapkit two degree of freedom (2 DOF) arrangement (which we will call the controller henceforth). The CAD files for the Hapkit device modifications to create the controller were courtesy of the Charm Lab at Stanford University. The lab’s initial device use case was a writing haptic tool, but the pantograph form factor their device implements proved to be effective for our use case as well. While we had considered a stylus form factor similar to what we presented on in the class presentation, this would have required far too much design for the short project timeline.(1)

A pantograph is useful in multiple ways. The grounding feature ensures that players remain on their side of the “table.” Additionally, the kinematics are well documented and easily modifiable given new geometries. Lastly, it allowed us to use components we already acquired from the class. A major disadvantage of this configuration is the useable workspace available as you approach the front wall. We encountered this issue as we realized that defending the goal becomes difficult since the pantograph workspace tapers to a singularity at the full extension of it’s workspace. If we were to superimpose a rectangle onto that viable workspace to improve the useable lateral motion, our other degree of freedom becomes severely limited and the game approaches a version of Pong.

We encountered many issues with our second controller, which had the same hardware components as the first. After having used the motors for the entire quarter, in retrospect, we should have replaced the motors before one failed just a day before the final demo. We also had repeatable position sensing issues with the magnetoresistive sensors. We frequently needed to reset the Arduino boards in order to re-calibrate the sensors. However, even then we would frequently see inconsistent readings from our sensors. Overall, we would have preferred to make the hardware more robust, which would have been feasible if we had focused on one controller. We also would recommend a better position sensor. For instance, purchasing motors with integrated encoders would likely provide more reliable position sensing.

System analysis and control

Kinematics and Geometry:

The figure above shows the relevant kinematic picture for our pantograph. The green links have equal length of about 13 cm, and the blue links have an equal length of about 15 cm. The joints are labeled from 1 through 5, and the two angles depicted are the angles we sense to compute the position and Jacobian of the end effector at joint 3. All positions are measured with respect to the frame centered at joint 1.

The figure above shows the relevant kinematic picture for our pantograph. The green links have equal length of about 13 cm, and the blue links have an equal length of about 15 cm. The joints are labeled from 1 through 5, and the two angles depicted are the angles we sense to compute the position and Jacobian of the end effector at joint 3. All positions are measured with respect to the frame centered at joint 1.

Dynamics & Simulation:

Our system includes a dynamic virtual environment consisting of virtual walls, static power ups, and the hockey puck. The virtual wall is rendered as a simple one-sided spring that can only push objects away. This is part of how we constrain the player from moving too close to the limits of the workspace and also how we get the puck to bounce off of the table walls. The power ups are static, but these could easily have their own motion variables and associated dynamics. The hockey puck is a simple particle which can experience forces from collisions with walls or the player. In addition to these applied forces, the puck continuously experiences linear viscous damping to represent the natural friction a puck has on a real hockey table (and to keep the puck stable).

To simulate the dynamics of the virtual environment, we update the state variables with forward Euler integration at a known time step. For objects that have masses, we use Newton’s second law to deduce the acceleration of the objects from the applied forces due to interactions with the player or other virtual objects. For example, the algorithm for computing the puck’s dynamics is as follows:

				ax = Fx / m
				vx = vx + ax * dT
				x = x + vx * dT 

Where ax is the acceleration of the puck, Fx is the force on the puck, m is the mass of the puck, vx is the velocity of the puck, x is the position of the puck, and dT is an incremental time step for one loop of the Arduino code.

Our haptic rendering to the player consists of the force interactions the player’s handle has with the virtual environment. When the handle collides with a wall, we render a spring force to push the player away from the wall. Similarly, when the handle collides with a puck, we render a similar spring force to push the two objects apart. Since the wall is a static object, we do not reflect the force onto the wall. However, we do reflect the haptically rendered force back onto the puck to achieve a convincing collision with the puck.

Graphics

To render the graphics for our virtual hockey playing board we used Processing 3.5.3. In Processing we created the avatars to represent the pantograph handle, the hockey puck, and the power up stars. Our processing code read in values from Arduino to set the position of the avatars, keep track of the game score, and to track the implementation of the power ups. We mapped the maximum range of x and y values of the handle to the virtual playing board in order to take advantage of the full range of motion of the handle. However, our playing board screen was a rectangle and the workspace of the handle had more of a triangular shape with an arched side. This caused us to have some corners that the handle could not get to on the board. We thought this approach was better than limiting the range of motion of the handle to a rectangle within its workspace. To ensure that the hockey puck did not get stuck in an inaccessible corner, we set the puck to have very little damping. Another challenge we had was that the handle did not have a symmetric range of motion in the x axis which caused our goal to be offset. This was something we could have shifted in the Arduino conditions but due to time constraints we did not get to it.

Demonstration / application

The day of the demo, we only had one of the two pantographs up and running, preventing us from doing a one v. one game. We opted for a game where the player has to hit the puck into a small opening at the bottom of the playing field. This simulated the functionality of half of the air hockey table. After 5 seconds, different shaped stars appear. If the player passes the paddle over the star, the player receives one of multiple power ups for a certain amount of time. New stars appear allowing the user to experience different types of power ups. For example, some stars change the weight of the puck. This causes the puck to move slower and also makes changing the direction of the puck more difficult. The player also feels more force when colliding with the heavier puck. Another power up causes the ball to randomly change directions, making more difficult for the player to predict its motion and hit it. If the player scores three points, they win!

Results

At our open house demo, we had several people both internal and external to the class test our device. In general, people were able to feel when they made contact with the puck. Several people noted that when the puck travelled faster or had greater mass, the collision felt stronger, as one would expect from a real hockey puck. Physical workspace limitations and unreliable sensing at high speeds meant that players tended to reach the physical boundaries of the workspace before they reached the virtual walls.

Reactions to the device were generally positive, but many people seem confused about the objective of the game. If we had time to replace our faulty hardware on demo day, we expect that the game would have been more intuitive to play and also more engaging. Most people enjoyed the brief learning curve involved in using or avoiding power ups and accurately hitting the puck in a desired direction.

Future Work

In this first iteration of our project we achieved a one player hockey game due to last minute hardware issues. This first iteration was a satisfying proof of concept for what our vision of the two-player version of the game could have been. We learned a lot from the challenges that came up throughout our work and we would like to see certain aspects improved upon in a future version. Future work we would like to see regarding the hardware we used is modification in the arrangement of the handle links to allow a larger workspace without a bulkier controller, better position sensors or direct driving of the linkages with the motors to avoid slips in position tracking, and an attempt at using an ungrounded controller like that demonstrated in the paper Harold’s Purple Crayon. In terms of the graphics, we would like to have experimented with better ways to make use of the entire playing board despite the shape of the controller workspace. We also would like to make the power ups more intuitive and clear in which effect each star will have. Our primary entertainment application was to render air hockey but while working to implement the power ups we were inspired to implement other games involving catching stars and increasing the amount of balls the player has to avoid. In the future we would also like to make this a multi game system that allows you to select what game you’d like to play.

Acknowledgments

We would like to thank Professor Allison Okamura and the teaching team, Cara Nunez and Julie Walker for their help and feedback throughout our project and for openly sharing the Graphkit resource we used for our hardware.

Files

Processing and Arduino code for Single Player Air Hockey: Attach:SinglePlayerAirHockey.zip

Materials list:

http://hapkit.stanford.edu/files/GraphkitPurchasePartsList.pdf http://hapkit.stanford.edu/files/Hapkit3DPrintedPartsList_v11_11_15.pdf

References

(1) S. Kianzad and K. E. MacLean, "Harold's purple crayon rendered in haptics: Large-stroke, handheld ballpoint force feedback," 2018 IEEE Haptics Symposium (HAPTICS), San Francisco, CA, 2018, pp. 106-111.

(2) Jeffrey A. Kessler, R. Curtis Lovelace, and Allison M. Okamura, “A HAPTIC SYSTEM FOR EDUCATIONAL GAMES: DESIGN AND APPLICATION-SPECIFIC KINEMATIC OPTIMIZATION”, ASME 2013 Dynamic Systems and Control Conference, Palo Alto, CA, 2013

(3) Knoerlein, Benjamin, Gábor Székely, and Matthias Harders. "Visuo-haptic collaborative augmented reality ping-pong." In Proceedings of the international conference on Advances in computer entertainment technology, pp. 91-94. ACM, 2007.

(4) Karissa Bell, “Sony’s augmented reality air hockey is better than the real thing”, in Mashable, MAR 10, 2018

(5) Bau, Olivier, and Ivan Poupyrev. "REVEL: tactile feedback technology for augmented reality." ACM Transactions on Graphics (TOG) 31, no. 4 (2012): 89.

(6) G. Campion, Qi Wang and V. Hayward, "The Pantograph Mk-II: a haptic instrument," 2005 IEEE/RSJ International Conference on Intelligent Robots and Systems, Edmonton, Alta., 2005, pp. 193-198.

---

Appendix: Project Checkpoints

Checkpoint 1

Our goal for checkpoint one was to successfully build one graph-kit and start working on the second one. This week we managed to reach our set goals, we finished printing all the parts for the two graph-kit (panto-gram) and acquired all the pieces for its assembly except for one piece (half the bass for the second graph-kit). We successfully build one graph-kits and ready to build the second one within the day!

WE faced a few problems when building out graph-kit especially with the 3D printing. Often the parts would bend quite a bit. We had to print a few parts in room 36 where the generated support structure allowed for better printing. Some parts were made with the Maker-bot, to prevent to much bending, using blue tap at the base helped. This increased the sticking of the base of the print. Adding nylon spacers helps allot with friction. We decided to add them in the hinges.

Here are images of the two graph-kits:

Checkpoint 2

Here you will write a few paragraphs about what you accomplished in the project so far. Include the checkpoint goals and describe which goals were met (and how), which were not (what were the challenges?), and any change of plans for the project based on what you learned. Include images and/or drawings where appropriate.

We have assembled the two graphkits that will be used as the air hockey handles that the user interacts with. We are working on tidying up the cables so that they do not get disturbed by excited users. For our second checkpoint, our goal was to have all the controls, position tracking, and graphics completed. We were able to meet these goals and are now working to calibrate the sensors and to integrate the Arduino code and the Processing code. We realized that we might have trouble with the position slipping if the handles are moved to fast while playing.

Two graphkit handles with the play screen in between:

A sample of the graphics for the two player hockey game is shown below: