Haptic Mini-Golf

Project team member(s): Sam Rodriguez, Shail Trivedi, and Bianca Jurewicz

Haptic Mini Golf is an interactive game where a user holds a 6-DOF handheld haptic golf club and tries to putt the virtual golf ball into a hole at the end of the green. Our motivation for this project was to create an engaging and fun haptic device users will enjoy on demo day. This game was designed to teach students how collisions with a small mass can be simulated haptically using vibrations accompanied with a familiar game experience.

The handheld device is accompanied by a virtual display that shows the ball on the green as well as a virtual representation of the club head. The physical device contains an IMU that tracks the orientation of the club. The sensor information is passed through an Arduino UNO mounted nearby. The club also contains ERM motors that vibrate when the club comes in contact with the virtual ball. The dynamics of the putter are simulated as a mass-spring-damper system and the golf ball is also affected by friction as it rolls along the virtual grass. 22 users experienced how changing their applied force and orientation affected the ball’s trajectory. The majority of the users commented that the vibrations were noticeable and the graphics were accurate, but the device could have felt more realistic.

Introduction

For this project, we wanted to study the dynamics between a putter and a golf ball along with simulating this interaction haptically while creating an entertaining and realistic game. In regards to realism, this was best accomplished by an unconstrained haptic device that could be translated and rotated in any direction giving the user full control over the putter. An unconstrained device created challenges with properly measuring position, orientation, and user applied forces. It also created challenges in that there is no grounded actuators that can apply haptic forces back to the user. The first challenge was partially overcome with a well-calibrated IMU sensor which allowed us to capture orientation of the haptic device. Using orientation and its rate of change, position and user applied forces could then be estimated. The second challenge was solved by simulating the haptic forces as vibrations which is acceptable due to the physics of the problem. In a putter and golf ball collision, there are noticeable but small forces applied back to the golfer. The haptic vibrations indicate when the force would occur giving the illusion of a collision with a small mass.

Background

This section summarizes the key publications that we reviewed in order to gain background knowledge on other haptic golf applications.

The paper titled "Golf Club-type device with force feedback for modifying club posture" (Nakamura et al., 2020) proposes a haptic device that provides dynamic force feedback to the user as they swing a golf club. The system uses a gyroscopic pan-tilt mechanism in order to produce a torque at the handle and convey real-time feedback regarding the required face angle of the club to achieve a particular golf ball trajectory. The authors noted that existing solutions or devices on the market all track motion or provide feedback after the swing is complete, which does not allow the user to correct their motion in real-time or understand how minor changes in the golf club’s face angle and acceleration may affect the final golf ball trajectory. Although we did not implement an identical mechanism for force feedback, the concept and intention of this device is very similar to what we intended to achieve in our final project. We used this paper’s findings in required torque as a baseline to help guide us in our vibratory force feedback rendering.

Cao et al., analyzed the motion of a golf swing using data from inertial and vision sensors in their paper titled ("3D dynamics analysis of a golf full swing by fusing inertial sensor and vision data"). The system used an aluminum frame as the club, mounted to a motor with an encoder at the end of the frame, where a user would typically hold a club. The club made a full rotation around the motor axis and made contact with a ball at the bottom of the swing. An inertial sensor was mounted on the head of the club, and two cameras with IR sensors were mounted on a separate frame to capture the movement of the club. Although we did not use vision sensors for our project, this paper identified key elements of a golf swing that result in a good hit. Namely, the velocity and orientation of the golf club head when the club comes in contact with the ball influences the trajectory of the golf ball. Furthermore, the motivation behind this paper was to make players aware of their swing motion, allowing them to improve their swing. We initially intended to incorporate this motivation into our project by using haptic feedback to correct a player’s swing, but later pivoted to a different reach goal due to feasibility.

Finally, we reviewed a paper that focused on the introduction of haptics into a gaming environment through the creation of a PC game called “Haptic Battle Pong” (Morris et al., 2016). The goals of the paper were to create a physical simulation of pong, a realistic force rendering, and improve the enjoyment of the game through haptic interaction. The takeaways from the paper relating to our project involved the contact simulation of a paddle with a ball, the anecdotal experiences of the researchers creating a haptic game, and the test subject feedback about enjoying a haptic game. The authors modeled the contact force of the ball with the paddle as spring and noted that users reacted more positively to a constant haptic feedback instead of proportional feedback over a certain force threshold. They also made insightful comments about game design stating that, while complicated haptic games can be fun and exciting to create, there is a large learning curve for new haptic devices.

Methods

The following sub-sections detail the hardware design, system dynamics, and simulation of the project.

Hardware Design

Sensor Integration

The Inertial Measurement Unit (IMU) was mounted to the base of the golf club as shown in the image below. This location was chosen so that the IMU could be fixed onto a flat surface that would not interfere with the user's grip or normal putter swing. The corresponding PWR, GND, serial data (SDA), and serial clock (SCL) wires from the IMU were routed up the side of the base and then connected to the 5V, GND, SDA, and SCL pins on the Arduino UNO.

Actuator Integration

Two Jameco 9000 RPM ERM motors were mounted to the inside of the handle and grip of the putter. This number of motors and the motor mounting location was selected after trialing several different configurations and qualitatively assessing the haptic feedback for realism (strength of perceivable vibration and being relatively noise-free). The corresponding PWR and GND wires of each motor were routed down through the putter and out from an opening near the base where they connected with the digital Pulse-Width Modulation (PWM) and GND pins on the Arduino UNO.

Furthermore, we varied the duty cycle of the vibrations based on the velocity of the putter when it contacts the ball. This means larger impact forces will result in larger vibrations from the motors.

Device Prototyping

After the first iteration from Checkpoint 1, the Computer Aided Design (CAD) model was updated to include mounting platforms and retention features on the middle and grip parts for each of the vibration motors. We also added a hole near the bottom of the middle part for improved vibration motor and IMU wire routing back to the Arduino UNO. Finally, we reduced wall thickness down to 2mm to reduce 3D printing time of the system and modified the sub-component mating clearances after evaluating the previous assembly iteration.

System Analysis and Control

The putter is controlled by the user and orientation is measured with an IMU. The position of the putter is estimated by modeling the haptic device as a fixed-length pendulum. The roll of the device is measured by the IMU and the x-position is calculated as function of the roll, theta, and the length of the pendulum, L, as expressed with the equation below. The pendulum length was set to be 0.75m.

The virtual golf ball is controlled by two forces: the force from the golf club and the force due to friction. We decided to represent the golf club as a mass-spring-damper that exerts a force on the golf ball. The friction force acts like damping and is a function of velocity, v. The ball starts near (0,0) and travels to the left, so the impact force from the user acts in the negative x direction and the friction force acts in the positive x direction.

Drane et. al. (2013) found the coefficient of friction between the golf ball and artificial turf as the ball is rolling is 0.26. This value is used to calculate the frictional force in our dynamic model. After tuning our gains, we found k=850N/m worked best for the system.

The total force is projected in the plane of the green with respect to the angle of the putter. This means the forces acting on the ball are normal to the yaw orientation of the club at the point the club comes in contact with the ball. Using x as the horizontal direction across the screen (towards the hole), and y as the vertical direction across the screen, the projection can be calculated as follows:

Where phi is the yaw angle of the golf club with respect to the z axis. The z axis is pointing out of the screen in simulation and perpendicular to the ground in the real world.

The acceleration of the ball is calculated as the force projected in the x and y simulation directions divided by the mass of the ball, which was found to be 45.9g for a Titleist Pro V1 golf ball in the paper from Drane et. al (2013). The velocity and position of the ball are calculated through integration. To accomplish this, we stored the previous position, velocity, and acceleration of the ball, and compared it with the current values given the forces acting on the ball at each time step. The equations used are as follows:

Simulation Development

Virtual Environment

We used Processing to create a realistic virtual environment with a 5m x 2m putting green and used ratios of the world frame to the pixel frame to calculate the appropriately scaled size of the golf ball and putter. We represented the putter as a rectangle that rotates according to the orientation of the IMU (as shown below) and restricted its movement to a horizontal motion so the user will always contact the virtual ball.

We then rendered a target hole with a diameter larger than the golf ball and added colors to all of the features.

Haptic Feedback

The Arduino code was set up to apply a spring force to the golf ball when the putter x-position was within the x-boundary of the golf ball. To render realistic haptic feedback during a swing of the putter, the vibration motors were run at a duty cycle directly proportional to the velocity of the putter once the impact was detected and a timer was used to set the motor vibration duration. The duty cycle was limited to values between 35-80% to ensure a noticeable haptic effect was created on the lower limit and the Arduino does not draw unsafe current at the upper limit. After several rounds of testing, we noticed that a motor on-time of 0.5 seconds was reasonably quiet and realistic. Shorter motor on-times could not be used because at lower duty cycles there was insufficient time for the ERM motors to reach full-speed and produce a noticeable haptic effect. Longer motor on-times would produce a haptic effect that is inconsistent with a quick tap of a putter with a golf ball.

Game Reset

In our testing, we used the reset button on the Arduino UNO to reset the virtual environment in Processing. This feature was quick and effective, so we maintained its implementation for switching between users on demo day as well.

Score Tracking (Reach Goal)

After testing our game, we realized it is quite difficult to score a hole-in-one. To improve and simplify gameplay for the user, we implemented rings around the target hole that allowed the user to score points even if the golf ball did not end up inside of the hole at the end of the swing. The Processing code checks the x and y position of the ball to continually update the score, based on whether the ball position is within any of the scoring rings. The score is displayed in the top center of the screen during game play.

Results

On demo day, we tested 22 users and recorded their best scores as well as their feedback on the project. The user group included people who are ME327 students as well as those who have never taken a haptics course. The users spanned a variety of ages, a balance of gender, and a mix of prior familiarity with putting.

As shown in the pie chart below, there is a nearly-even distribution between people who scored 5 points, 1 point, and 0 points. Only one person was successful in scoring a hole-in-one worth 15 points.

We received user feedback in the following four categories: Noticeability, Stability, Realism, and Graphics Correctness. The results are shown in the bar chart below:

Generally, users were able to notice the variety in vibrations as they contacted the ball with larger or softer forces and felt the generated vibrations were noise-free overall. Additionally, most users found the graphics accurately reflected the motion of the virtual ball based on their input. This validates the choice of gains, constants, and equations used to render the dynamics of the golf ball. Interestingly, users who performed very well and scored a 5 or above rated the project with a 4 or a 5 in every category. Therefore, there is correlation between how well a user did and how well they felt the system performed.

However, this project received lower ratings for Realism. Our hypothesis is this low rating is correlated with the difficulty of the game and the placement of the IMU. 95% of users were unable to score a hole-in-one after several tries. The instantaneous yaw rotation of the golf club as it contacts the ball is used for the projection of the ball's motion. Perhaps an average of the club's yaw rotation throughout the swing would have been easier to control and felt more realistic for the users. We also believe putting the IMU on the top end of the handle may have resulted in more control and thus a more realistic feeling for the user. The simple colors in the graphics may also contribute to the low realism rating.

Future Work

The haptic game was thoroughly tested by the developers and over 22 users through continuous gameplay. A few areas were identified for improvement and expansion. The first improvement would be to reduce the distance of the IMU from the center of rotation by relocating the IMU from the putter head to the handle. This would significantly reduce unwanted linear accelerations measured by the accelerometers which manifested as oscillations of the putter head on the processing display. The relocation of IMU would result in an increased swing consistency. The second improvement would be supplemental tracking of the haptic device's translation with a camera as the pendulum model of the putter is not intuitive to the users. A camera would add additional feedback to confirm the position of the putter head in the world frame and track translations of the club.

Haptic Mini-Golf can be expanded into virtual reality gaming to further enhance the user's experience. The game environment can be displayed directly to the user allowing the user to look down at the golf ball instead of looking ahead at a screen. It may also further improve realism by restricting the user to only view putter movements within the game avoiding any visual discrepancies with the real-world movements of the putter. A 3-D environment would also allow for more complicated putting scenarios with visible slopes and walls.

Files

Attach:Arduino_Final_Golf.txt

Attach:Processing_Golf_Display.txt

Attach:Final_CAD_Model.zip

Attach:Project_Cost_Estimate.xlsx

References

N. K. N. Cao, Y. S. Suh, & Q. K. Dang. "3D dynamics analysis of a golf full swing by fusing inertial sensor and vision data," (2013). 13th International Conference on Control, Automation and Systems (ICCAS 2013), pp. 1300–1303. https://doi.org/10.1109/ICCAS.2013.6704153.

Nakamura, T., & Koike, H. “Golf Club-type device with force feedback for modifying club posture.” (2020). Extended Abstracts of the 2020 CHI Conference on Human Factors in Computing Systems, pp. 1–7. https://doi.org/10.1145/3334480.3383024

Patrick Drane, Michael Duffy, Jonathon Fournier, James Sherwood, Michael Breed, 'The Behavior of Golf Ball Putting on Artificial Turf', Procedia Engineering, Volume 72, 2014, Pages 599-604,ISSN 1877-7058, https://doi.org/10.1016/j.proeng.2014.06.107.

Morris, D., Joshi, N., & Salisbury, K. “Haptic Battle Pong: High-Degree-of-Freedom Haptics in a Multiplayer Gaming Environment.” (2016). https://www.microsoft.com/en-us/research/publication/haptic-battle-pong-high-degree-freedom-haptics-multiplayer-gaming-environment-2/

---

Appendix: Project Checkpoints

Checkpoint 1

For the first week of the project we had the following goals in mind:

• Select and procure sensors and actuators for the project
• Devise a state diagram and initial control strategy
• 3D print a prototype of the golf club with sensor and actuator mounting points
• Write Arduino code to output forces from actuators based on inputs from sensors

We were able to meet all of our goals for this week.

Component Selection

We selected two Jameco 9000 RPM ERM motors for this project. These motors will vibrate simultaneously to render the feeling of impact when the golf club impacts the virtual ball. This particular motor was selected because we have familiarity using it through other assignments in this course and using two motors together will render a strong enough force to be felt through the handle of our device.

Control Strategy

The state diagram for our device is shown below. We anticipate having a linear process and the ability for users to take endless “practice swings” is captured with the arrow going from the “Swing” state to the “Backswing” state.

For the control strategy, we decided to represent the golf club as a mass-spring-damper that exerts a force on the golf ball. The golf ball will also be slowed by the force of friction as it rolls across the grass. To add to the complexity of the simulation, we decided to make part of the green have an incline with an angle theta. Theta will be 0 for the first half of the green and set to 15 degrees for the second half of the green.

The forces acting on the ball will be the force from the user, modeled as a spring force, as well as the frictional force:

Drane et. al. (2013) found the static coefficient of friction for a golf ball on artificial turf is 0.30, and the coefficient of friction between the golf ball and turf as the ball is rolling is 0.26. These values are used to calculate the frictional force in our control law.

The total force must be projected in the plane of the green with respect to the angle of the putter. Using x as the horizontal direction across the screen (towards the hole), and y as the vertical direction across the screen, the projection can be calculated as follows:

Where phi is the angle of orientation of the golf club with respect to the z axis. The z axis is pointing out of the screen in simulation and perpendicular to the ground in reality.

The acceleration of the ball will be calculated as the force projected in the x and y simulation directions divided by the mass of the ball, which was found to be 45.9g for a Titleist Pro V1 golf ball in the paper from Drane et. al (2013). The velocity and position of the ball will be calculated through integration using the trapezoidal rule. To accomplish this, we will store the previous position, velocity, and acceleration of the ball and compare it with the current values given the forces acting on the ball at each time step.

Prototyping

Our Computer Aided Design (CAD) model of the golf club consists of three sub-components (grip, middle, base) that are divided up in order to fit within the available volume on the 3D printer and optimize for printing time. The middle and base both include a cylindrical mating feature to allow for assembly without any adhesive or tape. At this time, there are no vibration motor mounting holes or wire retention features on the golf club because we want the ability to experiment with different locations to find the optimal placement.

The images below show our 3D CAD model and final assembly.

Programming

Our final goal was to output forces to actuators based on inputs from the sensors. We were able to calibrate the IMU and receive readings through Arduino. We also wrote code to turn on the vibration motors once the IMU rotates at least -15 degrees in one direction followed by 15 degrees in the other direction. This means the motors turn on after a full backswing and forward swing to hit the virtual ball. We have also imported the IMU readings into Processing for visualization of the orientation of the club, as shown in the photo below.

---

Checkpoint 2

For the second week of the project we had the following goals in mind:

• Implement golf ball and club graphics into Processing
• Simulate the physics of the golf ball
• Incorporate game reset
• Tune haptic actuators to render realistic force
• (Reach goal: haptic feedback for club angle correction -> add rings around hole for more scoring opportunities/easier gameplay)

We were able to meet all of our goals for this week. After reviewing our reach goal with the teaching staff, we decided that the haptic feedback from the vibration motors may not be effective in helping correct the putter orientation, so we did not pursue this reach goal. Instead, our new reach goal was to implement rings around the hole that could be used to calculate a score based on how close the ball ended up near the hole.

Hardware Design

Sensor Integration

The Inertial Measurement Unit (IMU) was mounted to the base of the golf club as shown in the image below. This location was chosen so that the IMU could be fixed onto a flat surface that would not interfere with the user's grip or normal putter swing. The corresponding PWR, GND, serial data (SDA), and serial clock (SCL) wires from the IMU were routed up the side of the base and then connected to the 5V, GND, SDA, and SCL pins on the Arduino UNO.

Actuator Integration

Two Jameco 9000 RPM ERM motors were mounted to the inside of the handle and grip of the putter. This number of motors and the motor mounting location was selected after trialing several different configurations and qualitatively assessing the haptic feedback for realism (strength of perceivable vibration and being relatively noise-free). The corresponding PWR and GND wires of each motor were routed down through the putter and out from an opening near the base where they connected with the digital Pulse-Width Modulation (PWM) and GND pins on the Arduino UNO.

Furthermore, we varied the duty cycle of the vibrations based on the velocity of the putter when it contacts the ball. This means larger impact forces will result in larger vibrations from the motors.

Device Prototyping

The Computer Aided Design (CAD) model was updated to include mounting platforms and retention features on the middle and grip parts for each of the vibration motors. We also added a hole near the bottom of the middle part for improved vibration motor and IMU wire routing back to the Arduino UNO. Finally, we reduced wall thickness down to 2mm to reduce 3D printing time of the system and modified the sub-component mating clearances after evaluating the previous assembly iteration.

Simulation Development

Virtual Environment

We used Processing to create a realistic virtual environment with a 5m x 2m putting green and used ratios of the world frame to the pixel frame to calculate the appropriate size of the golf ball and putter. We represented the putter as a rectangle that rotates according to the orientation of the IMU (as shown below) and restricted its movement to a horizontal motion so the user will always contact the virtual ball.

We also rendered a target hole with a diameter larger than the golf ball and then added colors to all of the features.

Golf Ball Dynamics

We rendered the position of the golf club using the formulas outlined in Checkpoint 1. However, we noticed our friction force should act more like damping and be a function of velocity rather than a constant. The new formula for the friction force is:

where v is the velocity of the ball. We also changed the signs of our forces based on our decision for the coordinate system. The ball starts near (0,0) and travels to the left, so the impact force from the user acts in the negative x direction and the friction force acts in the positive x direction.

Furthermore, after testing our game, we realized the friction already adds a large element of difficulty and did not find it necessary to put part of the playing field on an incline, as this would make the game even harder.

We solved for the acceleration of the ball by dividing the total force by the ball's mass. Then, we integrated to get the ball's velocity and position.

Putter Dynamics

We mapped the orientation of the putter as it rotates around the user's wrist to the putter position in the x direction. We also constrained the putter graphics to the y=0 axis. This ensures the putter will always contact the ball when a user swings. Lastly, we mapped the putter's rotation about the handle's axis to the rotation about the z axis in Processing. This changes the angle the putter comes in contact with the ball and in turn affects the ball's dynamics.

Haptic Feedback

The Arduino code was set up to apply a spring force to the golf ball when the putter x-position was within the x-boundary of the golf ball. To render realistic haptic feedback during a swing of the putter, the vibration motors were run at a duty cycle proportional to the velocity of the putter once the impact was detected and a variable was used to confirm the motor on-time was at 0.5 seconds. After several rounds of testing, we noticed that the haptic feedback was reasonably quiet and realistic, but there was a time delay between the user's swing and golf ball impact to the time when haptic feedback was first delivered. We resolved this issue by starting motor vibration slightly sooner than real golf ball impact to eliminate the perceivable time delay without disrupting the realism of the haptic device.

General Gameplay

Reset

We found that pressing the reset button on the Arduino UNO also reset the virtual environment in Processing, so we plan to use this feature when switching between users on demo day as well.

Score Tracking (Reach Goal)

After testing our game, we realized it is quite difficult to score a hole-in-one. To improve and simplify gameplay for the user, we implemented rings around the target hole that allowed the user to score points even if the golf ball did not end up inside of the hole at the end of the swing. The Processing code checks the x and y position of the ball to continually update the score, based on whether the ball position is within any of the scoring rings. The score is displayed in the top center of the screen during game play.