Caption:
Thrilliards

Thrilliards: A Haptic Billiards Simulator

Project team member(s): Anthony Macias, Carlos Arias, Sebastian Urdaneta,

Thrilliards is a exploration of taking a classic physical game, turned into a virtual game and then brought back into the physical realm with haptic capabilities. Our project focuses on allowing the user to interact with a virtual pool game using a physical cue and haptic feedback. Using a 3D printed cue attached to a hapkit the user is able to strike the ball and feel the collision forces. Embedded vibrational motors in the cue enhance the collision feel. Our device brings a virtual game of pool back into the physical realm and adds another layer realism.

Introduction

In an attempt to better understand and accurately simulate realistic haptic simulations, our project gives us the opportunity to finely tune parameters that affect the perception of collision within the device. Programming the device to react to high speed collisions with devices that are in contact on the scale of milliseconds requires some new techniques and control systems that have not yet been practiced in the course. The new set of challenges brought up from our project is what allows us to explore new dynamic simulations and the control systems behind them in order to create a realistic simulation of a pool cue and its interaction with pool balls.

Background

As a group, we were able to find previous work with haptic devices that explore the rendering of collisions and how to add to the programming that aids in increasing realism in collision perception. This includes a paper that provides a straightforward explanation of how a collision can best be rendered using a model of infinite stiffness for high speed collisions and finite stiffness for sustained contact (Constantinescu et. al., 2004).

Another insightful reading that helped us determine specific device features is "Realistic Haptic Rendering of Collision Effects Using Multimodal Vibrotactile and Impact Feedback" by Park et. al. The use of both kinesthetic and cutaneous phenomena allows for a more complete sensation of collision and contributes to the realism of the simulation. Though they conduct a much more mathematically modeled computation for force delivery, it helps us better understand the way that bodies transmit forces to humans that best take advantage of our haptic perception. For our project specifically, we made sure that we implemented a combination of kinesthetic forces that represents the physics and dynamics of a cue hitting a ball, as well as a more qualitative vibrational force on the handle that adds an extra layer of realism that is felt whenever an object interacts with another object.

Methods

Hardware Design and Implementation

Our hardware design was a modified hapkit connected to a 3D printed pool cue handle. The pool cue handle was connected to the top of the hapkit handle via a pin joint, which allowed users to input a purely linear motion, which is more representative of the desired pool cue action. We maintained a single degree of freedom in this design, and limited this modification's effect on the system's dampening by implementing a ball bearing. In the handle itself, we embedded 3 vibrational coin motors spaced 15 mm apart. These vibrational motors displayed a sensory saltation, or a perceived motion throughout the pool cue handle after collision between the pool cue and the virtual cue ball. This perceived motion was the rippling vibration that travels through the length of the pool cue that a user would experience when striking a pool ball. The 15mm spacing between the motors was the longest distance that the continuous motion was perceived during our calibration.

System Analysis and Control

The main event that occurs within our system is when the pool cue collides with the ball to launch down the table. To model this in a realistic fashion, we modeled the collision as a spring compressing with a very high k-rate. We applied a damper on the ball movement to simulate friction. There was also a similar effect applied to the cue movement to make it feel more realistic for the user.

The spring gain was dialed in in a way to make sure that the pool cue colliding with the ball feels stiff enough to be like an impulse which is aided by the dynamics of the system. An excessive spring constant would saturate the motor and cause it to over compensate and leads to high frequency oscillations that are unnatural and hurt the realism and function of the device.

The spring constant found was 1100 N/m and the damping constant was set to 2. These values qualitatively made the dynamics of the interaction seem as realistic as possible when considering how a normal billiards ball would move. Since the ball is not a spring and only linearly damped, there is no instability that we worry about and it just took some trial and error with the values to produce a smooth experience.

The force of collision felt on the user is modeled as treating the ball as a stiff wall. The harder that the user pushes on the ball, the stronger force they will feel. However, since the ball only experiences damping, the ball is moving away from the cue as soon as they make contact. Therefore the force on the cue (that the user feels) is delivered on a short time scale, thus acting more like an impulse force. This delivered force is used to determined the acceleration of the ball based on momentum principles. Numeric integration in the code will calculate the velocity, and another integration produces the ball's position. The damping force is proportional to the ball's velocity and is representative of friction. The position is fed through the serial port of arduino and into the processing code and allows the ball's graphics to be updated appropriately.

Demonstration / Application

During our demonstration, we asked random participants to try our device under specific conditions. One random group will interact with the device with full implementation of visual and haptic inputs as they try out the device. A second random group will not be able to see the visuals that go with the device and only feel the forces as delivered by the device; however, they are given a brief on what the visuals would be. The last random group will be able to see the visuals as they interact with the device but all power to the motors will be disconnected and they will have no haptic feedback. We take inspiration from the paper by Park et. al about how each user is able to evaluate the quality of the device. Each user is asked to rate the experience based on realism, unnaturalness, and in a more qualitative sense, how much they enjoyed the whole experience. Each of these ratings will be made on a scale of 1-100.

Below are the results of this analysis:

Results

Qualitatively, we can see that having both the haptic and the visual input during the interaction increases both realism and likeability and keeps unnatural feelings on the lower end. It is interesting to note that the missing haptic feedback actually makes the user feel as though it is less realistic and more unnatural compared to the other groups. While there may be confounding variables given that the demonstration was done during a haptic project open house based around a course about the design of haptic systems, there is evidently more appreciation for novel interactions that implement haptic feedback.

Users are accustomed to computer games that offer exciting visuals but no haptic feedback, so the addition of these features make the interaction more enjoyable and push towards realism.

Future Work

Future work for this project could focus on adding a variety of additional features to the game. The first additional feature we would add would be to add an element of rotation. Currently the cue can only move along the x-axis, adding rotation and movement along the y-axis would greatly increase the complexity and the immersiveness of the experience. This could be achieved by adding a second DOF hapkit to achieve the x and y-axis motion. A third motor on the tip of a 2DOF system could also be added to sense rotation.

Apart from these improvements, we could also improve this project by obtaining higher fidelity motors for higher resolution movement. Tighter tolerances on the joints and a more rigid attachment for the arduino board would also be beneficial. This iteration of Thrilliards struggles with having the arduino board move around a lot which led to the rotational position being lost easily.

Acknowledgments

We want to acknowledge the teaching staff for this course for their great help and advice throughout the whole process of this project.

Files

Attach:ArduinoCode

Attach:ProcessingCode

References

1. Constantinescu et al. 2004 Haptic Rendering of Rigid Body Collisions https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=1287171

2. Park et al. 2019 Realistic Haptic Rendering of Collision Effects Using Multimodal Vibrotactile and Impact Feedback https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=8816116

3.Conference on Robotics and Automation, Rome, Italy, 2007, pp. 485-490, doi: 10.1109/ROBOT.2007.363833. https://ieeexplore.ieee.org/abstract/document/4209138

Appendix: Project Checkpoints

Checkpoint 1

At this first checkpoint, our team has settled on some important design characteristics of the hardware and written a bit of pseudocode. For hardware, we chose the effective length of the pool cue handle that our team would like to use, determined the force the motor should apply to the handle that will be realistic, and conceptualized a multi-vibrational motor cutaneous sensation. For these initial components, we have created a CAD model and have begun acquiring materials and 3D printing. Specifically, we have purchased bearings and a D-shaft from the PRL, and are 3D printing a modified Hapkit handle that will slide onto the D-shaft. This will allow us to build the main pin joint between the pool cue and the modified Hapkit handle tonight and begin hardware calibration for pool cue position. This pin joint is our method of translating the rotation of the Hapkit handle to the linear motion of the pool cue striking a billiards ball. This linear motion is going to be constrained by a concentric cylinder around the pool cue as depicted in the CAD. Our next steps for the hardware include planning cable management. At the moment, we want to employ three vibration coin motors inside the handle to create the sensation of a ripple of vibration through the pool cue after striking the ball. The vibrational motors will activate sequentially, starting further away from the user's hand and ending at the hands of the user. We are currently unsure how this sequential vibration will feel to a user, so we are making a prototype that is very easy to manipulate and does not necessarily hide any wires. We are keeping in mind our pin layout as we add components like this and still have plenty of available pins. Our hardware goal was to have a CAD assembly and begin rapid prototyping, and we are meeting this regardless of starting a day or two later than we wanted. Attach:Macias_CAD1.jpg Δ

For software, we have been exploring the idea of state transitions. All of the code we've written for this class so far is looking at handle position, and applies force continuously when the handle is in a certain range of locations. We instead only want force applied to the handle the first time the pool cue hits the ball location, and then no longer be in a state that applies force to a particular handle position. This will help the haptic display show the ball has been moved away. Upon collision, we want a pulse of force, likely a spike and exponential decay of force from the motor. Immediately after this force, we want to trigger the sequential vibrating motors as if the pool cue has lingering vibration after the shot. This may only be a sensation you experience if you strike a ball poorly, but we are building this off of personal experience. For the demo, we believe we will situate our hardware in front of a screen so users are looking down the pool cue and at the virtual ball directly in front of them. Our graphics are going to be 2D, but imply depth via perspective lines. After the ball is struck, the ball (circle) will become smaller and its y-coordinates will increase. This will give the perception that the virtual ball is traveling away from the user. Also, we are exploring the idea of seeing the tip of the pool cue on the bottom of the screen as well. We think we can achieve this by creating a trapezoid and changing the leg and base dimensions as the pool cue moves towards the ball. This will likely be important for the user to see the point of contact and predict the force they will experience from our motor. The forced perspective and exponential decay pseudocode is depicted here:

Forced Perspective:

// Sphere parameters int sphereX = 4; int sphereY = 4; int sphereRadius = 1;

// Viewpoint parameters const int viewPointX = 4; const int viewPointY = 4;

void updateSphereRadius() {

  int dx = sphereX - viewPointX;
  int dy = sphereY - viewPointY;
  float distance = sqrt(dx * dx + dy * dy);

  sphereRadius = max(1, 3 - distance / 2); // Adjust the formula as needed

}

void setup() {

  // put your setup code here, to run once:

}

void draw() {

  // Update sphere radius based on distance from viewpoint
  updateSphereRadius();

  // Draw the sphere (as a filled circle for simplicity)
  for (int i = -sphereRadius; i <= sphereRadius; i++) {
    for (int j = -sphereRadius; j <= sphereRadius; j++) {
      if (sphereX + i >= 0 && sphereX + i < 8 && sphereY + j >= 0 && sphereY + j < 8) {
        if (sqrt(i * i + j * j) <= sphereRadius) {
          circle(sphereX, sphereY, sphereRadius);
        }
      }
    }

}

Collision Forces:

    //Serial.println(dxh);
    if (yh > prevMassY) { force = 0; }
    float vdif = cueV - vh;

    momentum_i = dyh * h_m + cueV * cueM;
    float cueV_new = (cueM - h_m) / (cueM + h_m) * cueV;

    force = (cueV_new - cueV) / (0.001);   // time step
    cueX = cueX + 0.5(cueV*cueV_new)*0.0015;
    cueV = cueV_nem;

    Serial.println(cueX, 5);

Our goal was to have some baseline graphics code at this stage, and we have pseudocode at the moment so we might be a bit behind on this currently. Instead we've put more thought into exponential collision force equations and have written pseudocode for that as well. Only a slight deviation from plan, and likely very close to the pace we hoped for.

We are about where we expected to be at this checkpoint: hardware is mostly figured out and initial prototypes are made/purchased/printed. However, we are splitting the work differently than we expected: Anthony is leading hardware/documentation, Sebastian is leading C++ code, and Carlos is leading the graphics. The biggest challenge we currently face is calculating the new damping that will exist in our system. We have done what we can to reduce the friction in the pin joint between the handle and Hapkit, but the thickness, infill, and length of the handle will affect the magnitude of sensation we can deliver to our users. We think it will likely be fastest if we create a prototype that does not modify the Hapkit motor to see if the sensation we can achieve easily will be satisfying.

Checkpoint 2

For the second checkpoint of the project, we wanted to get our graphics up and running and edit some previous code in order to better reflect the dynamics we are trying to simulate. The virtual environment of a pool table accompanied by a pool cue and cue ball was generated in Processing. The movement of the pool cue is controlled by the Hapkit using the same 1 DoF mapping from previous assignments. Changes to the code were made from the spring damper simulation to actually reflect an impulse force. The force felt by the user (the force delivered by the motor) is now dependent on the acceleration of the pool cue and its impact on the cue ball. Since there is no longer a virtual spring in the simulation, the spring force from the dynamic equation for the cue ball goes away. Instead, the damping force and the user force from the user pushing the cue into the ball make up the forces felt by the cue ball. This means that the faster that a user moves the hapkit handle into the cue ball, the faster the ball will roll away and a stronger force is felt by the user. The damping force that remains is meant to simulate friction which means that no matter the force applied to the ball by the user, it will eventually come to a stop.

Otherwise there is still progress to be made to create a more accurate virtual environment in the way that the cue ball should be bouncing off the walls of the pool table, which is yet to be programmed. Also, there could be a more accurate dynamic simulation for the force felt by the user considering the perfectly elastic collision that we are simulating.

Other features that are still in development are vibrotactile features in the custom hapkit handle that we will develop will allow for a more realistic feel of impacting a cue ball from using a cue, since the tip of the cue is actually a distance away from the user's palm/hand. These vibrations will travel up the handle and better represent the sensations that one would feel when striking a cue ball during a game of pool.

Video of virtual environment being interacted with. https://youtu.be/IY2jDzdLrfA