Catchkit (A Fishing Game)

Project Team Members : Catherine Chen, Jennifer Lugardo, Omar Ramos, and Dominique (Domo) Toyozaki

Growing up, our team members used to play the ice fishing game on club penguin. This fond memory inspired our pursuit of a haptic fishing game where we could integrate the visuals of a fishing game along with the haptic feedback of a controller. We also thought that having haptic feed back from engaging with the virtual fish would make the game more enjoyable and bring the fishing experience to life. Finally, we really wanted to make a game because games are fun and a great way to take a break. The goal of this project is to allow a user to move our haptic handle in various speeds, which will move the fishing rod in the virtual environment. The user should also be able to feel haptic feedback when it captures the fish and the difference in viscosity in the water and air as they pull their virtual rod up to catch the fish. A successful implementation of this project would be to allow the user to play a game where they are able to catch fish based on the speed they pull their fishing rod, and so they can feel the difference of force they need to pull in water vs on air.

Introduction

With the growth of haptic feedback for video games (especially from arcade partners), fishing games with haptic feedback have become an area of interest. This is seen through Fishing Spirits, a Japanese fishing game, that has the user fish for different types of sea creatures with different rods. Depending on the rod and the size and type of sea creature the controller will give the user a different type of vibration and resistance. Additionally, Tactical Haptics released their SaberGrip in 2020 that allows users to have haptic feedback from sword fights and fishing games[4].

Our main goal was to find a way to integrate a fishing game with a haptic handle. In this case, we will use the haptic handle in order to determine the speed in which the virtual rod will move up and down to get the fish. We will also use the handle to signify when a fish is on the line and to help the user distinguish when the rod is still in the water or when it is out.

Background

In this paper, inspiration is drawn from Yun Lin from the University of South Florida constructed a haptic interface for fishing training that would allow engineering fishers to get accumulated with fishing without the long waiting periods [2]. In this study, a Novint Falcon, a 3 degrees of freedom impedance device, was used to represent the fishing rod. The fish and the hook are modeled as dynamic objects while the fishing line is a rigid object. The fish has an attraction force (modeled as a spring that makes it go to the hook once it is submerged into the water. When the fish is on the hook, the user will feel a vibration (modeled as a sinuous function with a frequency of 50 HZ) to mimic a bite and then the weight of the fish on the rod. This study also incorporated two different sizes of fishes, two different vibration durations, and periods of biting frequency that would be randomly selected to add complexity into the system to make it feel more realistic. This study received good reviews from beginners to fishing lovers, and users agreed that the haptic feedback and visuals would be helpful for new fishers.

While Lin’s paper has many similarities, this paper is focused more on looking at the intersection of fishing and gaming. Because of this, haptic feedback for other sport games also guided this paper. Looking at Morelli et al (2010), vibrotactile/audio feedback was used for exercise games, namely virtual tennis, for the visually impaired [3]. Morelli et al created a game inspired by Wii Sports tennis (called Visually impaired (VI) Tennis)in which players receive feedback through visual, auditory, and tactile cues. These cues help players understand the game and create a mental map of how the game should be played. Two versions of VI Tennis were used in the study, one providing audio/tactile cues and one with the tactile cues turned off. The most relevant finding to our paper was that the children scored significantly better with the tactile and audio version of VI Tennis compared to the audio only version. Additionally, the children expressed having a better time playing the tactile and audio version and found it easier to play.

Lastly, exploring how to simulate the relevant cues are important. This paper explored the importance of feeling the viscosity of the water in fishing. A study from Hover et al (2009) used interpolated measured data to produce haptic feedback for viscoelastic bodies and viscous fluids [1]. Using the model (Maxwell Model) and transfer function below to guide the researchers on how to model material properties like elasticity and viscosity and effects like stress relaxation in solid bodies or surface waves in case of fluids as nonlinear and mutually dependent.

The researchers used a PHANToM Desktop 1.5/3 DOF (Sensible) to record the trajectory of the probe. During the study, the user will interact with either a solid or liquid via the probe. This study is set up in a way that for the solid materials only forces resulting from deformation are recorded, and for the liquid materials only contact forces from movement are recorded. After capturing this data, the researchers used it to model the forces for a haptic simulation. When examining the liquids, the researchers found that the key interaction cue is velocity. The participants for the user feedback viscous liquid study consisted of one expert and four non-experts. The participants were first told to explore the liquid freely along the x-axis while the forces at the probe were being recorded. Then the participants were told about the fluid’s properties and given specific instructions for how to move the tool. The relative force errors between the expert user and the non-experts users were recorded and found that non-expert users would move the tool slower in the liquid than instructed.

Methods

Hardware Design and Implementation

The hardware consists of a modified version of the hapkit that we used in class, with a sector that has a range of 180 degrees rather than the approximately 80 degrees in the original. The purpose of this is to increase the motion for the user as well as prevent the user from hitting the limits of the range in order to keep the calibration accurate. The handle is much longer at approximately 4 inches for the user to grasp comfortably. We used the MR sensor on the Arduino board for position tracking along with a magnet and cable. As for the enclosure, we laser cut a box with a slot for the handle to be accessible as well as an open back to allow for clamping the device on to a surface to keep it stable when in use. The original hapkit base is mounted horizontally onto the enclosure by four screws and hex nuts on a surface in the middle of the enclosure box and attached via a shoulder screw to the sector and the handle. The whole box is secured together through fitted slots, screws, and hex nuts. The motor we use is the same motor on the original hapkit, and we also made of two of the vibrations motors mounted onto the handle. All of these are wired to and powered by the Arduino board.

Final hapkit:

Images of the modified hapkit:

Images of the total assembly:

Serial Communication Analysis

The serial communication component of this project allows the graphics to interact with the hardware to control the position of the rod. The Arduino communicates with the python code to determine the current position of the handle, which the Python code then maps to a specific location of the fishing hook on the screen. The Python code then communicates with the Arduino code when the x and y position of the handle and the x and y position of the fish match and triggers a case in the Arduino code that activates the vibration motors and the main force actuator. The python code also communicates back to the Arduino to trigger the air or water damping depending on where the handle position has mapped onto the game. The python and arduino code used to accomplish this, as well as the images and sounds used, can be found zipped in the files section.

Demonstration

Hardware and game being played:

https://drive.google.com/file/d/1oIY-KtFHbZR-mjLzhx6YenLcf3OMDgbi/view?usp=sharing

Results

After resolving timing issues between our python code and the arduino with help from the teaching team, we had a fishing game that was quite enjoyable to play! We added the Club Penguin fishing game song to the background to make it even more nostalgic, added difficulty levels, and redesigned the handle to be more comfortable to hold as well as have better housing for the vibration motors. Paired with our original idea of how the device would work, this was a really fun experience to everyone that we demonstrated our project to.

The finalized game consisted of having two environments rendered with different damping levels: low damping while in the air and higher damping while moving the fishing hook in the water. Once the fish had been caught, the fish would rotate and follow the fishing hook, the vibration motors would be activated, and the user would feel a constant force pulling them downwards. At this point, the user had 2 seconds to bring the fish up to the surface or else the fish would escape and not be added to the score. Along with varying difficulties, which increased the range of speeds that the fish could spawn with, the game was very responsive and immersive, which was a big part of our goal. Every person that we tested it with in our dorm wanted to play more than once and expressed how "cool" it was to relive a game they had played so many years ago but now in a much more interactive way.

Future Work

For future work in respect to the hardware, we would redesign the box to make it more portable and easier to access the electronics for purposes of testing and calibrating the whole system.

For future work, we would like to make some upgrades to the software. First, we would like to display the count-down timer to allow users to know how much time they have left to fish. We would also wish to change the way the game is structured, as of right now we have a set time in which the user will fish but we would like to incorporate sharks and other fish so that the user can play for an unlimited time and the duration of the game is determined by how many sharks are encountered, for example. Additionally, we would like to associate different fish or object in the water with more creative forms of haptic feedback such as different pulling forces or vibration patterns that the user can then use to discern if a fish should be caught or how many points a certain fish would be worth.

This game could be applied to other games, such as a swimming game where a user can use two controllers and feel the resistance of the water as they swim. This could also translate to paddling or anything that interferes with water and air or two surfaces that have different damping. However, we think that the best upgrade that we could make to our system is to make it very child-friendly. We wanted to do this project because of a very simple game that in some way impacted us as children, leaving us with good memories that were enough to inspire this project as adults. If we could then take what we have now and introduce small systems like these to children so they could not only have fun playing the game but also get introduced to the world of haptics and get to see hardware in action, it would hopefully leave a memory on them that they can perhaps build on in the future, just like it happened for us.

Acknowledgments

We would like to thank Allison for teaching us and introducing us to the world of haptics. We would also like to thank Allison, David, Nathan and Elyse for their continuous support and advice during this project. Furthermore, we would like the thank Nathan for his support in debugging the biggest coding issues that we encountered and all the teaching team for being understanding about our situation and allowing us to complete our project and learn from the experience.

References

[1] Raphael Hover, Gabor Kosa, Gabor Szekely, Matthias Harders. Data-Driven Haptic Rendering—From Viscous Fluids to Viscoelastic Solids. IEEE TRANSACTIONS ON HAPTICS, VOL. 2, NO. 1, Pages 15-27, 2009. https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=4752819

[2] Yun Lin. Haptic Interface for Fishing Training. 2011 Haptic Class Project Paper Presented at the University of South Florida, Pages 1-3, May 02, 2011. http://reedlab.eng.usf.edu/photos/HapticsDemos2011/2011Lin.pdf

[3] Morelli Tony, John Foley, Luis Columna, Lauren Lieberman, and Eelke Folmer. (2010a). VI-Tennis: a vibrotactile/audio exergame for players who are visually impaired. InProceedings of the Fifth International Conference on the Foundations of DigitalGames (FDG '10). ACM, New York, NY, USA, 147-154.DOI=10.1145/1822348.1822368http://doi.acm.org.proxy.bib.uottawa.ca/10.1145/1822348.1822368

[4] Feltham, Jamie. “Tactical Haptics reveals new SaberGrip for VR fishing and sword fights” VentureBeat, August 2020. https://venturebeat.com/2020/08/22/tactical-haptics-reveals-new-sabergrip-for-vr-fishing-and-sword-fights/

Files

Hardware

Final Box Enclosure Hardware CAD and Ai Files: Attach:Enclosure_Box.zip

Hapkit Components CAD Files: Attach:Hapkit_Components.zip

Electronics Enclosed

In order to transfer haptic feedback to the user, we created the vibrations felt when the user would interact with a virtual fish with two vibration motors that we connected to pins 3 and 6. We also created a sensation of dampening that is felt while the user is pulling the fish in the water vs in the air.

Image Depicting Electronic Hardware

Arduino and Python Code (including supporting images and sound)

Attach:FishingGame.zip

Appendix: Project Checkpoints

Checkpoint 1

For our first check point, we decided to use most of our time to develop the new haptic device. Our first goal was to have the cad finalized. We were able to complete the CAD designs. In total, we created 3 parts: the base, box and rod. We are also considering two designs, one where we use the box we have made and the base and the rod and the other with an easier box that we can use on the existing hap kit along with the base and rod.

Hardware Parts

Here we have included images of the various parts we have designed for our first prototype of the haptic device. The motor would sit at the bottom of the base and track the position similarly to how the hapkit tracks position. The two ends of the rod would be connected by music wire that would be guided by metal shafts inside the box and would wrap around the shaft of the motor. The rod would attach to the base via a shoulder screw. However, after some testing, we realized this design would not work as the wire would have no way to constantly stay tightly wounded.

Base

New Box

Rod

Box to Adapt to Original Hapkit

CAD Files: https://drive.google.com/file/d/1nMT_qfNW0ojY9LpSVo85OEE8IoiNarRM/view?usp=sharing Follow this link to see our STL files of the components of our hap kit handle.

The second part of the first check point was to have a code draft done for the game. We were unable to get to this point because we were having trouble deciding how to organize our code, We were able to iron out a general structure that we would like the code to follow.

Code Outline

Making window size

Outside function, determining how fast the fish will swim and how often

Loop which sets up game

Read in serial position of the rod to calculate speed and position of virtual rod

If the position of the virtual rod = of fish

- Then provide haptic feedback

As the user pulls, there will be a random number generated that will determine if the fish is able to captured or swims away

The users will also feel the difference between pulling a rod through water and pulling it through air

Once the rod feels like air, they will earn a point since they officially reeled a fish

The users will have a certain amount of time and will have to pull quickly to win

The game will keep count.

The user will have a time limit to collect as much fish as possible.

There will also be random sharks that will bite the rod and make the user lose.

The screen will prompt them to try again when they lose

Next Steps After testing this design, we realized that the pulley system would not work correctly since we didn't have a way of adding tension to the string when moved in one direction. With this reasoning, we decided to change our design and focus on reusing the current hapkit but changing the handle.

Our next step is to test and build our updated haptic device, and we hope to have it finalized by Wednesday night. We will now begin writing the code for the game and further develop code that achieves the goals outlined in the outline above. We will have the code functioning by Thursday and will work to integrate it together Saturday night. Furthermore, we will then have the weekend to work and debug.

Checkpoint 2

The final creation of the hardware was developed. It will include two vibration motors to allow the user to feel the vibration of the handle when they catch a fish. The final component missing for an aesthetic view is a box that will house the motor and bottom platform and only expose the handle, which the user will use to control the fishing rod in the virtual environment.

We also created a box that would allow us to enclose all the electronics.

We have decided to use python for the visual and graphics of the game. This section of code will interface with the serial ports of the haptic hardware.

Here is the Code for the Python Attach:pythonTempCode.pdf

This python code is setting up the window and controlling the graphics that the user will see on screen.

From this code we need to integrate the serial communication with the hardware and we need to also integrate the Arduino code. For this to work, we will need to determine how to write serial commands and pick up serial commands between the python code and the Arduino code. This code will be created and tested Saturday. The Arduino code will include signals to the vibration motors when a fish is caught and the change in force felt from the viscosity of the fish being underwater and out.