Trum-Pal: A Haptic Aid for Musical Practice

Project team members: William Heap, Ilana Cohen, Carter Hughes, Huy Tran

The Trum-Pal is a haptic musical training device designed to help users learn trumpet fingering and timing through synchronized vibrotactile, kinesthetic, and visual feedback. Inspired by rhythm-based games such as Guitar Hero and prior research demonstrating the benefits of haptic guidance in musical learning, the system combines three fingertip-mounted eccentric rotating mass (ERM) motors, force-feedback trumpet valves actuated by linear solenoids, and a custom Unity-based game interface. Users receive tactile cues indicating when to press (vibrotactile cues) and release (kinesthetic cues) the specific valves while interacting with a visual note-tracking system that provides real-time scoring and performance feedback. The completed device was successfully demonstrated during the ME327 Open House 2026, where 25 participants interacted with the system and provided overall positive feedback regarding both the haptic sensations and the trumpet-inspired gameplay. The project demonstrates the feasibility of integrating multimodal haptic/visual feedback into musical practice and provides a foundation for future studies to further investigate the effects of haptics on musical motor learning and timing accuracy.

Introduction

The Trum-Pal is a musical practice aid designed to teach users the fingering and timing required to play songs on the trumpet through real-time haptic and visual feedback. The system prompts the user to press specific trumpet valves by applying ERM vibrations to one of the user’s three fingertips. Once the note duration is complete and the user is still pressing the valve, they will feel an upward force from a linear solenoid actuator, prompting them to remove their finger. Simultaneously, the user follows a custom virtual interface in which notes move across the screen, providing both timing guidance and performance accuracy feedback. The addition of haptic stimuli to traditional training methods, which are limited to visual and auditory feedback, provides another means of conveying information to the musician without interfering with their natural playing motion. Through this project, we aim to investigate the question: Can haptic stimuli, in addition to visual aids, help expedite musical learning and improve timing accuracy?

Our project was inspired by rhythm-based games such as Guitar Hero, which led us to explore whether incorporating haptic feedback could further enhance a user’s learning experience and performance. Additionally, previous research has suggested that combined tactile and visual feedback may improve the effectiveness of musical instruction [1,2], motivating us to investigate this relationship further through an interactive and engaging platform.

The primary educational objectives of this project were to create an original, dynamically responsive, and sensory-compelling haptic device while learning how to effectively synchronize multiple forms of feedback in an intuitive way for the user (kinesthetic, vibrotactile, and visual). We also aimed to gain experience developing a virtual game interface and integrating it with a real-time physical device and its associated sensors and actuators. More broadly, we sought to evaluate and support prior findings indicating a positive correlation between haptic feedback and improved timing and learning accuracy during practice of a musical piece.

Background

There has been significant prior research exploring the use of haptic feedback as a tool for musical education and motor learning. Existing work has investigated both vibrotactile and kinesthetic haptic systems for improving timing accuracy, rhythm recall, and overall musical performance. These studies strongly motivated the development of the Trum-Pal and informed many of our design decisions.

One particularly relevant work is Haptic Tutor – A Haptics-Based Music Education Tool for Beginners by Tom et al. [1]. This paper presented a wearable vibrotactile system designed to help beginner drummers learn rhythmic patterns using wireless ERM and pager motors. The researchers compared traditional visual instruction against two forms of haptic feedback: a short pulse and an exponential “ramp” vibration profile. Their experiments showed that the ramp-style haptic cue yielded the best timing accuracy and lowest stroke error, suggesting that users benefited from anticipatory tactile information prior to the desired note timing. The paper also highlighted practical considerations such as lag compensation, calibration, and the need to ensure consistent physical contact between the actuator and the user. These findings strongly influenced our project, particularly our emphasis on timing synchronization and intuitive haptic prompting.

Another important study was Haptic Guidance Benefits Musical Motor Learning by Grindlay (2008) [2], which explored the use of kinesthetic haptic guidance for rhythm training. Participants learned rhythmic drum sequences under three conditions: audio-only, haptic-only, and combined audio plus haptics. The haptic system used a servo motor with PD control to physically guide drumstick motion. Results showed that haptic guidance significantly improved both timing recall and velocity accuracy, with the combined audio-and-haptic condition yielding the lowest overall error. The study reported approximately an 18% reduction in timing recall error when haptic feedback was incorporated into training [2]. This work provided strong evidence that haptic guidance can improve motor learning in musical tasks, directly supporting the motivation behind the Trum-Pal system.

Several papers specifically explored haptic guidance for wind instruments, which is especially relevant to our trumpet-based project [3, 4]. ShIFT: A Semi-Haptic Interface for Flute Tutoring introduced a flute-training system that used servo motors to physically guide users’ fingers into correct fingering positions as they learned musical pieces. The authors found that the system improved learning speed by roughly 30% compared to video-based learning alone [3]. The system demonstrated that direct physical guidance can reinforce correct fingering patterns and reduce user error during practice.

Building on this work, Zhang et al. developed Adaptive Multimodal Music Learning via Interactive-Haptic Instrument, an adaptive haptic flute interface that dynamically adjusted the level of guidance provided to the learner [4]. Their system incorporated a clutch mechanism that allowed the device to selectively engage or disengage haptic control, enabling a transition from passive learning toward more active user participation. The tutoring system included multiple guidance modes, ranging from strict positional enforcement to feedback that only corrected mistakes. Experimental results showed that this adaptive learning strategy improved learning speed by 45.3% while significantly reducing “forgetting chance” from 43.75% to 8.25%, a reduction of around 86% [4]. This work demonstrated the importance of balancing guidance with user autonomy; rather than fully constraining user motion, we are choosing to guide them in the correct direction at a note’s onset and finish.

Additional research has explored collaborative and bidirectional musical haptics. MoveMe: 3D Haptic Support for a Musical Instrument by Fujii et al. (2015) investigated a theremin-training system in which expert performers could record motions that were later replayed to guide beginners. The system also supported bilateral interaction, allowing an expert and learner to physically feel each other’s movements in real time [5]. Similarly, more recent work has explored wearable haptic technologies and exoskeleton-style systems that allow bidirectional haptic communication between teachers and students during musical instruction [6]. These systems suggest that haptics can serve not only as a timing cue, but also as a medium for transmitting physical intuition and motor behavior between performers.

Collectively, these prior works demonstrate that haptic feedback can improve timing accuracy, motor learning, and user engagement in musical education settings. They also reveal several important design considerations, including actuator timing, latency compensation, user acclimation to tactile stimuli, and the balance between intuitive and intrusive feedback. Our project builds on these findings by combining vibrotactile fingertip feedback with a rhythm-game-style visual interface to create a lightweight, interactive trumpet-learning aid focused on timing accuracy, note association, and musical motor learning.

Methods

System Overview

The Trum-Pal system consists of a physical, trumpet-like device that users interact with, three ERMs mounted to the user’s hand, an electronics control system, a user interface, and a software control system. To use the Trum-Pal, participants hold the device with their right hand, with their three middle fingers placed on the trumpet valves. An ERM is strapped to each of their three fingers on the top side of the middle phalanx.

The user then interacts with a GUI on a laptop to start the training software. While the training program is running, discs representing musical notes traverse the screen from left to right. When the discs reach the trumpet visual, the user is instructed to press the corresponding valve. When a solid line extends from the disc, the note must be held (the valve depressed), until the end of the solid line reaches the trumpet. In the upper right, a health bar for the user is displayed, which decreases as the user misses notes. Additionally, a scoring system awards the user points for playing notes correctly, with multipliers for playing multiple notes in a row. This training program heavily mimics common video games such as “Guitar Hero” and “Dance Dance Revolution”, allowing most users to easily understand the program.

As the training program runs, before a note needs to be played, the user feels a ramping vibration from the ERM on the finger corresponding to the valve to be depressed. Additionally, before a note is released, the user feels a ramping force from the valve that attempts to lift their finger. These stimuli were intended to provide the user with a warning as to what actions to perform next, and help them build muscle memory for the correct fingerings needed to play a given song.

Mechanical Subsystem

The system's mechanical elements consisted of a 3D-printed facsimile trumpet, actuators for haptic feedback, sensors for detecting valve presses, and straps for mounting the ERMs to the user.

Six actuators were used in the system. To provide force feedback through the valves, solenoids (uxcell) with a 12V nominal operating voltage, 25 N force output, and 10 mm stroke length were used. Three ERMs (Hapkit) were used to provide vibrational feedback to the user’s fingers. Three time-of-flight sensors (Pololu VL6180X) were used to track how depressed each valve was, determining whether a valve was being pressed.

The facsimile trumpet was printed in PLA and served as a case for the valves, solenoids, and time-of-flight sensors. The valves were made by epoxying a small cap to the solenoid’s plunger. To mimic the feel and operation of a real trumpet, a compression spring was mounted to the bottom end of the solenoid’s plunger, which passively pushed the valve up. Additionally, a hard stop was inserted between the plunger end and solenoid body to ensure the solenoid remained within its more linear operating region. The time-of-flight sensor was screwed into the bottom of the case, spaced outside the sensor’s minimum detection range. Each sensor measured the valve’s displacement with a resolution of 1 mm.

The three ERMs were mounted to the user’s fingers using velcro loops. The ERMs were securely glued into pouches sewn into the velcro loops, while a wrist strap provided a location to route the ERM power supply cables.

Electronics Subsystem

To control the various actuators and collect sensor readings, a robust electronics setup was required. The low level data collection and actuator command signals were provided by a microcontroller (PJRC, Teensy 4.0). The microcontroller communicated over serial through a USB cable with a laptop. The microcontroller communicated with the time-of-flight sensors over I2C, using an I2C multiplexer (GODIYMODULES, TCA9548A) to handle all three sensors sharing the same I2C address.

To control the ERMs, the microcontroller provided PWM signals to two dual-direction motor drivers (Pololu, TB6612FNG), and to control the solenoids, it provided PWM signals to three single-motor driver boards (Pololu, DRV8876). Power was supplied to the actuators with two power supplies.

Software Control System

To provide visual feedback alongside the haptic feedback from Trum-Pal, we developed a rhythm game in Unity 6.4 titled trum-Pal triumph!. The game was inspired by Guitar Hero, but instead of pressing buttons on a controller, the user interacts with physical trumpet valves. During gameplay, notes move toward three valve targets on the screen, and the user must press the correct valve fingering at the correct time. In parallel, vibrotactile feedback from eccentric rotating mass (ERM) motors attached to the user’s fingers cues them to prepare for the next set of incoming notes.

The Unity software controls the visual interface, gameplay logic, valve animation, scoring, and communication with the Teensy microcontroller unit (MCU). Unity organizes this behavior through C# scripts attached to objects in the scene. Each script controls a specific part of the game, such as spawning notes, moving notes, reading valve input, updating the score, or sending haptic commands to the hardware.

The main game flow begins with NoteSpawner.cs, which creates notes from randomly generated trumpet fingerings. After each note is spawned, FlyingNote.cs controls its movement from the spawn location to the target valve location. The note motion is based on time rather than frame count, which helps the visual movement remain synchronized with the rhythm of the game.

Player input is handled through TeensySerialInput.cs. This script opens a serial connection with the Teensy MCU at 115200 baud and reads incoming sensor data. The Teensy sends time-of-flight distance measurements, sensor channel information, solenoid duty values, and ERM duty values. Unity then processes the time-of-flight distance data and converts it into stable valve press states.

To reduce false inputs, TeensySerialInput.cs applies several filtering steps before accepting a valve press. These include startup calibration, press thresholds, hysteresis, debounce filtering, out-of-range rejection, and live re-baselining. Together, these steps help prevent sensor noise from causing incorrect scoring or jittery valve animation.

The core scoring logic is handled by RhythmGameManager.cs. The game represents trumpet fingerings using a bitmask system. Valve 1, Valve 2, and Valve 3 are each assigned a separate bit, which allows both single-valve and multi-valve fingerings to be checked efficiently. For example, pressing Valves 1 and 2 together creates a combined valve mask. The rhythm manager compares the current valve mask with the fingering required by each note. If the user presses the correct fingering within the timing window, the note is scored as a perfect or good hit. If the note passes the timing window without the correct input, it is counted as a miss.

Once the game determines the user’s performance on a note, the haptic feedback system responds through HapticFeedbackManager.cs. During gameplay, Unity sends command strings to Teensy to trigger different haptic events. For example, Unity can send a pre-cue command before a note reaches the target, a hit command after a successful note, a hold command during a sustained note, a miss command after an error, or an emergency shutdown command. The emergency shutdown command can be triggered manually by pressing CTRL + SHIFT + X on the keyboard. These commands allow the Teensy to activate the ERM motors and solenoids at the correct moments during gameplay.

The Teensy firmware was written and uploaded using the Arduino IDE. The firmware reads the time-of-flight sensors, sends valve telemetry to Unity, receives haptic commands from Unity, and controls the ERM motors and solenoids. To maintain responsiveness, the firmware uses non-blocking timing with millis() rather than blocking delays. This allows the Teensy to continue sending sensor data while haptic feedback is active.

In addition to real-time gameplay control, the Unity game included a built-in leaderboard and data collection feature to evaluate user interaction during the demo. After each game session, the software recorded the participant’s score and saved the leaderboard data to a .csv file. Users were given the option to enter their name on the leaderboard. If a participant did not want to enter a name, the entry was manually listed as “Unknown.” This allowed the team to preserve user anonymity while still collecting score data from each trial.

The leaderboard and data export system were accessed through the software’s debug interface. This interface was managed through DebuggingSceneManager.cs. Pressing the debug game button redirected the game managers to a separate Unity scene defined as the debug page. Although this scene was originally designed to support several debugging features during Teensy integration, the final MVP focused on data collection and leaderboard export. To prevent accidental changes during the demo, the data export page was password-protected. Entering the password 52413 granted the team access to the data-saving functions. From this page, pressing the “Reset Leaderboard + New CSV” button saved the current leaderboard as a `.csv` file on the host computer and reset the leaderboard for future use.

Several additional scripts support the visual interface and user experience. TrumpetValveAnimator.cs moves the virtual trumpet valves based on the filtered valve input, while NoteGlowPulse.cs creates visual hit effects. GameplayUIFeedback.cs updates the score, health, combo, feedback text, and game-over screen. Additional support scripts handle score saving and leaderboard display, including LeaderboardStore.cs, GameOverLeaderboardSubmitter.cs, and LeaderboardRowShine.cs. Other scripts support hardware and debug controls, including UnitySolenoidTester.cs and DebuggingSceneManager.cs. Home-screen behavior is managed by scripts such as HomeScreenManager.cs, HomeTrumpetDancer.cs, ButtonHoverGlow.cs, TitleGoldGlowWave.cs, and SilverTextShine.cs. Emergency shutdown behavior is supported by EmergencyAbortController.cs, GameAbortState.cs, HapticFeedbackManager.cs, and TeensySerialInput.cs.

Altogether, the Unity software and Teensy firmware create a closed-loop interaction between the game and the physical Trum-Pal device. Unity provides the visual rhythm task and scoring system, while the Teensy handles real-time sensing and haptic actuation. This allows trum-Pal triumph! to function as both a training game and a control interface for the haptic trumpet valve system. All code used to develop and implement the user interface and Teensy communication system is available in the public GitHub repository: https://github.com/mhuytran/Haptics-ME327-Project.

Solenoid Characterization

To analyze the performance of the device's solenoids, a force-displacement plot was collected at 12V (nominal operating voltage) and 6V:

Where 0 mm of displacement is when the solenoid is fully actuated. The solenoids exhibit clear nonlinear behavior that worsens near the fully actuated location. Additionally, the solenoids have a very high “startup force,” i.e., the amount of force required to move the solenoid at all. These values were 39.4 N at 12 V and 26.0 N at 6 V (not shown on the graph). This implies that the non-linearity is particularly pronounced in the range 0-2 mm. Therefore, we limited the solenoid’s degree of freedom to the latter 7 mm of actuation (~3-10 mm) to approximate linear behavior.

Dynamics and Stability Analysis

The solenoid-finger system can be modeled as above. The finger and solenoid positions, xh and xv, can be defined at the same location because they are in direct contact. The equilibrium positions are when the solenoid is at its maximum height. bv, kv, and fv are the effective damping constant, spring constant of the external spring, and input force of the solenoid due to the input voltage. While the solenoid's applied force is nonlinear with position, the overall design limits the solenoid plunger's travel to keep it within the device's approximate linear regime as discussed above. For defining the system, fv and fh (the total force exerted on the solenoid due to the finger) are treated as inputs, and xv and d/dt(xv) are treated as outputs. Therefore, the equation of motion for the solenoid is:

Since the force due to the finger has been collapsed to a single term, the effective spring and damping constants do not appear in the equation. The state-space system is defined as such:

The eigenvalues of the A matrix are given by:

Therefore, the system is inherently stable due to the negative real component of the eigenvalues. Measured/cited values for kv, mv, and bv are 285 N/m, 0.05 kg, and 3 Ns/m, which gives a response as follows:

This response accounts for the valve's limited stroke length (limits of 0mm and 7mm). This represents a 5.5 N input from the finger at t=0, a 4 N input from the solenoid at t=250 ms, and the release of the finger at t=500 ms (representing a 250 ms reaction time for the user to release the valve). 5.5 N is a rough estimate of how hard a person might press the valve, and 4 N is the approximate average output of the solenoid when engaged. This demonstrates that the user receives a measurable response from the device and that there are no intense oscillations or unsatisfactory behavior.

Results

The Trum-Pal haptic device and Unity-based user interface received strong engagement during the ME327 open house. The system was demonstrated as an interactive trumpet-based rhythm game in which users pressed physical valves while receiving haptic feedback from the device. Overall, users responded positively to both the game concept and the valves' force-feedback behaviors.

A total of 25 individuals played the game during the open house. The recorded scores showed a wide range of user performance. The highest score recorded was 82, while the lowest score recorded was 1. This spread is expected because users varied in their familiarity with rhythm games, musical timing, and trumpet-style valve interaction. The range of scores also showed that the game created a measurable performance challenge rather than producing identical outcomes for all users.

Qualitative feedback from participants was highly positive. Several users commented on the novelty of combining a rhythm game with a physical haptic interface. One participant, Chiara, stated that “it was great.” Ava commented, “I do like the force feedback,” which directly supports the value of the haptic actuation component. Lester described the system as “really cool,” reflecting the general engagement generated by the interactive demo. Vicky noted that “the valves almost feel like the trumpet valves themselves,” which was especially meaningful because one of the goals of the project was to create a valve interface that felt physically connected to trumpet performance.

These results suggest that the Trum-Pal system successfully achieved its MVP goals. The device supported repeated user interaction during the open house, collected quantitative score data, and received positive qualitative feedback from users. The leaderboard system also provided a simple but effective way to document user performance across trials. Most importantly, user comments indicated that the haptic feedback was noticeable, enjoyable, and relevant to the trumpet-inspired interaction. Future work should build on these results by collecting more structured survey responses, measuring timing accuracy, and comparing gameplay performance with and without haptic feedback.

Future Work

A possible continuation of this device would be to add songs to the game already implemented. This would not only make the device more enjoyable but also support practice more directly, since the trumpet fingerings corresponding to the notes in a given melody could be included. However, adding real songs also introduces a new design issue: some trumpet notes do not require pressing any valves. Therefore, additional feedback or input features would need to be added to represent this case, such as a “blowing” input that detects when a user blows into a fake mouthpiece.

In addition to expanding the game's musical content, another important improvement would be to make the software control system more robust to environmental noise and startup calibration errors. During demo day, a major inconsistency occurred in the valve animation: the virtual valves sometimes moved on their own, even when the user was not intentionally pressing them. This was most likely caused by users resting their fingers on or near the physical valves during the time-of-flight sensor calibration period, which occurred shortly before the game started. As a result, the ToF sensors may have calibrated an incorrect “resting” distance, causing later readings to be interpreted as valve presses even when the valves were actually released.

To address this issue, the proposed fix would focus on improving the TeensySerialInput.cs, ValveInputState.cs, and TrumpetValveAnimator.cs scripts. First, the game should pause on a short calibration screen before gameplay begins. This screen would instruct the user to keep their fingers off the valves while the system calibrates. During this period, the software would collect several ToF readings while the valves are open and calculate a stable resting distance for each valve. If the readings are too low or too high, missing, or inconsistent, the game should reject the calibration and prompt the user to retry rather than starting with unreliable sensor values.

After calibration, the software should not treat every small ToF distance change as a valve press. Instead, it should use separate press and release thresholds, creating hysteresis, along with debounce timing. In practice, this means a valve would need to remain past the press threshold for a short time before the game accepts it as pressed. Likewise, it would need to rise past the release threshold before being considered released. This would prevent random sensor noise from moving the virtual valves or affecting scoring.

A live re-baselining feature would also help recover from bad startup calibration. If the game later detects that a valve has clearly opened but the stored rest distance is inaccurate, it could update the baseline instead of continuing to use the bad calibration data. Finally, valve animation should be gated by the filtered input state, so the virtual valves move only when a press is confirmed, not whenever the raw ToF value fluctuates. This would make the interaction more reliable during demos and reduce false valve animations caused by noisy or miscalibrated sensor readings.

Beyond improving the current trumpet-based system, future work could also investigate the development of similar devices for other instruments. One benefit of this device is that it maps one-to-one with the physical construction of a trumpet: the index finger always stays on the first valve, the middle finger on the second valve, and so on. Other instruments that could follow this approach include tubas, euphoniums, French horns, cornets, and flugelhorns. Instruments such as saxophones, clarinets, flutes, and bassoons would be more difficult because a single finger can press multiple keys. For example, the left pinky finger on a saxophone can press four different keys for different notes. Therefore, future work could investigate different types of haptic cues for fingers that correspond to multiple possible notes.

Lastly, it could be worthwhile to develop a device with similar goals but a very different mechanical construction. For example, a trombone plays different notes by moving the entire arm and wrist, so a force-feedback device for trombone training would need to safely apply forces across those degrees of freedom.

Files

BOM: https://drive.google.com/file/d/13ovszNb32q5Fs8SRGuHetppKeVuUwqDd/view?usp=sharing

Unity models, Unity Code, and Teensy Code, STLs of the 3D printed parts used to fabricate the Trum-Pal can be found in the public GitHub repository: https://github.com/mhuytran/Haptics-ME327-Project/tree/main

References

[1] A. Tom, A. Singh, M. Daigle, F. Marandola, and M. M. Wanderley, "Haptic Tutor - A haptics-based music education tool for beginners," in Proceedings of the 15th International Workshop on Audio and Haptic Interaction Design (HAID 2020), Montreal, QC, Canada, Aug. 2020, Art. no. hal-02901205.

[2] M. Grindlay, "Haptic guidance for rhythmic motor learning," in Proceedings of the 16th Symposium on Haptic Interfaces for Virtual Environment and Teleoperator Systems (HAPTICS '08), Reno, NV, USA, 2008, pp. 401–407. doi: 10.1109/HAPTICS.2008.4479986.

[3] G. Xia, C. Jacobsen, Q. Chen, X. Yang, and R. Dannenberg, "ShIFT: A Semi-haptic Interface for Flute Tutoring," arXiv preprint arXiv:1803.06625, 2018. doi: 10.48550/arxiv.1803.06625.

[4] Y. Zhang, Y. Li, D. Chin, and G. G. Xia, "Adaptive Multimodal Music Learning via Interactive-haptic Instrument," in Proceedings of the 2019 International Conference on New Interfaces for Musical Expression (NIME '19), Porto, Portugal, 2019, pp. 140–145.

[5] K. Fujii, S. S. Russo, P. Maes, and J. Rekimoto, "MoveMe: 3D haptic support for a musical instrument," in Proceedings of the 12th International Conference on Advances in Computer Entertainment Technology (ACE '15), Johor Bahru, Malaysia, 2015, pp. 1–8. doi: 10.1145/2832932.2832943.

[6] A. Michałko, N. Di Stefano, A. Campo, and M. Leman, "Enhancing human-human musical interaction through kinesthetic haptic feedback using wearable exoskeletons: theoretical foundations, validation scenarios, and limitations," Frontiers in Psychology, vol. 15, Art. no. 1327992, 2024. doi: 10.3389/fpsyg.2024.1327992.

---

Appendix: Project Checkpoints

Checkpoint 1

Preliminary Prototyping and Proof of Concept:

Our group was uncertain about the effectiveness of our proposed setup, and so we constructed a simple functional prototype to test the concept and potential design options.

Our prototype consisted of two eccentric rotating mass (ERM) motors, two different solenoids, and the Hapkit board.

The two solenoids were powered by the hapkit’s two motor drivers. Though this meant the solenoids could only be powered with 5V even though they were nominally 12V solenoids, the actuation force generated was adequate for testing and reduced system complexity. The ERMs were powered directly from the hapkit board.

To test our proposed project concept, one member of our group had ERMs taped to their fingers. Based on the results of prior work (https://hal.univ-lorraine.fr/HAID2020/hal-02901205v1), we initially tested a ramping vibrotactile sensation over a short period followed by a short hold at maximum intensity to signify when a valve depress should occur.

The two solenoids we found in the Charm Lab were compared as potential valve options. We found that one was too weak and heated up quickly, while the other had neither issue. The solenoid was used in the prototype to signify when the user should release the note. After the ERM ramp was completed, the user depressed the valve (solenoid), and held it for some amount of time, the force exerted by the solenoid would ramp quadratically. This was also found to be effective at conveying information to the user.

Finally, we tested the use of haptics on two fingers simultaneously, and with overlapping stimuli. We found that stimuli was still helpful and parseable within a reasonable amount of overlap, and that having stimuli going to both fingers was not overly confusing. While ideally we would like the final device to utilize three fingers, we did not have a third solenoid to test with and are convinced from the two finger test that the concept has a reasonable chance to scale well to three fingers.

Initial Dynamics Modeling:

The valve was modeled as a mass m_v with forces from the solenoid’s spring k_v*x_v, natural damping b_v*dot{x}_v, input force as a function of voltage (and potentially position, ignored for the current analysis) f_v, the finger’s natural “spring” force k_h*(x_h-x_v) and damping force b_h*(dot{x}_h-dot{x}_v). This therefore gives equations of motion as follows:

The state variables to fully describe the system are x_h, dot{x}_h, x_v, and dot{x}_v. The output variables are x_v and dot{x}_v. The input variables are x_h and f_v. Therefore, the state-space model is as follows:

We found that the system is not inherently stable, so careful consideration must be given to both the full range of possible user inputs and the selection of the solenoids.

Preliminary Arduino Code for Checkpoint 1:

To test our proposed setup, an Arduino sketch was developed to actuate two ERM-motor pairs sequentially, with distinct ramp-up and ramp peak hold times for the ERM and solenoids respectively. The duty cycles for the solenoids and ERMs were tuned. The sketch enabled us to test our concept on one of our team members (Ilana) to ensure that the concept is feasible. Our Arduino sketch code for Checkpoint Week 1 is uploaded on on a public GitHub repository: Checkpoint Week 1 Code

Next Steps:

Following the successful proof of concept, we will now transition to our final system design. We will CAD and manufacture a mount and valve caps for the solenoids, ergonomic finger mounts for the ERMs, and a robust, soldered wiring network between the actuators, motor drivers, and microcontroller. We will also source and integrate a time of flight or similar sensor to collect data on when the user plays a note.

Additionally, we will improve the control system code to enable easier refinement of experimental parameters, potentially using object-oriented programming to abstract the ERM-motor pairs, and the programming of haptic stimuli corresponding to music. In combination with this, we will begin programming a user interface using Unity engine which communicates with the microcontroller to display to the user what notes they should play and their performance. Also, we will integrate audio feedback as each note is played.

Finally, we want to expand on our analysis of the system dynamics to look at system stability under different operating and design conditions. This could be used to influence our physical system’s gain selection, or we can use our physical device to validate our analytical modeling.

Checkpoint 2

Solenoid Selection and Characterization:

Due to shipping delays with our originally selected solenoid, our group had to pivot to a different actuator choice for our project. After evaluating several additional options, we finalized the use of a Uxcell DC 12V 25N force solenoid, sourced from Amazon. While this delayed some aspects of integration and testing, the selected solenoid demonstrated significantly better performance in linearity (though still non-linear), stroke length, and force application than the other alternatives we tested.

One additional solenoid option (Fielect DC 12V 15N Tubular Solenoid) was evaluated and ultimately rejected because it exhibited even more drastic nonlinear force behavior and heated up excessively during operation. In contrast, the selected Uxcell solenoid provided more consistent actuation behavior while operating at a more acceptable temperature and current draw, in addition to offering sufficient stroke length and pushback force, and was therefore deemed more suitable for continued development.

Following actuator selection, we began detailed characterization of the solenoid and developed an initial CAD model to integrate it into the final device housing. During characterization, we identified a highly resistant and nonlinear region at the beginning of the solenoid stroke. To avoid operating within this undesirable regime, we decided to mechanically limit the stroke length of the actuator. Our proposed design includes a mechanical stop which catches on the metal ring at the bottom of the plunger assembly, preventing full retraction into the nonlinear high-force region.

Based on this characterization, we determined that approximately 9 mm of the lower stroke region should be excluded from operation. With this restriction, the usable stroke length is approximately 18 mm. Additional measured dimensions include a total internal plunger length of 96 mm and a casing height of 65 mm. The actuator is currently operated at 12V and approximately 1.3–1.4 A.

Thermal management also emerged as a significant design consideration during testing: the solenoid we selected still gets quite hot after being on for a given period. To address overheating concerns, our group began brainstorming methods to improve heat dissipation within the system. One proposed solution involves wrapping the body of the solenoid in copper foil to encourage heat spreading and improve conductive heat transfer away from the actuator body. We determined that the final enclosure design must incorporate substantial ventilation to promote airflow and reduce heat buildup during repeated operation.

Uxcell DC 12V 25N force solenoid:

Deconstructed solenoid showing the plunger and outer body:

Electronics Integration and Sensor Development:

This week, our group also began hardware integration of the sensing and control electronics. The ToF sensors, microcontroller, and motor drivers were acquired and partially soldered to their corresponding boards, with approximately two-thirds of all the required soldering work completed at the time of this checkpoint.

The microcontroller and motor drivers board was fully soldered and assembled, and we ran preliminary testing today; this solenoid control board is functional. However, additional validation and testing of the ToF sensors is still required before final integration into the system. Teensy-based hardware control code for the sensing boards has not yet been completed and will be developed in the coming week.

Soldered solenoid driver board:

Board testing setup:

Vibrotactile Attachment Design:

Development also continued on the wearable vibrotactile system used to convey musical timing information to the user. This week, the final concept for the ERM attachment system was selected and initial construction began.

The current design uses small sewn velcro pouches mounted at the fingertips to securely hold the ERM motors against the user’s fingers while maintaining comfort and modularity. Additional wire-securing velcro straps are positioned around the lower fingers and wrist. These straps serve two primary purposes: first, to guide wiring away from the user’s playing area so that the wires do not interfere with movement or dexterity, and second, to provide strain relief and reduce stress concentrations at electrical connection points during repeated use. Velcro also provides highly customizable sizing for different users, which should improve overall comfort, fit, and reliability.

The complete vibrotactile attachment system currently consists of three fingertip ERM straps, three lower-finger wire management straps, and one wrist wire-management strap.

Full vibrotactile device attachment diagram:

ERM straps on finger tips (x3), wire-holding straps on lower finger and wrist (x4)

ERM-securing velcro strap (x3):

Wire-securing velcro strap (x3 fingers, x1 wrist):

Device Housing and Mechanical Design:

Initial CAD development of the device housing also began this week. The preliminary casing geometry has been modeled, though several major features still remain to be integrated into the final design. Planned additions include ergonomic finger holds, a spring integration mechanism at the bottom of the solenoid plungers, the mechanical stop for the plungers, and mounting features for the ToF sensor paddles.

Rough casing CAD, need to add finger holds, spring, stop, and ToF paddles:

Unity Interface and Animation Development:

Work also began on the visual interface and animation environment in Unity. A trumpet model was imported into Unity as the basis for the visual training interface. However, during implementation we encountered a challenge related to animating the trumpet valves. In order for the valves to move independently within Unity, each valve required a separate solid body rather than being embedded within the original imported geometry.

As a result, the trumpet valve components had to be redesigned in Fusion and re-exported into Unity to align properly with the existing trumpet model and enable animation of valve motion.

This portion of the project is currently being completed on a significant learning curve, as this is the first time anyone in our group has worked extensively within Unity. Despite this, Huy has successfully completed preliminary rendering and integration of the redesigned trumpet valves, and further interface development will continue in the coming week.

Unity rendering of designed trumpet valves: