CHARM LAB Main/Teleoperation And Haptics For Surgery

Teleoperation And Haptics For Surgery

Automatic Tissue Palpation and Discrimination

Localizing tumors, as well as measuring tissue mechanical properties, can be useful for surgical planning and evaluating progression of disease. This work focuses on developing supervised machine learning algorithms that enable mechanical localization of stiff inclusions in artificial tissue after autonomous robotic palpation. Elastography is used to generate training data for the learning algorithms, providing a non-invasive, inclusion-specific characterization of tissue biomechanics. The test set is generated by approximating stiffness values from individual palpations acquired during autonomous robotic palpation. Hard inclusions are then localized from this classification of the data.

Skin Stretch Feedback

During everyday interaction with objects using a stylus-like device, we often experience both kinesthetic force feedback and cutaneous skin stretch feedback. We are investigating the use of imposing artificial skin stretch as a haptic feedback or augmentation method.

Sensory Substitution

Despite improving user task performance in some teleoperated tasks, haptic feedback is not incorporated in clinical surgical robotic systems due to stability and safety concerns. Researchers have attempted to convey this force information through other sensory channels in a technique called Sensory Substitution. We attempt to convey this force information through fingerpad skin deformation feedback, which involves fingerpad tangential skin stretch and normal skin deformation. During everyday manipulation tasks, skin deformation feedback is present and contributes to your perception of interaction forces. We hypothesize that skin deformation feedback could be a more intuitive feedback modality to replace force feedback when compared with other current sensory substitution methods such as vision and audio feedback.

We built a series of devices ranging from 1-Degree-of-Freedom (DoF) tangential skin stretch devices to 3-DoF skin deformation device that can convey both tangential skin stretch and normal skin deformation cues to the user's fingerpad. Through a series of human-subject experiments, which involves psychophysical experiments and programmed virtual environment tasks, we characterized users' ability to use our devices to perform surgically related task such as palpating for tissue stiffness and tissue suturing. Users' performance using our device are compared against that obtained using force feedback.

Sensory Augmentation

The same devices that we built for sensory substitution can be used for sensory augmentation of force feedback. Through sensory augmentation, force-feedback will provide the physical resistance while skin deformation feedback will provide the force information to the user. Such augmentation will be useful in situations when high fidelity force feedback is unattainable, such as in time-delayed teleoperation in which the time-delay causes instability in force-feedback systems, or in safety critical applications such as surgical teleoperation, in which the potential instability brought about by force feedback is undesirable.

Augmenting perception of stiffness using skin stretch augmentation

It is hypothesized that by imposing artificial skin stretch cues, we can increase the perception of stiffness perceived by a user. Such a sensory augmentation scheme will prove useful in situations in which force feedback is hard or impossible to achieve, including surgical teleoperation in which force feedback gain has to be tuned low in order to achieve stability and safety. Other situations call for higher forces to be rendered than actuators are capable of achieving. In these scenarios, skin stretch feedback could help increase the range of applications of under-powered actuators.



  • KineSys MedSim SBIR subcontract
  • U.S. Army Medical Research and Materiel Command (USAMRMC; W81XWH-11-C-0050
  • NSF NRI-Large: Multilateral Manipulation by Human-Robot Collaborative Systems (NSF; 1227406)

Click here to return to the main research page.