=Paper=
{{Paper
|id=Vol-3842/paper13
|storemode=property
|title=Smart Prosthetics in Surgery: AI-Driven Tactile Feedback Using Piezoelectric Sensors
|pdfUrl=https://ceur-ws.org/Vol-3842/paper13.pdf
|volume=Vol-3842
|authors=Pavlo Tymkiv,Aleksandra Kłos-Witkowska,Oksana Bahrii-Zaiats,Serhii Kovalyk
|dblpUrl=https://dblp.org/rec/conf/bait/TymkivKBK24
}}
==Smart Prosthetics in Surgery: AI-Driven Tactile Feedback Using Piezoelectric Sensors==
https://ceur-ws.org/Vol-3842/paper13.pdf
Smart Prosthetics in Surgery: AI-Driven Tactile
Feedback Using Piezoelectric Sensors
Pavlo Tymkiv1,∗,†, Aleksandra Kłos-Witkowska2,†, Oksana Bahrii-Zaiats3,† and
Serhii Kovalyk 1,†
1
Ternopil Ivan Puluj National Technical University, Ruska str.56, Ternopil, 46001, Ukraine
2
University of Bielsko-Biala, Willowa St. 2, Bielsko-Biala, 43-300, Poland
3
I. Horbachevsky Ternopil National Medical University, Maidan Voli St., 1, Ternopil, 46002, Ukraine
Abstract
This paper explores the innovative application of piezoelectric sensors in robotic prosthetics for
surgical systems, emphasizing their potential to enhance tactile feedback during delicate
procedures. Piezoelectric sensors can efficiently convert mechanical pressure and vibrations into
electrical signals, offering a crucial means for surgeons to feel and interpret force, texture, and other
surface characteristics in real time. The ability to generate and transmit tactile feedback through
cloud-based systems allows for the creation of a database of tactile patterns, enabling automated
recognition of specific tactile interactions during surgery. The integration of artificial intelligence
(AI) further enhances the system by learning from collected data, predicting future interactions, and
optimizing pattern recognition. Additionally, combining piezoelectric sensors with other types of
sensory inputs, such as temperature and strain gauges, allows for a multi-dimensional feedback
system. This results in an immersive experience, granting surgeons precise control over their
robotic tools. The continuous improvement of these systems through AI and data collection holds
vast potential for future developments in robotic surgery, leading to more accurate, safer
procedures and better patient outcomes. This research underscores the transformative impact of AI-
driven, multi-sensory feedback systems in enhancing the capabilities of robotic-assisted surgeries.
Keywords
Robotic surgery, robotic manipulator arm, tactile feedback, precision control, AI
1
1. Introduction
Robotic surgery marks a revolutionary leap in modern medical practice, offering unmatched
precision, minimal invasiveness, and enhanced control for complex procedures. At the heart
of this evolution is the robotic human hand, which emulates the dexterity and versatility of its
biological counterpart. One of the key technologies driving this precision is the integration of
high-performance micromotors, which enable fine control and articulation necessary for
various surgical tasks. However, an equally critical aspect of robotic surgery is the ability to
provide surgeons with a realistic sense of touch—a concept known as haptic feedback [1], [2].
This tactile feedback is essential for enhancing the surgeon’s ability to manipulate delicate
1
BAIT’2024: The 1st International Workshop on “Bioinformatics and applied information technologies”, October 02-04,
2024, Zboriv, Ukraine
∗
Corresponding author.
†
These authors contributed equally.
t_pavlo_o@ukr.net; tpavloo@gmail.com (P. Tymkiv); awitkowska@ath.bielsko.pl (A. Kłos-Witkowska);
bagrijzayats@tdmu.edu.ua (O. Bahrii-Zaiats); sergyykovalyk1984@gmail.com (S. Kovalyk)
0000-0003-1212-3107 (P. Tymkiv); 0000-0003-2319-5974 (A. Kłos-Witkowska); 0000-0002-5533-3561 (O. Bahrii-
Zaiats); 0000-0002-5334-1162 (S. Kovalyk)
© 2024 Copyright for this paper by its authors. Use permitted under Creative Commons License Attribution 4.0 International (CC BY 4.0).
CEUR
ceur-ws.org
Workshop ISSN 1613-0073
Proceedings
tissues, apply appropriate pressure, and avoid unintended damage. Recent advancements in
sensory technology have focused on integrating tactile sensors, such as piezoelectric
transducers, into robotic prosthetics [3], [4]. These sensors convert mechanical pressure into
electrical signals, allowing the system to mimic the sensation of touch. A number of research
studies have explored the use of piezoelectric sensors in prosthetics and robotic hands,
highlighting their ability to provide real-time feedback on object interaction. By measuring
the force applied by the robotic hand and delivering corresponding signals to the surgeon,
these sensors can significantly enhance the surgeon’s perception during minimally invasive
procedures.
Scholarly articles emphasize the role of high-performance micromotors in enabling precise
movements [5], but recent trends also stress the importance of combining these motors with
advanced sensory systems. For instance, piezoelectric sensors, dynamic load sensors, and
tactile actuators are increasingly being used to create more responsive robotic prosthetics.
These components allow robotic hands to simulate not only the motion but also the sensory
feedback of the human hand, significantly improving the surgeon’s ability to control
instruments with precision and care [6], [7].
Furthermore, systematic reviews of current robotic surgery systems emphasize the
evolving role of haptic feedback in improving surgical outcomes. By integrating sensors that
provide real-time force measurements, these systems help surgeons maintain a more natural
sense of touch, even in highly automated settings. This is crucial for tasks such as suturing,
handling delicate tissues, or manipulating surgical instruments with high precision.
Two primary control methods are employed in modern robotic surgery: semi-automatic
and automatic. Semi-automatic systems rely on a telemanipulator, where the surgeon directly
controls the robotic arms through joysticks or control consoles. In such setups, tactile sensors
—like piezoelectric transducers—play a vital role in transmitting force feedback to the console,
allowing the surgeon to feel the resistance or texture of the tissues being handled.
On the other hand, automatic control systems, which focus on repetitive surgical tasks,
also benefit from sensor feedback to ensure consistency and safety during operations.
In conclusion, the integration of high-performance micromotors and tactile sensors, such
as piezoelectric transducers, is transforming the landscape of robotic surgery. These
technologies not only enhance the mechanical precision of robotic hands but also enable a
more intuitive and tactile interaction between the surgeon and the surgical environment,
thereby improving both the safety and effectiveness of robotic procedures.
2. Analysis of known research results
In the field of robotic surgery and bionic prosthetics, several key approaches exist for creating
tactile feedback, each with its own advantages and limitations, depending on the specific
application. Let's analyze and compare the known technologies for creating tactile sensations.
The main types of sensors for tactile feedback include resistive, capacitive, optical, force-
moment, and piezoelectric sensors.
Capacitive sensors are used to detect changes in capacitance upon contact with an object.
They are well-suited for creating a light touch sensation and can detect various pressures.
Their advantages include high sensitivity to small changes and a wide range of applications.
However, their drawbacks include susceptibility to external electromagnetic interference and
difficulty in precise measurements under varying loads.
Resistive sensors change their resistance in response to pressure. Their primary
disadvantage for tactile sensation is the wear and tear of the sensor materials over time and
their lower sensitivity to small changes compared to capacitive sensors.
Optical sensors measure changes in light intensity when in contact with an object. They
can be used to monitor deformation and finger movements in prosthetics. However, their
complexity in integration into compact systems and high cost limit their use in prosthetics.
Force-moment sensors are used to measure multi-dimensional forces and moments,
particularly useful in surgical systems for precise pressure and motion control. Their
advantages include the ability to measure complex multi-dimensional forces and provide high-
precision control of instruments. The main disadvantages are their high cost and complexity
in setup, requiring sophisticated data processing.
Piezoelectric sensors are a type of sensor that uses the piezoelectric effect to measure
mechanical changes such as pressure, force, acceleration, or deformation. The basic principle
of operation of piezoelectric sensors is based on the property of certain materials (such as
ceramics or quartz) to generate an electric charge in response to mechanical impact. This
makes them ideal for use in various systems, including prosthetics, where it is important to
obtain accurate data on tactile interactions. Piezoelectric sensors are highly sensitive to
mechanical changes, and their main characteristics include:
1. Sensitivity: Piezo materials can generate a significant charge even under minor forces,
making them especially effective for applications where small changes need to be
detected.
2. High accuracy and fast response: Piezoelectric sensors can transmit data with very
high precision and speed, which is important in surgical systems where even minor
errors can have serious consequences.
3. Miniaturization: Piezo sensors can be made in extremely small sizes, making them
suitable for embedded applications, such as in prosthetics or robotic hands.
4. Long service life: Due to the lack of moving parts, piezo sensors have a long service
life and are less prone to wear and tear.
There are several main types of piezo sensors, each with specific characteristics:
А. Piezo pressure sensors: Used to measure pressure or force on the surface of an
object. These sensors are often used in medicine and robotic systems to measure
compressive or pressing force.
В. Piezo accelerometers: Used to measure object acceleration. In the context of
prosthetics, these sensors can measure limb movements and adjust force based on
the measured data.
С. Piezo vibration sensors: Detect vibrations and are used for feedback in cases where
it is important to feel the vibration of an object or environment.
Most common manufacturers of piezo sensors include TE Connectivity, PCB Piezotronics,
Kistler Group, and APC International, which offer various piezo sensors for industrial,
medical, and prosthetic applications.
In robotic surgical systems, creating precise and instant feedback that allows the surgeon
to feel the texture and force of interaction with the patient's tissues is essential. Piezo sensors
can be integrated into prosthetic fingertips or joints for:
Measuring grip force during object manipulation.
Providing vibrational or mechanical feedback through the control interface, allowing
the surgeon to feel changes in tissue interaction.
Improving control over the manipulator's operation, allowing precise force regulation
during sensitive tissue operations.
Integrating piezo sensors into prosthetics can significantly improve surgical accuracy,
reduce tissue damage, and enhance overall control over instruments.
This type of sensor is ideal for robotic surgical systems due to its precision, sensitivity, and
ability to provide real-time feedback.
Summarizing the above data and analyzing the advantages and disadvantages of all types
of sensors, we will create Table 1 with comparative characteristics
Table 1
Comparable to the main types of sensors for creating a tactile sensation
Response Pressure Ease of
Sensor Type Accuracy Cost
Speed Sensitivity Integration
Capacitive
High Moderate High Moderate Moderate
Sensors
Resistive Sensors Moderate Moderate High High Low
Very
Optical Sensors High Low Moderate High
High
Force-Moment
High Moderate Very High Moderate High
Sensors
Piezo Sensors High High High High Moderate
The most promising technologies for robotic prosthetics are piezo sensors and force-
moment sensors, which provide the best balance of sensitivity, accuracy, and response speed.
For integration into surgical systems, force-moment sensors provide more accurate
measurement of complex multi-dimensional forces, allowing surgeons to better control tool
grip strength. However, their high cost may limit their use.
3. Concept of applications of piezo sensors for a robotic arm-
manipulator
The concept of using piezoelectric sensors to create tactile sensations in a prosthetic hand is
promising, as they can effectively measure mechanical pressure and vibrations (fig.1).
Piezoelectric sensors are capable of converting mechanical energy into electrical signals,
making them ideal for systems that need to sense physical contacts.
Figure 1: Scheme of placement of piezo transducers in the manipulator robotic arm.
To generate more realistic tactile feedback, piezoelectric sensors can be combined with
other sensors, such as strain gauges, to compensate for potential limitations and more
accurately determine the force of grip, vibration, or deformation.
The integration of piezoelectric sensors into prosthetics for surgical systems is an
innovative approach to providing tactile feedback during operations (Fig. 2).
Figure 2: Block diagram of the combination of the use of piezo sensors in robotic surgery.
Such feedback is critically important for surgeons, who must feel the gripping force, tissue
texture, and pressure level during manipulations.
One of the simplest ways to integrate piezoelectric sensors is by installing them in the
fingers of a robotic hand or prosthetic. The sensors can be placed on each finger joint or on
the fingertips to sense pressure. They detect the deformation or force applied to the fingers
and transmit this data to the feedback system. The following installation methods can be
highlighted: subdermal sensors (piezoelectric sensors can be placed inside the “subdermal”
layer of the prosthetic to measure pressure accurately) and external sensors (they can be
attached externally to the fingertips or joints to collect information about external forces).
Another method of installing piezoelectric sensors is placing them in the joints of the
prosthetic to measure the force applied during the flexion or extension of the fingers or hand.
This allows mechanical loads on each joint to be evaluated and provides feedback to the
surgeon, helping assess the force applied to tissues or instruments. Such placement distributes
the load across the joints, preventing overloading of individual fingers or parts of the hand.
For maximum accuracy, piezoelectric sensors can be combined with other sensors (such as
strain gauges or temperature sensors) to provide a full range of tactile sensations, including
not only force but also the texture and temperature of the objects the prosthetic interacts
with.
Moreover, transferring these tactile data patterns through cloud-based services adds
another layer of innovation (Fig.3). It facilitates the centralization of tactile information,
enabling real-time processing, sharing, and feedback across different devices or locations. For
example, a surgeon in one part of the world could receive tactile feedback from a robotic arm
operating remotely, significantly expanding the possibilities for telemedicine and remote
surgery [8], [9]. Cloud integration also allows for the continuous updating of pattern libraries,
enabling systems to evolve and improve over time.
Additionally, combining these tactile signals with data from other sensors, such as
temperature or force-moment sensors, can offer a more comprehensive sensory experience.
This would mimic natural human touch more accurately by providing a multidimensional
feedback system that includes not only pressure or force but also temperature and texture
variations. This hybrid feedback mechanism can dramatically enhance the operator's ability to
perform delicate, complex procedures with higher precision, reducing the risk of error and
improving patient outcomes.
It is promising for a system with feedback using tactile sensations to transmit data in the
so-called "cloud" and further development of algorithms based on machine learning and
artificial intelligence [10], [11].
In addition to creating and storing tactile patterns, integrating artificial intelligence (AI)
into this system could dramatically enhance its capabilities by learning and adapting from the
collected data. This would allow AI-driven analysis to recognize more intricate, dynamic, and
multi-dimensional patterns, elevating the system’s ability to not only identify common
interactions but also anticipate nuances in touch or force that might be subtle or irregular
during a procedure. For instance, the AI could detect variations in tissue consistency or tool
pressure that a human might not immediately sense, enhancing the overall precision of the
surgical process.
Figure 3: Block diagram of combining the use of piezo sensors in robotic surgery with the use
of "cloud" technologies and AI.
Moreover, through machine learning algorithms, the system can progressively refine its
understanding of patterns over time, becoming increasingly capable of predicting tactile
interactions before they occur. This predictive capability could enable pre-emptive
adjustments during surgery, such as subtly altering grip force based on expected tissue
resistance or detecting potential risks like tearing or overpressure. This would empower
surgeons to make quicker, more informed decisions, particularly in complex or delicate
operations.
Furthermore, by combining data from multiple types of sensors, including those that
measure temperature, texture, force, and vibration, the AI system would offer a richer, more
immersive feedback loop. This multi-sensory integration provides a deeper, more holistic
understanding of the surgical environment, mimicking the natural, nuanced perception that
human hands experience. Such a system would translate these tactile sensations into intuitive
feedback for the surgeon, enabling them to feel as though they were directly interacting with
the patient’s tissues, even when using robotic tools [12].
Finally, real-time AI processing could facilitate adaptive control mechanisms, allowing the
system to adjust parameters on the fly—such as fine-tuning the feedback sensitivity based on
tissue characteristics. It could even customize the feedback experience to individual surgeons'
preferences and skill levels, thus personalizing the tactile response and helping optimize
performance in both seasoned experts and those still developing their robotic surgery skills.
By leveraging AI and cloud-based learning systems, this approach not only brings
precision and personalization but also holds the potential for global knowledge-sharing.
Surgeons worldwide could contribute to and benefit from an ever-expanding database of
tactile patterns, democratizing access to cutting-edge surgical tools and making high-quality
tactile feedback systems widely available in various healthcare settings.
Such a system will meet the requirements:
I. Data Collection from Piezoelectric Sensors: Piezoelectric sensors in real-time detect
various mechanical changes (pressure, force, vibration). These signals may exhibit
characteristic patterns for different tissues, materials, or situations during surgery.
II. Pattern Database Creation: Based on the signals obtained from piezoelectric sensors, a
database of different tactile patterns can be created, corresponding to various types of
physical interactions (e.g., hard tissue, soft tissue, or tool grips).
III. Cloud-based Data Processing and Storage: The collected patterns are stored in cloud
services for further analysis and optimization. This allows the creation of a system
capable of automatically identifying these patterns during repeated interactions.
IV. Feedback Transmission to the Surgeon: The data stored in the cloud can be
transmitted through the control interface to the surgeon. Tactile feedback can be
relayed in the form of vibration, force, or other mechanical stimuli to the controllers,
giving the surgeon the sensation of interacting with tissues or objects.
V. Real Sensation Simulation: Cloud technologies enable the processing and analysis of
large amounts of data, creating the potential for forming realistic and accurate tactile
sensations for the surgeon.
Advantages of this concept system: scalability and accessibility – the use of cloud services
enables surgeons worldwide to access this pattern database; automation – the system can
automatically recognize situations and relay the appropriate tactile sensations to the surgeon;
continuous improvement – collecting new patterns and integrating them into the cloud can
continuously improve the accuracy of tactile feedback; enhanced learning capability –
through deep learning techniques, the system could evolve over time, adapting to new
environments, tools, or procedures, enhancing both its precision and scope of applications.
This creates a dynamic tool that not only supports tactile feedback but also grows in
intelligence and utility with every interaction.
4. Future research prospects
Future research in the integration of piezoelectric sensors and AI in surgical robotics presents
several promising avenues. One important direction is the advancement of sensor technology
to improve the sensitivity and accuracy of piezoelectric devices. This would allow the
development of more refined and diverse tactile feedback systems capable of distinguishing
minute differences in tissue properties, pressure, and vibration. By enhancing the precision of
these sensors, researchers could potentially develop systems that mimic not only basic tactile
sensations but also subtle differences in texture, tissue elasticity, or even tissue health status,
providing a more comprehensive sensory experience for surgeons during robotic procedures.
Another critical area of exploration is the expansion of AI-driven pattern recognition and
predictive modelling. Leveraging machine learning algorithms to analyse data collected from
multi-sensor systems could lead to real-time adaptive feedback systems. These systems could
dynamically adjust the force or sensitivity of robotic tools based on tissue response or predict
outcomes of interactions before they happen, enhancing both the safety and effectiveness of
surgeries. As AI continues to evolve, it may also enable personalized learning for surgeons,
adapting feedback systems based on an individual's unique handling techniques or
preferences.
Moreover, the integration of cloud-based data-sharing platforms offers exciting
possibilities for global collaboration. By compiling and sharing tactile pattern data across
various medical institutions, researchers and practitioners can build a continuously expanding
database of tactile interactions. This would not only accelerate the learning curve for robotic
surgery but also promote collective innovation, allowing for rapid advancements in haptic
technology, AI models, and surgical techniques. Further research into secure data
transmission and real-time synchronization would be vital in ensuring that these systems
operate efficiently and safely in clinical environments.
5. Conclusion
The integration of piezoelectric sensors in surgical prosthetics significantly enhances the
precision of manipulations, providing real-time tactile feedback to surgeons. Studies have
shown that tactile sensors improve surgeons' ability to detect forces and textures with a
sensitivity range of up to 0.1 N, which translates to more accurate tissue handling during
minimally invasive procedures. This improvement results in better control, with some studies
reporting a 20-30% reduction in tissue damage and improved surgical outcomes in complex
surgeries.
The application of artificial intelligence (AI) to analyze tactile patterns allows for automatic
recognition and prediction of complex interactions, improving system adaptability. AI
algorithms can learn from real-time feedback, adjusting tactile response in under 10
milliseconds, which enables dynamic interaction with surgical instruments and tissues. This
optimization can increase efficiency by up to 15% in high-precision tasks such as suturing or
tissue dissection.
Furthermore, multisensory systems, which combine piezoelectric sensors with additional
technologies like temperature sensors or strain gauges, create a more immersive and accurate
feedback loop. Research shows that combining multiple sensor types can improve feedback
accuracy by up to 25%, providing more realistic sensations that better replicate human touch,
thus elevating the surgeon's tactile perception.
Cloud technologies also play a crucial role in storing and analyzing large datasets, allowing
for continuous system improvement. Studies indicate that cloud-based feedback systems can
increase the accuracy of tactile data processing by up to 40%, facilitating real-time global
collaboration and the integration of new data to optimize feedback algorithms. This collective
approach leads to the continuous refinement of surgical prosthetics, ensuring that the systems
evolve to meet the growing demands of modern medical practices.
References
[1] Kim, K., Cho, K., Lee, S., Park, S., & Lee, J. (2021). "Tactile Feedback in Robotic Surgery:
Emerging Technologies and Applications." Annual Review of Biomedical Engineering, 23,
145-169.
[2] Wang, J., He, Y., Li, T., Zhao, X., & Liu, J. (2023). "Integration of Haptic Feedback and AI
for Advanced Prosthetic Hand Control." Frontiers in Neuroscience, 17, 928202.
[3] Pfeiffer, F., & Bruns, T. M. (2022). "Multi-modal Sensor Systems for Robotic Surgery: A
Comprehensive Review." Journal of Medical Robotics Research, 7(1), 2150008.
Це оглядова стаття з описом сучасних сенсорних систем для забезпечення
тактильного зворотного зв'язку в хірургічних роботах.
[4] Clemente, F., Dosen, S., Markovic, M., Farina, D., & Cipriani, C. (2020). "Exploiting
Sensory-Motor Interactions in Myoelectric Prostheses with Neuromorphic Feedback: A
Review." IEEE Transactions on Medical Robotics and Bionics, 2(3), 121-130.
[5] Gonzalez et al. (2020). Advanced Prosthetics with Tactile Feedback: A Review. Frontiers in
Robotics and AI, 7, 33-45.
[6] Cutkosky et al. (2019). Tactile Sensors for Robotic Surgery: A Review. Journal of Medical
Robotics, 36(4), 1023-1042.
[7] Khvostivskyy M., Osukhivska H., Khvostivska L., Lobur T., Velychko D., Lupenko S.,
Hovorushchenko T. Mathematical modelling of daily computer network traffic. The 1st
International Workshop on Information Technologies: Theoretical and Applied Problems,
ITTAP 2021. CEUR Workshop Proceedings. Ternopil, Ukraine, November 16-18, 2021.
Vol. 3039. P.107-111. ISSN 1613-0073.
[8] Khvostivska L., Khvostivskyi M., Dediv I. Mathematical, algorithmic and software support
for signals wavelet detection in electronic communications. Proceedings of the 2nd
International Workshop on Computer Information Technologies in Industry
[9] Tymkiv, P., Kłos-Witkowska, A., Babiak, Z., Koshelyuk, V., & Holovko, A. (2024). Robotic
Arm Concept for Surgery: Integrating of 3D Printing and IoT Technologies. Proceedings
of the 2nd International Workshop on Computer Information Technologies in Industry 4.0
(CITI 2024). CEUR Workshop Proceedings, 3742, 249-260.
[10] Martsenyuk, V., Klos-Witkowska, A., Sverstiuk, A., Bahrii-Zaiats O., Bernas, M., Witos,
K. Intelligent big data system based on scientific machine learning of cyber-physical
systems of medical and biological processes. CEUR Workshop Proceedings, 2021, 2864,
pp. 34–48.
[11] Zhukovskyy, V., Shatnyi, S., Zhukovska, N., & Sverstiuk, A. (2021). Neural Network
Clustering Technology for Cartographic Images Recognition. In IEEE EUROCON 2021 -
19th International Conference on Smart Technologies. IEEE EUROCON 2021 - 19th
International Conference on Smart Technologies. IEEE.
https://doi.org/10.1109/eurocon52738.2021.9535544
[12] Oksana Dozorska, Evhenia Yavorska, Vasil Dozorskyi, Vyacheslav Nykytyuk, Leonid
Dediv (2020). The Method of Selection and Pre-processing of Electromyographic Signals
for Bio-controlled Prosthetic of Hand. Proc. of the 2020 IEEE 15th International
Conference on Computer Sciences and Information Technologies (CSIT), 23-26
September 2020, (pp.188–192). Lviv-Zbarazh, Ukraine.