Socially Assistive Robots (SARs) are increasingly used in therapeutic settings to support cognitive, social, and motor development through personalized, interactive engagement. However, deploying SARs in real-world clinical environments requires software frameworks that are both adaptable to individual patient needs and accessible to non-technical users. This work presents a modular full-stack REST framework designed to operationalize robot-assisted therapy with the iCub humanoid robot. The system supports the rapid deployment of personalized therapeutic workflows-referred to as iCub Applications-by composing modular robot actions into reusable templates. These applications are executed through a Wizard-of-Oz (WoZ) paradigm, enabling a human operator to teleoperate the robot online and maintain adaptive, patient-centered control. An intuitive, web-based frontend allows clinicians and researchers to configure and supervise sessions without requiring programming expertise. The interface dynamically generates its content based on the backend configuration exposed via a REST API, allowing multiple applications to be supported without modifying the frontend code, since all application logic and structure are defined on the backend. By integrating flexibility, modularity, and human-in-the-loop interaction, the framework bridges the gap between low-level robotic control and clinical usability, contributing to the advancement of socially-aware, human-centered AI in assistive and rehabilitation contexts.
Integrating robotics into therapeutic care has opened new opportunities to deliver engaging, adaptive,
and personalized support for individuals with developmental, cognitive, and motor challenges. In
particular, Socially Assistive Robots (SARs) have shown promise in domains such as autism therapy,
motor rehabilitation, and cognitive training [
However, the efectiveness of SAR-based interventions depends heavily on the underlying software
architecture. Simply deploying robot hardware is not suficient; what is needed is a system that
enables real-time control and adaptation of robot behavior according to individual user responses. This
highlights the central role of Human–Robot Interaction (HRI) design, particularly in clinical settings
that require personalization, transparency, and therapist supervision [
In this context, the iCub humanoid robot[
Despite this potential, developing therapeutic sessions with iCub is a challenge for non-technical users. Therapists and clinical staf typically require programming skills to define and execute interaction scenarios, which limits the scalability and day-to-day usability of the platform in healthcare environments. This creates a gap between what the robot is capable of and what clinicians are able to deploy independently.
To address this limitation, we propose a new software framework that allows the configuration and
execution of structured iCub Applications, customizable therapeutic workflows built from modular robot
actions. These applications are modeled using Finite State Machines (FSMs) and follow a Wizard-of-Oz
(WoZ) execution model [
The system is built on a full-stack REST architecture that cleanly separates the backend—responsible for protocol logic, application definition, and robot coordination—from the frontend, which provides an intuitive web-based interface for therapists and researchers. Notably, the frontend dynamically generates its content based on definitions received via the REST API, allowing new applications to be deployed or reconfigured entirely from the backend without requiring changes to the frontend code. This design supports accessibility, scalability, and adaptability, making the platform well-suited for both clinical and research settings.
The iCub humanoid robot is a flexible research platform for Human–Robot Interaction (HRI), cognitive
robotics, and therapeutic settings [
iCub’s software is built on the modular YARP middleware [
Within this framework, we define an iCub Application as a structured sequence of robot actions designed to support a specific therapeutic activity. Each application orchestrates the robot’s multimodal capabilities into a coherent interaction flow that aligns with clinical objectives, such as promoting joint attention, encouraging verbal interaction, or guiding motor tasks.
Applications are executed using a Wizard-of-Oz (WoZ) approach [
This framework supports a human-in-the-loop interaction model in which robot behavior is not pre-scripted but flexibly directed by the therapist’s clinical judgment. It ofers a balance between repeatability—important for structured therapy—and real-time adaptability—essential for personalization. As such, iCub serves not just as a programmable robot, but as a responsive therapeutic partner capable of supporting individualized, interactive interventions.
To support the execution of structured yet flexible robot-assisted therapy sessions using the iCub humanoid platform, we introduce a modular, full-stack REST software framework. The proposed framework is designed to abstract the technical complexity of robotic control and provide an accessible interface for clinicians and technical operators to define, configure, and supervise therapeutic protocols without requiring robotics or programming expertise.
At the core of our framework is the support for executing iCub Applications, reusable, high-level therapy programs defined as structured sequences of robot actions. Each action corresponds to a capability of the robot, such as gaze control, speech synthesis, gesture execution, or facial expression rendering. These applications are executed under the WoZ paradigm, which places a human operator in the control loop, enabling real-time guidance of robot behavior based on the patient’s engagement and responses. This approach supports patient-specific adaptation while maintaining the safety and repeatability required in clinical settings.
To support the operator during the execution of each application, the framework incorporates three key design choices. First, each iCub Application is modeled as a Finite State Machine (FSM), where individual states correspond to specific robot actions and transitions represent the flow of interaction. According to the WoZ paradigm, these transitions are triggered by the therapist in real time, based on their observation of the patient’s behavior. This approach ensures that clinicians retain full control over the session while allowing for dynamic, personalized adjustments. An example of an FSM representation of an iCub Application is illustrated in Figure 2.
do: ShakeHand do:
Ask Question do: Happy Emotion do: Sad Emotion do: Grab Cube do: Rotate Cube
do: Drop off Cube do:
Say Goodbye
Second, to facilitate therapist interaction with the application during execution, the system provides a web-based dashboard based on the selected iCub Application. This dashboard is composed of modular graphical widgets, as shown in Figure 3. The main widget consists of an FSM visualizer that can be used by the operator to track and control the execution of the application. Other dashboard widgets correspond to a specific robot function or protocol control, such as Text-to-Speech control, emotion display configuration, and video stream monitoring.
Third, the frontend dashboard is dynamically generated at runtime based on definitions retrieved from the backend via the REST API. This design allows the addition or customization of therapy applications solely through backend configuration, without requiring any changes to the frontend code. A key feature of this approach is the ability to define configurable session arguments—parameters that can be selected by the user at the start of the session (e.g., task dificulty, target emotion, or number of repetitions). The structure and allowed values of these arguments are fully defined in the backend and automatically reflected in the user interface at runtime. Moreover, the layout of the dashboard widgets is highly customizable and can be modified, imported, or exported as a JSON configuration file. This enables flexible adaptation of the interface to specific clinical needs or user preferences, supporting a variety of use cases and session types. Finally, custom Python methods defined in the main application backend are automatically exposed as REST endpoints and can be selectively included in the dashboard on request. This allows developers to quickly integrate new robot capabilities or protocol logic without additional frontend development, further enhancing the modularity and extensibility of the framework.
At the start of a rehabilitation session, the operator can select one of the available iCub Applications and, therefore, the frontend automatically configures the dashboard to reflect the structure and parameters of the selected FSM. Therapists can further personalize the interface by rearranging, resizing, or disabling widgets based on the therapeutic goals and interaction context. Through this configurable dashboard, the therapist remains central to the interaction, manually triggering state transitions in response to the patient’s real-time behavior. This human-in-the-loop approach ensures that sessions are not only responsive and personalized, but also maintain the consistency and structure necessary for clinical reliability and experimental reproducibility.
The overall software architecture adopts a client–server model, as shown in Figure 4. The backend component, named Pyicub, handles robot coordination, FSM-driven session execution, and service orchestration. It exposes REST APIs to the frontend, enabling seamless, asynchronous communication between user-facing components and the robot’s control logic.
Frontend
PyiCub Backend Desktop HTTP/REST and Mobile «component» Angular HTTP
Client «component» Dashboard Configuration
Manager
HTTP/REST «component» Services Registry «component» iCub Application «component»
Robots Registry «component» iCub Controller
YARP iCub Robot
The backend is structured into four main modules that handle robot communication and therapeutic
session execution. The iCub Controller abstracts low-level interactions with the robot through the
YARP middleware [
The frontend is a responsive Single Page Application (SPA) built with Angular. At its core, the HTTP Client handles all asynchronous REST communications with the backend and user-triggered events, keeping the interface synchronized with system logic. The Dashboard Configuration Manager controls the layout and widget setup, allowing the interface to adapt flexibly to diferent therapeutic sessions while ensuring an intuitive user experience.
In summary, the proposed system delivers a cohesive full-stack REST architecture that bridges the gap between iCub’s technical capabilities and the real-world needs of clinicians and researchers. By modularizing robot functionality, supporting dynamic session configuration, and providing a user-friendly frontend, the system enables accessible, repeatable, and adaptive robot-assisted therapy—advancing the practical integration of socially assistive robotics in clinical practice.
This work introduced a modular, full-stack REST framework that supports the configuration and execution of structured, adaptable therapeutic sessions using the iCub robot. By abstracting low-level control through a unified interface and adopting the Wizard-of-Oz paradigm, the system allows therapists to configure and execute iCub Applications—reusable therapeutic protocols composed of modular robot actions—directly via a web-based frontend. This approach enables real-time personalization based on patient needs, without requiring any programming skills, while preserving the structure of clinical workflows. The backend, built on the YARP middleware, ensures scalability, reusability, and hardware abstraction via RESTful APIs. Meanwhile, the frontend ofers a dynamic, customizable dashboard that supports human-in-the-loop interaction. Overall, the proposed system advances the integration of socially assistive robotics by aligning technical flexibility with the practical needs of adaptive, human-centered care.
The proposed solution was preliminarily validated through a case study involving technical operators and therapists who used the web application to conduct therapy sessions with actual patients and a single robot. Their feedback provided valuable insights into the usability and clinical applicability of the system, confirming its potential to support personalized, structured robot-assisted interventions in everyday therapeutic practice.
As future work, we plan to extend the experimental validation by involving a broader range of clinicians and therapists to evaluate the system’s efectiveness in real-world therapeutic workflows. Furthermore, we intend to develop a graphical user interface that will enable users to define new iCub Applications directly from the browser. This tool will guide the composition of new FSM-based protocols by selecting and chaining available iCub actions—exposed as REST services through the YARP middleware—thus making application creation more accessible to non-technical users. Finally, we are exploring the integration of semi-autonomous behavior layers to complement the Wizard-ofOz approach, enabling a more balanced interaction between clinician control and robot initiative in long-term interventions.
The author(s) has not employed any Generative AI tool during the preparation of this work. This work was supported by the Italian Ministry of Research, under the complementary actions to the NRRP “Fit4MedRob - Fit for Medical Robotics” Grant (# PNC0000007).