Hostname: page-component-6bb9c88b65-g7ldn Total loading time: 0 Render date: 2025-07-22T05:13:40.571Z Has data issue: false hasContentIssue false

Dynamic workload reallocation for human–robot teams based on real-time stress analysis

Published online by Cambridge University Press:  22 July 2025

Rukiye Kirgil-Budakli
Affiliation:
Concordia Institute for Information Systems Engineering, Gina Cody School of Engineering and Computer Science, https://ror.org/0420zvk78 Concordia University , Montreal, QC, Canada
Yong Zeng
Affiliation:
Concordia Institute for Information Systems Engineering, Gina Cody School of Engineering and Computer Science, https://ror.org/0420zvk78 Concordia University , Montreal, QC, Canada
Ali Akgunduz*
Affiliation:
Department of Mechanical, Industrial and Aerospace Engineering, Gina Cody School of Engineering and Computer Science, https://ror.org/0420zvk78 Concordia University , Montreal, QC, Canada
*
Corresponding author: Ali Akgunduz; Email: ali.akgunduz@concordia.ca
Rights & Permissions [Opens in a new window]

Abstract

As artificial intelligence grows, human–robot collaboration becomes more common for efficient task completion. Effective communication between humans and AI-assisted robots is crucial for maximizing collaboration potential. This study explores human–robot interactions, focusing on the differing mental models used by humans and collaborative robots. Humans communicate using knowledge, skills, and emotions, while robotic systems rely on algorithms and technology. This communication disparity can hinder productivity. Integrating emotional intelligence with cognitive intelligence is key for successful collaboration. To address this, a communication model tailored for human–robot teams is proposed, incorporating robots’ observation of human emotions to optimize workload allocation. The model’s efficacy is demonstrated through a case study in an SAP system. By enhancing understanding and proposing practical solutions, this study contributes to optimizing teamwork between humans and AI-assisted robots.

Information

Type
Research Article
Creative Commons
Creative Common License - CCCreative Common License - BY
This is an Open Access article, distributed under the terms of the Creative Commons Attribution licence (http://creativecommons.org/licenses/by/4.0), which permits unrestricted re-use, distribution and reproduction, provided the original article is properly cited.
Copyright
© The Author(s), 2025. Published by Cambridge University Press

Introduction

As artificial intelligence (AI) advances, human–robot collaboration (HRC) is becoming increasingly common, both in personal and professional contexts. With computerized systems expected to engage in social and work-related activities through reciprocal communication with humans, adapting human and collaborative robot behaviors to new group norms is essential. This necessitates a well-established communication structure for advanced human–robot systems (HRS).

Communication involving machines has been extensively studied, primarily in Machine-to-Machine communication, while human-involved communication has been explored through human-centric approaches. While research in human–machine interaction traditionally focuses on improving machine utility within human–machine systems (HMSs), there is a growing trend toward exploring machine analysis of human behaviors to enhance machine intelligence in serving human needs adaptively. In the context of human–robot teams, humans and robots possess different capabilities and mental models for task completion. It is crucial to maintain shared understanding among team members, as their behaviors and decisions rely on different mental models. Communication should be segmented into subchannels, each aligned with corresponding mental models, facilitating task completion through a shared mental model.

Effective communication in human–robot teams is crucial for maintaining high productivity. While robots are not subject to emotions or stress, human performance is influenced by stress levels related to workload, knowledge, skills, and affective states. Optimal human function requires a certain level of stress, with both low- and high-stress environments reducing performance. Improved collaboration necessitates partners to communicate while considering each other’s specific processes and characteristics. Accordingly, creating a human–robot communication model that identifies and understands the unique characteristics of various communication channels and guiding tailored actions and solutions to enhance productivity is crucial for improving communication effectiveness.

This article aims to provide a theoretical mathematical framework for researchers in human–robot interaction, categorizing communication channels within a human–robot team to address technological prerequisites. While not focused on quantitative productivity computation, it offers a qualitative exploration of communication dynamics and productivity between humans and robots. The communication model draws inspiration from design theories, aiming to meet project objectives by exploring human–robot communication complexities. Additionally, this article explores methodologies to measure human stress using an inductive research approach, offering insights into factors affecting human stress and performance measurement. However, challenges arise in measuring vague entities such as human stress levels and emotions, limiting precision and leading to more subjective discussions.

The remainder of this article is structured as follows. In the next section, a brief literature review that is closely related to human–robot communication files is discussed. In Section “Formulation of smart human-robot system as a function of human stress”, the proposed human–robot communication model is given. In Section “Case study: task reallocation management for optimized performance in human-SAP collaboration”, a case study that demonstrates the feasibility of the proposed model is presented. Finally, in Section “Conclusions and future works”, the conclusions and future works are summarized.

Literature review

HRC, driven by AI for comprehension, analysis, and cognitive computing simulating human mental models, has significantly impacted diverse domains, including resilient factories, autonomous vehicles, robotic assembly lines, and more (Sreedevi et al., Reference Sreedevi, Harshitha, Sugumaran and Shankar2022; Prasad et al., Reference Prasad, Kumar and Prasad2024). Over time, this collaboration has evolved through several phases, including human–robot coexistence, human–robot interaction, human–robot cooperation, and human–robot collaboration, culminating in proactive human–robot collaboration (Li et al., Reference Li, Zheng, Liu, Wang, Wang, Zheng and Wang2023).

HRC is sustained through generating data, detecting data through various sensors (in the robot’s case) and cognitive cues (in the human case), and eventually converting data into information that leads to actions/decisions. In this collaborative framework, raw data may be collected by robots through analyzing human behavior and speech using various technologies, including perceptual computing or brain–computer interfaces (BCIs; Perrin et al., Reference Perrin, Chavarriaga, Colas, Siegwart and Millán2010; Kanjo et al., Reference Kanjo, Younis and Ang2019; Ji et al., Reference Ji, Liu, Xu, Yao, Liu and Zhou2021; Lei et al., Reference Lei, Su and Cheng2023). Subsequently, these data undergo processing using AI techniques such as machine learning, deep learning, and natural language processing (Gupta et al., Reference Gupta, Johri, Srinivasan, Hu, Qaisar and Huang2022). Techniques such as image and speech recognition, along with perceptual technologies like facial expression and eye-gaze recognition, further enhance the understanding of human interactions with robots (Guo et al., Reference Guo, Pu and Chen2018; Fahn et al., Reference Fahn, Chen, Wu, Chu, Li, Hsu and Tsai2022). While robots continuously strive to emulate human intelligence by analyzing accumulated data, there remains a disparity between computerized intelligence and the organic learning processes of humans (Yun et al., Reference Yun, Lee, Ahn, Park and Yigitcanlar2016; di Fiore and Schneider, Reference di Fiore and Schneider2017). The complex interplay of these dynamics unfolds within the intricate network of communication channels, a subject that warrants further exploration in subsequent discussions. In their review paper, Soori et al. (Reference Soori, Arezoo and Dastres2023) introduce a large spectrum of learning techniques and AI that are applied in HRC.

Robots, revered as cutting-edge AI collaborators within smart systems, continually refine their capabilities through meticulous analysis of accumulated data, striving to replicate the nuanced facets of human intelligence (di Fiore and Schneider, Reference di Fiore and Schneider2017). The depth of their intelligence is intricately linked to the breadth and richness of accessible data sources. These data reservoirs are replenished through diverse channels: robots autonomously glean insights from human interactions, employing various tools and techniques, or they tap into the vast expanse of information on the internet, guided by the ethos of open innovation (di Fiore and Schneider, Reference di Fiore and Schneider2017). Despite significant strides in AI sophistication toward emulating human-like cognition, a notable gap persists between computerized intelligence and the organic learning processes of humans, as evidenced by autonomous learning (Yun et al., Reference Yun, Lee, Ahn, Park and Yigitcanlar2016). This concept encapsulates the essence of human learning: a dynamic process characterized by the organic assimilation and reconfiguration of acquired knowledge, diverging from the structured approach of direct instruction. These intricate dynamics unfold within the multifaceted network of communication channels, a realm that warrants further exploration in subsequent discussions. Accordingly, in the next three sections, we categorized the literature into three groups: (i) communication in HRC; (ii) smart HRS; and (iii) human emotions and intentions from robots’ perspective.

Communication in human–robot collaboration

Claude Elwood Shannon, a foundational figure in information theory, proposed that communication is essentially a statistical process, where senders offer multiple messages for receivers to select from (Shannon, Reference Shannon1948). Additionally, Shannon and Weaver identified three levels of communication problems: technical, semantic, and effectiveness (Shannon and Weaver, Reference Shannon and Weaver1949). Originally developed for human-to-human communication, this framework has been extended to encompass various communication forms. Floridi (Reference Floridi2020) emphasizes that while machines may outperform humans in certain tasks, they do so through fundamentally different mechanisms, highlighting the need for communication theory to adapt to hybrid environments featuring both human and intelligent computerized systems.

The evolution of communication within HMSs has been extensively studied. Recent advancements necessitate humans to collaborate with AI team members akin to human colleagues, despite humans typically preferring interactions with fellow humans (Nass et al., Reference Nass, Fogg and Moon1996). Merritt et al. (Reference Merritt, McGee, Chuah and Ong2011) found that individuals derive greater enjoyment from collaborating with presumed human partners, even if they are AI team members, illustrating the nuanced nature of human-AI interactions. Conversely, Bergman et al. (Reference Bergman, de Joode, de Geus and Sturm2019) suggest that current robotics advancements have yet to fully replicate human physical and cognitive communication abilities.

In exploring AI technology capabilities, scholars have pursued diverse approaches, ranging from technological advancements to human-centered strategies. Integrating both perspectives is essential in understanding the dynamics of hybrid human–robot teams. McNeese et al. (Reference McNeese, Schelble, Canonico and Demir2021) categorize communication channels into four distinct types, while Krupitzer et al. (Reference Krupitzer, Müller, Lesch, Züfle, Edinger, Lemken and Becker2020) provide a detailed taxonomy of human–machine interaction components.

Damacharla et al. (Reference Damacharla, Javaid, Gallimore and Devabhaktuni2018) conceptualized the human–machine team as a collaborative endeavor across various disciplines, emphasizing mutual objectives and performance enhancement. The literature on human–robot/machine/computer interaction underscores the multifaceted nature of modern communication paradigms, with Shannon’s communication framework applied across diverse channels.

Smart HRS

In the context of smart systems, the major expectation is understanding partners without explicit comments. Rather, communication takes place through observations and experience. In recent years, presumably self-driving or driver-assisted cars have been introduced to our highways. Such a car’s interaction with drivers takes place in two different modes. First, reactive response, where the car reacts to a certain signal, such as lane changes without proper lane-change signal. While enabling a car to identify the concept of the lane and interpreting the notion of violating lane change rules is an extremely complex process, the car’s reactive response to a single event may not qualify the process to be categorized as a smart system. On the other hand, identifying the current state (or capability) of the driver through observing his/her driving quality by collecting data from several sensors over a period and deciding the course of actions including taking over the steering or auto parking, and so forth may be considered as smart HRS.

Intelligence in HRC has been a focal point of research, with scholars exploring various aspects of this dynamic interaction. Borges et al. (Reference Borges, Reis, Ariente Neto, Mattos, Cardoso, Gonçalves and Arezes2021) devised a decision-making framework prioritizing ergonomic compatibility to mitigate work-related musculoskeletal disorders while boosting productivity in HRC scenarios. Despite the capabilities of collaborative robots in handling physically demanding tasks, human involvement remains indispensable, particularly in healthcare settings like intensive care units, as highlighted by Kosa et al. (Reference Kosa, Morozov, Lehmann, Pargger, Marsch and Hunziker2023). Moreover, Kirtay et al. (Reference Kirtay, Hafner, Asada and Oztop2023) discussed the concept of robot trust in humans, suggesting that leveraging human cognitive load can ease the computational burden on robots, underscoring the ongoing necessity of human labor in collaboration with autonomous systems.

Research efforts have also focused on the technical aspects of HRC, emphasizing safety and efficiency. Merlo et al. (Reference Merlo, Lamon, Fusaro, Lorenzini, Carfì, Mastrogiovanni and Ajoudani2023), Lopez-de-Ipina et al. (Reference Lopez-de-Ipina, Iradi, Fernandez, Calvo, Salle, Poologaindran and Suckling2023), Yonga Chuengwa et al. (Reference Yonga Chuengwa, Swanepoel, Kurien, Kanakana-Katumba and Djouani2023), Zanchettin et al. (Reference Zanchettin, Messeri, Cristantielli and Rocco2022), Pereira et al. (Reference Pereira, Bozzato, Dario and Ciuti2022), Prendergast et al. (Reference Prendergast, Balvert, Driessen, Seth and Peternel2021), and Darvish et al. (Reference Darvish, Simetti, Mastrogiovanni and Casalino2021) explored robots’ physical support, particularly in close-proximity assembly operations within smart factories. Meanwhile, Wei and Ren (Reference Wei and Ren2018) concentrated on dynamic autonomous path planning to enhance robot adaptability in challenging environments. Cruz et al. (Reference Cruz, Dazeley, Vamplew and Moreira2021) contributed by developing explainable robotic systems that facilitate transparent communication of robot intentions to human collaborators.

Literature emphasizes the crucial role of human involvement in integrating robots, preserving the irreplaceable human touch. Yet, robots offer remarkable swiftness and precision in task execution. Therefore, it is imperative to cultivate a collaborative ethos and embrace hybrid teams within future technology-enabled systems. This approach ensures a harmonious synergy between humans and robots, promising optimal efficiency and effectiveness in achieving objectives.

Stress measurement in HRC systems

HRC relies on a deep understanding of human intentions, behaviors, and cognitive states to optimize interaction efficiency and safety. Several studies have explored human behavior in human-technology interaction, providing insights that contribute to the development of more intuitive technologies. For example, Huang et al. (Reference Huang, Andrist, Sauppé and Mutlu2015) and Orsag et al. (Reference Orsag, Stipancic and Koren2023) investigated human gaze patterns and upper body positions, respectively, while Ye et al. (Reference Ye, Bullo, Friedkin and Singh2022) applied Bayesian learning to model human-AI group dynamics. These findings enhance collaborative robotic systems by improving situational awareness and interaction fluidity.

Expanding on these behavioral insights, researchers have also examined how to facilitate more effective collaboration in assistive robotic systems. Carlson and Demiris (Reference Carlson and Demiris2012) developed collaborative control mechanisms for intelligent wheelchairs, while Perrin et al. (Reference Perrin, Chavarriaga, Colas, Siegwart and Millán2010) integrated BCI technology to enhance user experience. More recent advancements include BCI integration with augmented reality interfaces, as demonstrated by Ji et al. (Reference Ji, Liu, Xu, Yao, Liu and Zhou2021). These efforts underscore the continued importance of interpreting human intent and behavior in achieving seamless human–robot interaction, as highlighted by Varol et al. (Reference Varol, Sup and Goldfarb2009) and McColl and Nejat (Reference McColl and Nejat2014).

In parallel with understanding intent and behavior, researchers have increasingly recognized the importance of assessing human stress levels in HRC contexts, given that stress can significantly influence cognitive performance, attention, and decision-making. To this end, various physiological and subjective measures have been explored. Brain signals can be collected using Electroencephalography (EEG) (Al-Shargie et al., Reference Al-Shargie, Kiguchi, Badruddin, Dass, Hani and Tang2016; Katmah et al., Reference Katmah, Al-Shargie, Tariq, Babiloni, Al-Mughairbi and Al-Nashash2021; Perez-Valero et al., Reference Perez-Valero, Vaquero-Blasco, Lopez-Gordo and Morillas2021; Attar, Reference Attar2022; Hemakom et al., Reference Hemakom, Atiwiwat and Israsena2023). Eye movements and facial expressions are analyzed through tracking cameras, while heart-related data are monitored using sensor-based technologies (Bitkina et al., Reference Bitkina, Park and Kim2021; Behinaein et al., Reference Behinaein, Bhatti, Rodenburg, Hungler and Etemad2021; Del Carretto Di Ponti E Sessam, Reference Del Carretto Di Ponti E Sessam2023; Gazetta et al., Reference Gazetta, Miller, Clemency, Tanaka, Hackett, Norfleet and Cavuoto2023; Hemakom et al., Reference Hemakom, Atiwiwat and Israsena2023; Awada et al., Reference Awada, Becerik-Gerber, Lucas and Roll2024). Additionally, electrodermal activity (EDA) is measured using wearable sensors to capture stress-induced changes in skin conductivity (Pop-Jordanova and Pop-Jordanov, Reference Pop-Jordanova and Pop-Jordanov2020; Rahma et al., Reference Rahma, Putra, Rahmatillah, Putri, Fajriaty, Ain and Chai2022; Dao et al., Reference Dao, Liu, Solomon and Solomon2024). These physiological signals are typically processed through statistical and machine learning algorithms, often supplemented by subjective self-reports to improve validity and robustness.

However, despite the technological advances, the real-world application of these systems often faces ergonomic and usability limitations. Addressing this issue, Awada et al. (Reference Awada, Becerik-Gerber, Lucas and Roll2024) conducted a comparative study of physiological data collection methods and concluded that EDA alone provided the most significant results for stress measurement. In this context, wrist-worn devices – capable of capturing EDA, skin temperature, blood volume pulse, and triaxial wrist acceleration – have proven to be both effective and ergonomically suitable for real-world deployment (Gjoreski et al., Reference Gjoreski, Luštrek, Gams and Gjoreski2017; Nath and Thapliyal, Reference Nath and Thapliyal2021; Bello-Orgaz and Menéndez, Reference Bello-Orgaz and Menéndez2023; Mitro et al., Reference Mitro, Argyri, Pavlopoulos, Kosyvas, Karagiannidis and Kostovasili2023; Awada et al., Reference Awada, Becerik-Gerber, Lucas and Roll2024).

Nevertheless, an important gap remains in the current literature: few models adequately account for the dynamic and fluctuating nature of human mental states during real-time human–robot communication. To address this, our paper introduces a communication model for HRC that integrates a smart supervisory controller. This controller is designed to detect and respond to stress levels in real time, enabling adaptive workload redistribution and ultimately enhancing the performance and resilience of the collaborative system.

Formulation of smart HRS as a function of human stress

A system is considered “smart” when its components (in this article, an AI-supported service provider or robot) evaluate their counterparts (in our context, human partners) over time and propose actionable decisions to regulate functionality or enhance the productivity/efficiency of collaborative endeavors. In the context of HRS, smart collaboration can exist if both parties (robots and humans) continually assess each other and intervene when needed to sustain the collective output of the system at the optimum (or desired) level. As a non-emotional partner in the HRS, the robot’s performance changes due to limitation of hardware and software or natural wear-and-tears. Hence, it is the human partners’ or other intelligent systems’ responsibility to monitor robot’s functionalities and take preventive actions when and where it is required. On the other hand, as an emotional partner of a human–robot team, human’s performance changes due to several personal and environmental factors. Human’s contribution to a work is subject to perceived workload, human’s knowledge and experience, and the environment (Nguyen and Zeng, Reference Nguyen and Zeng2012). Even without changing the workload and the knowledge and experience, human’s performance deviates significantly due to environmental factors. The impact of environmental factors on humans may not be observable through sensors. Therefore, within a human–robot collaborative system, while human can continually monitor and take preventive actions to improve the robot’s contribution, the currently available communication channels do not equip robots to observe the environmental factors that are contributing to the human’s contribution to teamwork. In order to achieve genuinely smart human–robot collaborative systems, where both humans and robots excel in their contributions to collaborative work, it is crucial to establish effective communication channels. These channels should provide both humans and robots with the tools needed to monitor each other and propose executive decisions aimed at maximizing their partner’s contribution to collaborative work.

Modeling of communication channels through axiomatic theory of design

The environment-based design theory Sun et al. (Reference Sun, Zeng and Zhou2011) and Zeng (Reference Zeng2011) suggests that the world is shaped through three interlinked environments: the human, natural, and built environments. In the context of HRS, the robot embodies the built environment. The Axiomatic Theory of Design (Zeng, Reference Zeng2002) breaks down the relationships formed by human, natural, and built environment (robots) into interactions, which are called communication channels as the foundation for communications in collaborative HRS. Accordingly, possible communications channels that compose of an HRS can be defined using Zeng’s axiomatic theory of design as shown in Table 1.

Table 1. Communication channels and their application domains

Communication channels described in Table 1 enables us to formulate an HRS as a function of the interlinked communication channels:

(1)

Given that in Axiomatic Theory of Design, Equation 1 can be simplified as:

(2)

The significance of nature and its impact on an HRS is undeniable. However, given its inherent complexity, they are dropped from HRS formulation. Hence, the HRS system studied in this article is defined as follows:

(3)

HRC in smart systems

Collaboration is the notion of working with others to achieve a common goal, such as making or creating something. When they are tasked to achieve a common goal, humans and robots are expected to use their communication tools to ensure the collaboration efforts are optimum at all times. Given that humans are the emotional partners in such an environment, existing communication tools do not properly equip robots to understand their human partners’ needs. According to Nguyen and Zeng (Reference Nguyen and Zeng2017), human’s performance is closely coupled with his/her stress level. It is proven that human best performs when they are stimulated by a moderate level of stress. When the stress level is too low or too high, human creativity or performance is observed to be lower.

Accordingly, the work-associated stress is defined as a function of perceived workload ( ), knowledge (K), skills (S), and affective states (A) (Nguyen and Zeng, Reference Nguyen and Zeng2017; Yang et al., Reference Yang, Yang, Quan and Zeng2021).

(4)

Given that persons’ environment and its impact on the humans’ affective state constantly change, despite constant , , and , the human stress level will fluctuate. When the stress level is outside the desired stress level zone in Figure 1, human performance is expected to be no at its optimum level. Hence, in the context of a smart HRS, it is the role of robots to monitor human partners’ stress levels and intervene when the stress level is outside the desired levels.

Figure 1. Relationship between stress and creativity/performance.

Before we lay the groundwork for human stress level management within HRS, let us formally define the HRS performance

. The system performance consists of three components: human performance

, robot performance

and their collective performance

. Therefore, the system performance is:

(5)

where the performance of each participant is formulated as a function of stress (

) (in the case of human) and knowledge (

) and skill (

) (in the case of robot), allocated tasks

and their available capacity and/or time

as:

Assuming that during a given planning period, robot’s knowledge and skills do not change significantly and do not face any mechanical and computational challenges, its performance would be steady and predictable. On the other hand, the completion times and the quality of the work for tasks that are allocated to human ( ) or jointly with human and robots ( ) will deviate depending on the human’s stress level. In the remainder of this article, a human–robot communication system that enables robots to assess human stress and take corrective measures to keep the human stress within the desired levels is discussed.

Figure 2 illustrates the correlation between communication channels and their impact on system performance. Identifying communication channels during task distribution is essential, as it enables robots to address the appropriate workload zones, as shown in Figure 3, to implement necessary adjustments.

Figure 2. Connection between communication and collaboration.

Figure 3. Possible task distribution map among team members.

Next, we introduce two lemmas to formally define the problem to solve.

Lemma 1: With N tasks distributed among humans, robots, and a combination of both, each task assigned to a human contributes to their stress level. While some tasks may elevate stress, others can alleviate it. When the human stress level deviates from the desired range, redistributing tasks among humans and human–robot teams can help restore the stress level to its desired range.

Proof: In Equation 4, considering a planning horizon too short for significant knowledge and skill changes, any alterations in the affective state can be offset by adjusting the perceived workload.

Lemma 2: Perceived workload is a controllable factor.

Proof: Perceived workload, influenced by assigned tasks, past outcomes, and available time, can be regulated through task composition adjustments. By leveraging past experiences and assessing remaining capacity, modifying the task distribution can effectively manage perceived workload.

Tasks within a collaborative work environment are distributed among the team members based on their capabilities (skills and knowledge) and their available capacities. Therefore, members (human, robot or human–robot) who are suitable to handle each task are known. Accordingly, tasks can be categorized in seven different zones as illustrated in Figure 3.

The intelligent HRS leverages its communication channels to dynamically regulate task allocation across various zones, as depicted in Figure 3, to ensure that human team members consistently maintain an optimal level of stress. To achieve this goal, we propose an eight-step approach outlined below for controlling human stress levels within desired parameters, thus maximizing human contributions to collaborative work. Monitoring human stress entails measurement, as detailed in Section 3, and implementing the algorithm presented in the pseudo-code given below. Accurately identifying communication channels, fostering collaboration, and promptly detecting human stress levels for intervention in HRS reallocation are essential components for ensuring effective human–machine collaborative environments.

An Eight-Step Algorithm for Performance Optimization through Stress Management in Smart Systems Governed by Robots:

  1. 1. Utilize the Axiomatic Theory of Design to deconstruct the intertwined communication within smart systems.

  2. 2. Identify diverse communication channels: .

  3. 3. Address each communication channel within the smart system and specify the related tasks.

  4. 4. Define the roles of each system component: roles of human, robot, and HRC(s).

  5. 5. Specify roles that can be performed by multiple components where applicable.

  6. 6. Identify human-involved roles to focus on monitoring human stress levels.

  7. 7. Develop an algorithm for robot systems to detect and measure human stress levels and adjust human’s stress by redistributing tasks. Implementation requires a smart supervisory controller, which can be embedded within robot systems or can be supported by an external supervisory system, depending on the robots’ capabilities. The pseudo-code for this algorithm is provided below.

    Task Reallocation Algorithm:

    Inputs:

    1. i. Resources: Set of Human ; Set of Robots/Machines

    2. ii. Set of tasks

    3. iii. Capabilities of Human, Robot and Human–robot team: Distribute tasks according to resource capabilities as illustrated in Figure 2.

      • Step 0: distribute tasks among Human, Robot, and Human–robot jointly

      • Step 1: , assess human stress

      • Step 2: Task reallocation:

        if :

        Human is over-stressed: Transfer tasks from Human to Robot or Human to Human–robot team

        If includes tasks belong to Zone 4 → Transfer tasks to Robot elseif includes tasks belong to Zone 5 → Transfer tasks to Human–robot team elseif includes tasks belong to Zone 7 → Transfer tasks to Robot or Human–robot team

        elseif if :

        Human is under-stressed: Transfer tasks from Robot or Human–robot team to Human

        If includes tasks belong to Zone 4 → Transfer tasks to Human elseif includes tasks belong to Zone 5 → Transfer tasks to Human elseif or includes tasks belong to Zone 7 → Transfer tasks to Human

        Subject to following constraints

        (6)
        (7)
        (8)
        (9)
        (10)

        Where:

        is the estimated completion time when task k is completed by a human alone

        is the estimated completion time when task k is completed by a robot alone

        is the estimated task completion time when handled by Human–robot jointly.

        is the available time (remaining capacity) and the is the current period

        is the estimated perceived workload at period t

        is the performance of human (percentage of successful completion of tasks) at the t-1

        , and are knowledge, skill, and the affective state of human at time

        is the estimated stress level at period

        is the lower bound for desired stress level

        is the upper bound for desired stress level

        else

        Continue with the current task assignment

      • Step 3: Has the job completed?

        NO: Go to Step 1

        YES: Go to step 4

      • Step 4: End intervention

    Outputs:

    1. i. Updated , , and

    2. ii. Optimized Human stress ( )

  8. 8. Optimize human stress levels to lead to overall optimization of smart system performance.

By following the steps above, the aim is to optimally utilize human and robot capacity along with their collaboration capacity. Controlling this relationship through effective communication channels allows participants in collaborative work to sustain the desired human stress level to reach the expected system productivity level.

As explained previously, human performance ( ) is a multifaceted construct contingent upon several variables: the workload allocated to individuals ( ), the timeframe available for task completion ( ), and the impact of human stress ( ). This stress, in turn, is intricately linked to the perceived workload ( ), individual knowledge ( ), skills ( ), and affective states ( ). In parallel, robot performance ( ) is delineated by factors including the workload designated to robots ( ), their knowledge base ( ), proficiency in various skills ( ), and the time required for task completion ( ). Similarly, collaboration performance ( ) hinges upon the workload assigned to the collaborative effort ( ), the available time for task completion ( ), and the influence of human stress within collaborative settings ( ). The stress levels within collaborative endeavors are directly influenced by the stress experienced by individual humans, which is, in turn, influenced by the perceived workload allocated to the collaborative group ( ), the collective knowledge ( ) and skills of the collaborators ( ), and the emotional states of individuals during collaborative interactions ( ).

In collaborative projects, the interplay between humans and robots complicates dynamics, underscoring the importance of cohesive team performance. Robots must not only interact with individuals but also gauge the overall group stress levels. This enables them to intervene appropriately, such as by reallocating workload, to optimize performance and foster effective collaboration. Consequently, navigating collaboration necessitates nuanced adjustments, integrating disjoint union logic to illustrate the complex interaction among diverse elements. Below, we outline the performance metrics for human, robot, and HRC, culminating in the determination of system performance in scenarios where multiple humans and robots work together toward a shared goal.

Let , , and be performances of hth human, rth robot, and hth human and rth robot interactions, respectively. Accordingly:

(11)
(12)
(13)

The disjoint union formula for individual sets delineates the performance levels of overall human, robot, and collaboration group performances as follows:

(14)
(15)
(16)

In this article, we adopt a comprehensive approach to measuring performance, defining it as the ratio of completed tasks to the total tasks assigned to each system component. This encompasses various metrics, such as the monetary value generated by the smart system, the number of clients effectively served, and the completion rate of allocated tasks. Under this framework, the collective performance of a human group results from the amalgamation of each individual’s contribution, while the performance of a group of robots is the sum of each robot’s individual performance.

To accurately gauge the level of success, it is imperative to normalize performance levels by dividing them by the total tasks assigned to all system components, including humans, robots, and collaboration groups. This normalization ensures a fair comparison and provides insights into the efficiency and effectiveness of each entity within the system (Equation 17). The entirety of the elucidated logic is now captured in mathematical formulation as follows:

(17)

Under the prevailing conditions, the assessment of system performance entails a holistic consideration, incorporating the combined elements of average human proficiency, average robotic capability, and the average efficacy of collaborative groups. Central to this evaluation is the pivotal role of human stress as a barometer for robots to recognize the equilibrium within system dynamics. Given that human performance is intricately linked to the stress experienced, robots are tasked with the crucial responsibility of monitoring and fine-tuning the performance of human counterparts. The ensuing formula encapsulates the qualitative measurement of smart systems’ performance, wherein robots are charged with the dual mandate of optimizing human stress levels and elevating both individual human and collaborative performance metrics.

Collaboration relies on everyone being on the same page intellectually, which helps build understanding among collaborators. In the domain of HRC, the effectiveness of such partnerships is contingent upon the robots attaining a level of intellectual judgment that aligns with human cognition. A critical component in rendering robots intelligent and collaborative is the deployment of a sophisticated supervisory controller. Notably, the smart supervisory controller facilitates synchronous information flow, capturing insights from both human performance and collaborative actions. In this dynamic exchange of information, the smart supervisory controller assumes the responsibility of discerning whether a reallocation of workload is warranted, employing embedded algorithms to make informed decisions. Should the need arise, this controller orchestrates the redistribution of tasks, deftly assigning roles to humans or seamlessly taking over certain responsibilities through a judicious interchange of workload among the collaborators. The ensuing mechanism is illustrated in Figure 4.

Figure 4. Workload reallocation between humans, robots, and their collaborations in smart systems (the details of “Identify Intervention Opportunities” are available in Figure 3; human, robot, human–robot expressions are given in Equations 1116)

In the next section, capabilities of the aforementioned HRC management model are demonstrated through a case study.

Case study: Task reallocation management for optimized performance in human–SAP collaboration

As part of this case study, the interaction of human and SAP systems in a real-life setting is analyzed, and the proposed HRC control model is tested. The study examines the before and aftereffects of implementing SAP transportation management (TM) software at organizations. Tasks and assignments of tasks in Zones (see Figure 2) are decided according to SAP functionalities. The integrated nature of the SAP system enables seamless interaction with other systems used by organizations for various business tasks. However, some communication conflicts occur during collaboration decrease the performance in human–robot (SAP in our case) collaboration. Therefore, the focus of this study was to identify the timing and nature of conflict and interview using task-reallocation options according to the previously introduced task reallocation algorithm, as depicted in Figure 4.

Introduction of the system structure

SAP is a complex system composed of many different IT components serving various business processes. In our case, eight SAP modules are considered: S/4 HANA (Cloud ERP); TM; Event Management; Extended Warehouse Management; Business Network Global Track and Trace; Business Network for Logistics; External Geographic Information Systems, and VSR (Vehicle and Routing) Optimizer. These modules are integrated through various integration technologies, including IDOC, SOAP, REST, RFC, Proxy, File, EDI, JDBC, and BAPI. Additionally, there is a specific integrator system called SAP Process Integration, developed particularly for integrating different systems.

In our study, the SAP TM system takes center stage, with a focus on its collaborators who directly or indirectly influence its operational mechanisms. The analysis of overall system performance includes these collaborators. Ensuring the smooth functioning of SAP TM processes relies on support from complementary systems, as they provide vital information necessary for its operations. The remainder of this article introduces a scenario involving an S/4 Hana sidecar, demonstrating how the SAP TM system integrates with external systems. Lauterbach et al. (Reference Lauterbach, Sauer, Gottlieb, Sürie and Benz2019) provided a comprehensive illustration of the continuous integration between SAP TM and various supporting systems within the context of an S/4 HANA sidecar scenario (Figure 5). Although the focus of our case study revolves around the adaptation of logistic service providers to the SAP LBN (Business Network for Logistics) system, it is imperative to recognize that SAP TM remains the cornerstone, as it serves as the primary source of operational data (see Figure 6 for the illustration of information flow in SAP TM centric system).

Figure 5. Overview of SAP TM integration when not embedded in SAP S/4 HANA (adapted from Lauterbach et al., Reference Lauterbach, Sauer, Gottlieb, Sürie and Benz2019).

Figure 6. Illustration of robot–robot communication; systems integration in SAP S/4 HANA sidecar scenario (robots are subsystems such as SAP LBN and SAP S/4 HANA).

To facilitate clear identification and understanding of the roles within the system, robots are denoted by “R” followed by numerical identifiers (e.g., R1, R2, R3), while human personas, including users, developers, analysts, consultants, and managers, are represented by “H” with corresponding numbers (e.g., H1, H2, H3). Although current SAP systems do not possess autonomous intelligence to actively engage with humans, we anticipate their evolution into sophisticated smart robots capable of continuous collaboration with their human counterparts. Furthermore, integration technologies serve as the backbone for enabling communication among robots. These facilitative components are also referred to as robots within the context of system interactions. Additionally, there are human collaborators responsible for orchestrating integration processes within SAP PI, known as SAP PI consultants.

In light of this background, robot systems communicate with each other through another robot system, which acts as an integrator, in the R R channel. This integration is supervised by humans in the H R channel. Each robot system also serves its users to facilitate their interactions. In this case, system users interact with the systems in the H R channel. Furthermore, humans communicate with each other in the H H channel. Accordingly, the collaborative ecosystem envisioned for a future smart SAP system is depicted in Figure 7, encompassing both robotic and human contributors.

Figure 7. Dynamic collaboration between human–SAP systems (robots): Navigating the smart SAP S/4 HANA sidecar ecosystem with human in the loop.

Identifying tasks and their corresponding zones

For a thorough examination of HRC, the initial step is to meticulously identify and outline communication channels, as previously discussed. Rooted in the Axiomatic Theory of Design (Zeng, Reference Zeng2011), this process necessitates acknowledging three foundational relationship dynamics: human–human, robot–robot, and the pivotal HRC, which encompasses reciprocal interactions between humans and robots. Accordingly, possible tasks within an SAP system are identified, and possible undertakers for these tasks are identified (zones). As outlined previously in Figure 3, tasks can be categorized into seven zones depending on the capability of each partner (H, R, or H-R). In Table 2, identified tasks and their associated zones are given.

Table 2. Possible tasks available for human, robot, human–robot in the LBN (business network for logistics) system of SAP S/4 HANA sidecar scenario

Proceed by navigating through each task delineated within the designated workload zones illustrated in Figure 8. This strategic navigation empowers the collaborative robot to discern and intervene in tasks where necessary. As per the proposed framework, the robot or smart supervisory controller is strategically positioned to identify and intervene solely within the intersecting regions of its operational zones. These intersecting domains serve as pivotal points for potential reallocation of system tasks, as they present opportunities for tasks to be seamlessly transitioned between robotic, human, or collaborative efforts. This shared responsibility framework enables the robot or smart supervisory controller to dynamically reallocate tasks, optimizing efficiency by delegating tasks to human counterparts or assuming certain responsibilities from them.

Figure 8. Generating workload zones for human, robot, human–robot in the LBN system of SAP S/4 HANA sidecar scenario.

It is crucial to note that tasks assigned to workload zones 1, 2, and 3 cannot be reassigned to different system components but can only be internally transferred, such as redistributing tasks from one human worker to another to alleviate stress levels. Our primary objective is to optimize the stress levels of individual humans or human groups by initially integrating robots and subsequently implementing internal workload transfers among similar system components. In that case, , and are the tasks that can change the dynamics of workload allocation by being dependent on the algorithm introduced through pseudo code previously.

The initial six steps of the proposed procedure have been diligently executed thus far. The forthcoming section of this case study delves into the intricacies of detecting human stress levels and optimizing system performance by maintaining individuals in optimal emotional states. Through meticulous examination, we explore strategies to ensure that humans are nurtured in environments conducive to emotional well-being, thereby enhancing overall system efficiency.

Task reallocation based on stress/workload

As the assigned workload ( steadily increases, there is a natural escalation in the perceived workload ( ) experienced by individuals, compounded by the current stressors ( ) they face. The rise in perceived workload, assuming the task completion time or time-to-deadline ( ) remains constant, inevitably leads to a decline in human performance . This decrement in performance not only impacts the ongoing emotional and psychological states of individuals ( ), but also serves as a precursor for the level of stress ( ) in the next period. Consequently, this evolving stress level influences how individuals perceive and respond to the forthcoming workload assigned to them.

Smart supervisory system’s (robot) primary function within this context is to monitor human stress levels and correlate them with performance output. Should the estimated stress level deviates from predetermined thresholds, both below or above the acceptable range ( ), the robots are tasked with dynamically adjusting workload allocation. This intervention mechanism is crucial for maintaining human stress within an optimal range conducive to efficient performance. In our detailed case study, aforementioned task reallocation algorithm is applied on the case study as:

Task reallocation algorithm for SAP S/4HANA sidecar scenario (mathematical representation)

The algorithm outlined above for the case study provides a mathematical representation of the proposed logic, inspired by common problems encountered during the implementation of systems in IT projects. Below, the details are evaluated and explained verbally:

Task reallocation verbal assessment for SAP S/4HANA sidecar scenario:

Inputs:

  1. i. Resources: Set of human ; set of robots/machines

    Assuming the presented case study involves 19 humans, 13 robots, and 22 human–robot (H-R) teams, representing the collaboration between systems and their users.

    H: {H1, H2, H3, H4, H5, H6, H7, H8, H9, H10, H11, H12, H13, H13, H14, H15, H16, H17, H18, H19}.

    R: {R1, R2, R3, R4, R5, R6, R7, R8, R9, R10, R11, R12, R13}.

    HR: {H1-R1, H2-R1, H3-R1, H4-R1, H5-R1, H13-R1, H6-R2, H7-R2, H8-R2, H9-R2, H10-R2, H11-R2, H12-R2, H3-R2, H5-R2, H13-R8, H14-R3, H15-R4, H16-R7, H17-R5, H18-R9, H19-R13}

  2. ii. Set of tasks

    At the start of the project, the work breakdown structure should be clearly defined for each communication channel. In other words, each task should be specified along with the corresponding system component capable of undertaking it. This approach allows robots to first evaluate which communication channels can facilitate the assigned tasks and map these onto workload zones using a Venn diagram. Once the re-allocatable tasks on the Venn diagram are identified, the robots can then analyze these tasks to determine alternative communication channels for possible reassignment. The tasks to be completed using the SAP LBN system are outlined below:

    T: {T1, T2, T3, T4, T5, T6, T7, T8, T9, T10, T11, T12, T13, T14, T15, T16, T17, T18, T19, T20, T21, T22, T23, T24, T25, T26, T27, TAD1, TAD2, TAD3, TAD4, TAD5, TAD6, TAD7}

  3. iii. Capabilities of human, robot, and human–robot team: Distribute tasks according to resource capabilities as illustrated in Figure 2 .

    Systems comprising software-based programs and their users must ensure seamless collaboration. Users should be adequately trained on the system’s operation and equipped with strategies to resolve potential blockages effectively. Moreover, computerized systems must be fully integrated with other digital systems, functioning smoothly even when users are actively involved in the workflow.

    • Step 0: t = 0 distribute tasks among human, robot, and human–robot jointly:

      Initially, the tasks were allocated as follows:

    • Step 1: t = t + 1, assess human stress :

      After assessing human stress levels, the robot (assumed to be an intelligent SAP system in our case) determines that human stress is higher than expected. It also double-checks human performance outputs, such as whether tasks are completed within the given time frame, to identify any irregularities. Consequently, the robot reviews the tasks assigned to the human and analyzes how the workload is distributed, aiming to reduce stress and optimize performance.

    • Step 2: Task reallocation:

      When the human stress level exceeds the upper limit , the robot should take over some tasks from the human. Since performance is calculated based on group outputs using the disjoint union formula, the overall performance of the group must be considered when evaluating an individual human’s performance. Therefore, communication channels are emphasized here rather than focusing solely on humans, robots, or human–robot teams.

      Following the stress analysis and performance check, the robot detects a blockage caused by systems communication (R R) that is hindering human performance. While humans attempt to resolve the encountered issues, they lack sufficient knowledge to overcome them, leading to increased stress levels and higher perceived workload.

      The robot identifies that tasks T26(Z7) and T27(Z5), which belong to zone 7 (representing tasks that can be undertaken by humans, robots, or through collaboration) and zone 5 (representing tasks that can be undertaken by humans or through collaboration), were initially assigned to the (H H) communication channel. Since human involvement remains necessary to address the issues encountered while collaborating with the robot systems, these tasks will be reassigned from the (H H) channel to the (H R) channel to reduce human stress.

      However, during the second iteration, the robot determines that this reassignment alone may not sufficiently lower human stress levels, as humans still retain partial responsibility for these tasks despite robot involvement. Consequently, the robot further detects that tasks T6(Z6) and T8(Z6), currently assigned to the (H R) channel, can be fully automated and reassigned to the (R R) channel. As a result, T6(Z6) and T8(Z6) will be moved from the (H R) channel to the (R R) channel to further alleviate human workload and stress.

    • Step 3: Has the job completed?

      NO: Go to Step 1

      YES: Go to step 4

    • Step 4: End intervention

    Outputs:

    1. i. Updated , , and :

    2. ii. Optimized human stress ( ):

      Ultimately, human stress levels are effectively minimized, leading to enhanced overall performance and productivity.

Upon completion of the robot’s intervention process, wherein it reallocates the system workload among both humans and robots, the objective is to sustain human stress levels at levels that lead to optimum human performances. This approach ensures not only the maximization of human capacity but also acknowledges human contributions within the HRC, particularly in light of the robot’s limitless capabilities.

Results

The analysis of a comprehensive case study illuminates the complex web of communication channels within modern information systems, showcasing a diverse array, including human-to-human, robot-to-robot, and human-to-robot interactions. In our pursuit to augment human performance as pivotal contributors to the efficacy of intelligent systems, particular emphasis is placed on human-to-human and human-to-robot communication channels. Collaborative robots (robots) or intelligent supervisory controllers adeptly monitor variations in human stress levels, recognizing their impact on overall system optimization.

Significantly, it becomes evident in our case study that the interplay within robot-to-robot communication channels significantly influences human performance, presenting challenges beyond human intervention. Consequently, prolonged waiting periods can exacerbate the workload on individuals over time, culminating in heightened stress levels. To mitigate this, the intervention of intelligent supervisory controllers becomes imperative to regulate human stress levels within acceptable parameters. However, current systems have yet to attain the level of sophistication required to meet this demand. Notably, recent initiatives within SAP systems underscore the integration of AI technologies into their operations, leveraging large language models to gather insights from extensive historical data. While this approach signifies a step forward, our vision extends beyond mere utilization of AI as a tool or integration into business processes. Instead, we aspire to foster comprehensive collaboration with AI systems on physical, psychological, and cognitive levels. This goes beyond simplistic chatbot-like functionalities, aiming to deeply integrate AI into the fabric of user experiences within systems. By establishing a symbiotic relationship that transcends conventional human–machine interactions, we seek to unlock the full potential of AI as a true partner in advancing productivity, creativity, and innovation within organizational frameworks.

Our ultimate goal is to blur the lines between robots/smart systems and human counterparts, treating them as equals in a symbiotic partnership. Thus, our model not only envisages robots/systems as colleagues but also envisions a future where they seamlessly integrate into human-centric workflows, enriching interactions and driving collective success.

Conclusions and future works

We thoroughly examined how communication occurs in human–robot teamwork, viewing communication channels as platforms for collaboration between HRS components. We highlighted how variations in human stress, influenced by indirect factors, impact performance. Our research introduced conceptual formulas to identify factors affecting human stress and performance, addressing the absence of emotional intelligence in HRCs.

Our main contribution is an algorithm for collaborative robots to assess the need for intervention based on human stress levels. When stress exceeds predefined thresholds, tasks are reallocated between humans and robots to optimize system performance, creating a feedback loop for autonomous regulation within smart systems.

In a case study, we demonstrated how tasks in an HRS can be decomposed and classified based on their characteristics. We also observed mutual performance impacts in different collaboration types, suggesting that smart systems should integrate smart robots alongside humans to leverage their reasoning and emotional intelligence. One limitation is the absence of numerical results on human stress and performance, hindering precise objective outcomes. While initially focused on qualitative evaluation, our work provides a foundation for future quantitative research.

While the proposed framework introduces an innovative approach to stress-aware HRC, several limitations must be acknowledged. First, the current implementation focuses on a single-human, single-robot, single-task scenario. However, in real-world settings, collaborative environments often involve multiple humans, robots, and tasks. The absence of simulation or controlled experimentation in such complex n-human–n-robot–n-task environments limits the generalizability of our findings. Moreover, although the system is designed to focus on the task a human is attending to at a specific moment (t), we recognize that other tasks on the to-do list – although not actively being addressed – may still influence an individual’s stress levels. We believe these indirect effects are reflected in the individual’s affective states (e.g., mood) identified in Equation 4; however, further research is required to validate this assumption.

Second, the framework currently does not incorporate task reallocation strategies under conditions where multiple simultaneous reallocations are needed, as a human can only focus on one task at time t. Therefore, if multiple task reallocations are required at the same moment, we assume the presence of multiple humans working on different tasks. In this case, n-human–n-robot–n-task environments should again be considered. This limitation stems from the simplified system design and underscores the need for future work involving dynamic environments and adaptive task management.

Third, although our case study illustrates the potential of the system, it lacks quantitative performance metrics such as error rates and task completion times – measures that are essential for real-world validation. Additionally, the system’s adaptability to varying levels of AI sophistication across different robotic systems was not evaluated, which constrains its applicability across heterogeneous platforms.

Temporal changes in human capability and stress levels over extended periods were also not explored, highlighting the need for long-term adaptation mechanisms. This limitation stems from the absence of a long-term experimental setup, which prevented the observation and analysis of how human states evolve over time. As a result, the system’s ability to adapt to these changes remains an open area for future research.

Finally, while Section Stress measurement in HRC systems enhances the connection to psychological and ergonomic research, future work will explicitly align the model with established theories, such as the Yerkes-Dodson Law, to strengthen its theoretical foundation. The limitations identified here will serve as the foundation for future extensions of this research.

Funding statement

This work has been supported by NSERC Discovery Grants under grant Numbers RGPIN-2019-07048 (Y. Zeng) and RGPIN-2020-06759 (A. Akgunduz).

Competing interests

The authors declare none.

References

Al-Shargie, F, Kiguchi, M, Badruddin, N, Dass, SC, Hani, AFM and Tang, TB (2016) Mental stress assessment using simultaneous measurement of EEG and fNIRS. Biomedical Optics Express 7(10), 38823898.10.1364/BOE.7.003882CrossRefGoogle Scholar
Attar, ET (2022) Review of electroencephalography signals approaches for mental stress assessment. Neuroscience Journal 27(4), 209215.10.17712/nsj.2022.4.20220025CrossRefGoogle ScholarPubMed
Awada, M, Becerik-Gerber, B, Lucas, GM and Roll, SC (2024) Stress appraisal in the workplace and its associations with productivity and mood: Insights from a multimodal machine learning analysis. PLoS One 19(1), e0296468.10.1371/journal.pone.0296468CrossRefGoogle ScholarPubMed
Behinaein, B, Bhatti, A, Rodenburg, D, Hungler, P and Etemad, A (2021, September) A transformer architecture for stress detection from ECG. In Proceedings of the 2021 ACM International Symposium on Wearable Computers, pp. 132134.10.1145/3460421.3480427CrossRefGoogle Scholar
Bello-Orgaz, G and Menéndez, HD (2023, April) Smartphones and wristbands detect stress as good as intrusive physiological devices. In World Conference on Information Systems and Technologies. Cham: Springer Nature Switzerland, pp. 308319.Google Scholar
Bergman, M, de Joode, E., de Geus, M & Sturm, J (2019) Human-cobot teams: Exploring design principles and behaviour models to facilitate the understanding of non-verbal communication from cobots. In Proceedings of the 3rd International Conference on Computer-Human Interaction Research and Applications, pp. 191198.10.5220/0008363201910198CrossRefGoogle Scholar
Bitkina, OV, Park, J and Kim, HK (2021) The ability of eye-tracking metrics to classify and predict the perceived driving workload. International Journal of Industrial Ergonomics 86, 103193.10.1016/j.ergon.2021.103193CrossRefGoogle Scholar
Borges, GD, Reis, AM, Ariente Neto, R, Mattos, DLD, Cardoso, A, Gonçalves, H and Arezes, P (2021) Decision-making framework for implementing safer human-robot collaboration workstations: System dynamics modeling. Safety 7(4), 75.10.3390/safety7040075CrossRefGoogle Scholar
Carlson, T and Demiris, Y (2012) Collaborative control for a robotic wheelchair: Evaluation of performance, attention, and workload. IEEE Transactions on Systems, Man, and Cybernetics, Part B 42(3), 876888.10.1109/TSMCB.2011.2181833CrossRefGoogle ScholarPubMed
Cruz, F, Dazeley, R, Vamplew, P and Moreira, I (2021) Explainable robotic systems: Understanding goal-driven actions in a reinforcement learning scenario. Neural Computing and Applications 35, 1811318130.10.1007/s00521-021-06425-5CrossRefGoogle Scholar
Damacharla, P, Javaid, AY, Gallimore, JJ and Devabhaktuni, VK (2018) Common metrics to benchmark human-machine teams (HMT): A review. IEEE Access 6, 3863738655.10.1109/ACCESS.2018.2853560CrossRefGoogle Scholar
Dao, J., Liu, R., Solomon, S., & Solomon, S. (2024). State anxiety biomarker discovery: Electrooculography and electrodermal activity in stress monitoring. arXiv preprint arXiv:2411.17935.Google Scholar
Darvish, K, Simetti, E, Mastrogiovanni, F and Casalino, G (2021) A hierarchical architecture for human–robot cooperation processes. IEEE Transactions on Robotics 7(2), 567586.10.1109/TRO.2020.3033715CrossRefGoogle Scholar
Del Carretto Di Ponti E Sessam, E (2023) Exploring the impact of stress and cognitive workload on eye movements: A preliminary study (Doctoral dissertation, Politecnico di Torino).Google Scholar
di Fiore, A and Schneider, S (2017) Crowd-scanning: The future of open innovation and artificial intelligence. LSE Business Review. Retrieved May 9, 2024.Google Scholar
Fahn, CS, Chen, SC, Wu, PY, Chu, TL, Li, CH, Hsu, DQ and Tsai, HM (2022) Image and speech recognition technology in the development of an elderly care robot: Practical issues review and improvement strategies. Healthcare 10(11), 2252.10.3390/healthcare10112252CrossRefGoogle ScholarPubMed
Floridi, L (2020) What the near future of artificial intelligence could be. In 2019 Yearbook of the Digital Ethics Lab. Springer, pp. 127142.10.1007/978-3-030-29145-7_9CrossRefGoogle Scholar
Gazetta, G, Miller, C, Clemency, B, Tanaka, K, Hackett, M, Norfleet, J, … and Cavuoto, L (2023) Evaluating workload indicators for learning during stress exposure training of endotracheal intubation. In Proceedings of the Human Factors and Ergonomics Society Annual Meeting, 67(1). Thousand Oaks, CA: SAGE Publications, pp. 574578.Google Scholar
Gjoreski, M, Luštrek, M, Gams, M and Gjoreski, H (2017) Monitoring stress with a wrist device using context. Journal of Biomedical Informatics 73, 159170.10.1016/j.jbi.2017.08.006CrossRefGoogle ScholarPubMed
Guo, H, Pu, X, Chen, J, et al. (2018) A highly sensitive, self-powered triboelectric auditory sensor for social robotics and hearing aids. Science Robotics 3(20), 2516.10.1126/scirobotics.aat2516CrossRefGoogle ScholarPubMed
Gupta, C, Johri, I, Srinivasan, K, Hu, YC, Qaisar, SM and Huang, KY (2022) A systematic review on machine learning and deep learning models for electronic information security in mobile networks. Sensors 22(5), 2017.10.3390/s22052017CrossRefGoogle ScholarPubMed
Hemakom, A, Atiwiwat, D and Israsena, P (2023) ECG and EEG-based detection and multilevel classification of stress using machine learning for specified genders: A preliminary study. PLoS One 18(9), e0291070.10.1371/journal.pone.0291070CrossRefGoogle ScholarPubMed
Huang, CM, Andrist, S, Sauppé, A and Mutlu, B (2015) Using gaze patterns to predict task intent in collaboration. Frontiers in Psychology 6, 1049.10.3389/fpsyg.2015.01049CrossRefGoogle ScholarPubMed
Ji, Z, Liu, Q, Xu, W, Yao, B, Liu, J and Zhou, Z (2021) A closed-loop brain-computer interface with augmented reality feedback for industrial human-robot collaboration. The International Journal of Advanced Manufacturing Technology 124, 30833098.10.1007/s00170-021-07937-zCrossRefGoogle Scholar
Kanjo, E, Younis, EM and Ang, CS (2019) Deep learning analysis of mobile physiological, environmental and location sensor data for emotion detection. Information Fusion 49, 4656.10.1016/j.inffus.2018.09.001CrossRefGoogle Scholar
Katmah, R, Al-Shargie, F, Tariq, U, Babiloni, F, Al-Mughairbi, F and Al-Nashash, H (2021) A review on mental stress assessment methods using EEG signals. Sensors 21(15), 5043.10.3390/s21155043CrossRefGoogle ScholarPubMed
Kirtay, M, Hafner, VV, Asada, M and Oztop, E (2023) Trust in robot-robot scaffolding. IEEE Transactions on Cognitive and Developmental Systems 15(4), 18411852.10.1109/TCDS.2023.3235974CrossRefGoogle Scholar
Kosa, G, Morozov, O, Lehmann, A, Pargger, H, Marsch, S and Hunziker, P (2023) Robots and intelligent medical devices in the intensive care unit: Vision, state of the art and economic analysis. IEEE Transactions on Medical Robotics and Bionics 5(1), 25763202.10.1109/TMRB.2023.3240537CrossRefGoogle Scholar
Krupitzer, C, Müller, S, Lesch, V, Züfle, M, Edinger, J, Lemken, A and Becker, C (2020) A survey on human-machine interaction in industry 4.0. arXiv preprint arXiv:2002.01025.Google Scholar
Lauterbach, B, Sauer, S, Gottlieb, J, Sürie, C and Benz, U (2019) Transportation Management with SAP (3rd ed.). Rheinwerk Publishing, Inc. Available at https://www.perlego.com/book/2826166/transportation-management-with-sap-pdf (Original work published 2019).Google Scholar
Lei, Y, Su, Z and Cheng, C (2023) Virtual reality in human-robot interaction: Challenges and benefits. Electronic Research Archive 31(5), 23742408.10.3934/era.2023121CrossRefGoogle Scholar
Li, S, Zheng, P, Liu, S, Wang, Z, Wang, XV, Zheng, L and Wang, L (2023) Proactive human-robot collaboration: Mutual-cognitive, predictable, and self-organizing perspectives. Robotics and Computer-Integrated Manufacturing 81, 102510.10.1016/j.rcim.2022.102510CrossRefGoogle Scholar
Lopez-de-Ipina, K, Iradi, J, Fernandez, E, Calvo, PM, Salle, D, Poologaindran, A and Suckling, J (2023) HUMANISE: Human-inspired smart management, towards a healthy and safe industrial collaborative robotics. Sensors 23(3), 1170.10.3390/s23031170CrossRefGoogle ScholarPubMed
McColl, D and Nejat, G (2014) Recognizing emotional body language displayed by a human-like social robot. International Journal of Social Robotics 6(2), 261280.10.1007/s12369-013-0226-7CrossRefGoogle Scholar
McNeese, NJ, Schelble, BG, Canonico, LB and Demir, M (2021) Who/what is my teammate? Team composition considerations in human–AI teaming. IEEE Transactions on Human-Machine Systems 51(4), 288299.10.1109/THMS.2021.3086018CrossRefGoogle Scholar
Merlo, E, Lamon, E, Fusaro, F, Lorenzini, M, Carfì, A, Mastrogiovanni, F and Ajoudani, A (2023) An ergonomic role allocation framework for dynamic human–robot collaborative tasks. Journal of Manufacturing Systems 67, 111121.10.1016/j.jmsy.2022.12.011CrossRefGoogle Scholar
Merritt, T, McGee, K, Chuah, TL and Ong, C (2011). Choosing human teammates: Perceived identity as a moderator of player preference and enjoyment. In Proceedings of the 6th International Conference on Foundations of Digital Games, pp. 196203. https://dl.acm.org/doi/10.1145/2159365.2159392Google Scholar
Mitro, N, Argyri, K, Pavlopoulos, L, Kosyvas, D, Karagiannidis, L, Kostovasili, M, et al. (2023) AI-enabled smart wristband providing real-time vital signs and stress monitoring. Sensors 23(5), 2821.10.3390/s23052821CrossRefGoogle ScholarPubMed
Nass, C, Fogg, BJ and Moon, Y (1996) Can computers be teammates? International Journal of Human-Computer Studies 45(6), 669678.10.1006/ijhc.1996.0073CrossRefGoogle Scholar
Nath, RK and Thapliyal, H (2021) Smart wristband-based stress detection framework for older adults with cortisol as stress biomarker. IEEE Transactions on Consumer Electronics 67(1), 3039.10.1109/TCE.2021.3057806CrossRefGoogle Scholar
Nguyen, TA and Zeng, Y (2012) A theoretical model of design creativity: Nonlinear design dynamics and mental stress-creativity relation. Journal of Integrated Design and Process Science 16(3), 6588.Google Scholar
Nguyen, TA and Zeng, Y (2017) Effects of stress and effort on self-rated reports in experimental study of design activities. Journal of Intelligent Manufacturing 28, 16091622.10.1007/s10845-016-1196-zCrossRefGoogle Scholar
Orsag, L, Stipancic, T and Koren, L (2023) Towards a safe human–robot collaboration using information on human worker activity. Sensors 23(3), 1283.10.3390/s23031283CrossRefGoogle ScholarPubMed
Pereira, D, Bozzato, A, Dario, P and Ciuti, G (2022) Towards foodservice robotics: A taxonomy of actions of foodservice workers and a critical review of supportive technology. IEEE Transactions on Automation Science and Engineering 19(3), 18201858.10.1109/TASE.2021.3129077CrossRefGoogle Scholar
Perez-Valero, E, Vaquero-Blasco, MA, Lopez-Gordo, MA and Morillas, C (2021) Quantitative assessment of stress through EEG during a virtual reality stress-relax session. Frontiers in Computational Neuroscience 15, 684423.10.3389/fncom.2021.684423CrossRefGoogle ScholarPubMed
Perrin, X, Chavarriaga, R, Colas, F, Siegwart, R and Millán, JDR (2010) Brain-coupled interaction for semi-autonomous navigation of an assistive robot. Robotics and Autonomous Systems 58(12), 12461255.10.1016/j.robot.2010.05.010CrossRefGoogle Scholar
Pop-Jordanova, N and Pop-Jordanov, J (2020) Electrodermal activity and stress assessment. Prilozi 41(2), 515.10.2478/prilozi-2020-0028CrossRefGoogle ScholarPubMed
Prasad, A, Kumar, A and Prasad, P (2024) Conceptualizing a digital twin model for natural gas retailing in a geographic area in India. SPAST Reports 1(1). https://spast.org/ojspath/article/view/4791/255.Google Scholar
Prendergast, JM, Balvert, S, Driessen, T, Seth, A and Peternel, L (2021) Biomechanics aware collaborative robot system for delivery of safe physical therapy in shoulder rehabilitation. IEEE Robotics and Automation Letters 6(4), 71777184.10.1109/LRA.2021.3097375CrossRefGoogle Scholar
Rahma, ON, Putra, AP, Rahmatillah, A, Putri, YSAKA, Fajriaty, ND, Ain, K and Chai, R (2022) Electrodermal activity for measuring cognitive and emotional stress level. Journal of Medical Signals & Sensors 12(2), 155162.10.4103/jmss.JMSS_78_20CrossRefGoogle ScholarPubMed
Shannon, CE (1948) A mathematical theory of communication. The Bell System Technical Journal 27(3), 379423.10.1002/j.1538-7305.1948.tb01338.xCrossRefGoogle Scholar
Shannon, CE and Weaver, W (1949) The Mathematical Theory of Communication. University of Illinois PressGoogle Scholar
Soori, M, Arezoo, B and Dastres, R (2023) Artificial intelligence, machine learning, and deep learning in advanced robotics: A review. Cognitive Robotics 3, 5470.10.1016/j.cogr.2023.04.001CrossRefGoogle Scholar
Sreedevi, AG, Harshitha, TN, Sugumaran, V and Shankar, P (2022) Application of cognitive computing in healthcare, cybersecurity, big data, and IoT: A literature review. Information Processing & Management 59(2), 102888.10.1016/j.ipm.2022.102888CrossRefGoogle Scholar
Sun, X, Zeng, Y and Zhou, F (2011) Environment-based design (EBD) approach to developing quality management systems: A case study. Journal of Integrated Design and Process Science 15(2), 5370.Google Scholar
Varol, HA, Sup, F and Goldfarb, M (2009) Multiclass real-time intent recognition of a powered lower limb prosthesis. IEEE Transactions on Biomedical Engineering 57(3), 542551.10.1109/TBME.2009.2034734CrossRefGoogle ScholarPubMed
Wei, K and Ren, B (2018) A method on dynamic path planning for robotic manipulator autonomous obstacle avoidance based on an improved RRT algorithm. Sensors 18(2), 571.10.3390/s18020571CrossRefGoogle Scholar
Yang, J, Yang, L, Quan, H and Zeng, Y (2021) Implementation barriers: A TASKS framework. Journal of Integrated Design and Process Science 25(3–4), 134147.Google Scholar
Ye, W, Bullo, F, Friedkin, N and Singh, AK (2022) Modeling human-AI team decision making. arXiv preprint arXiv:2201.02759.Google Scholar
Yonga Chuengwa, T, Swanepoel, JA, Kurien, AM, Kanakana-Katumba, MG and Djouani, K (2023) Research perspectives in collaborative assembly: A review. Robotics 12(2), 37.10.3390/robotics12020037CrossRefGoogle Scholar
Yun, JJ, Lee, D, Ahn, H, Park, K and Yigitcanlar, T (2016) Not deep learning but autonomous learning of open innovation for sustainable artificial intelligence. Sustainability 8(8), 797.10.3390/su8080797CrossRefGoogle Scholar
Zanchettin, AM, Messeri, C, Cristantielli, D and Rocco, P (2022) Trajectory optimisation in collaborative robotics based on simulations and genetic algorithms. International Journal of Intelligent Robotics and Applications 6(4), 707723.10.1007/s41315-022-00240-4CrossRefGoogle Scholar
Zeng, Y (2002) Axiomatic theory of design modeling. Journal of Integrated Design and Process Science 6(3), 128.Google Scholar
Zeng, Y (2011) Environment-based design (EBD). In International Design Engineering Technical Conferences and Computers and Information in Engineering Conference (Vol. 54860, pp. 237250). ASME Digital Collection.Google Scholar
Figure 0

Table 1. Communication channels and their application domains

Figure 1

Figure 1. Relationship between stress and creativity/performance.

Figure 2

Figure 2. Connection between communication and collaboration.

Figure 3

Figure 3. Possible task distribution map among team members.

Figure 4

Figure 4. Workload reallocation between humans, robots, and their collaborations in smart systems (the details of “Identify Intervention Opportunities” are available in Figure 3; human, robot, human–robot expressions are given in Equations 11–16)

Figure 5

Figure 5. Overview of SAP TM integration when not embedded in SAP S/4 HANA (adapted from Lauterbach et al., 2019).

Figure 6

Figure 6. Illustration of robot–robot communication; systems integration in SAP S/4 HANA sidecar scenario (robots are subsystems such as SAP LBN and SAP S/4 HANA).

Figure 7

Figure 7. Dynamic collaboration between human–SAP systems (robots): Navigating the smart SAP S/4 HANA sidecar ecosystem with human in the loop.

Figure 8

Table 2. Possible tasks available for human, robot, human–robot in the LBN (business network for logistics) system of SAP S/4 HANA sidecar scenario

Figure 9

Figure 8. Generating workload zones for human, robot, human–robot in the LBN system of SAP S/4 HANA sidecar scenario.