Joint Action for Human-Robot Interaction




From  the  very  beginning, fundamental and multidisciplinary discussions have been at the heart of the project. It has led to an original contribution from  psychologists to robotics "Contribution of Developmental Psychology to the Study of Social Interactions: Some Factors in Play, Joint Attention and Joint Action and Implications for Robotics" by Hélène Cochet and Michèle Guidetti(2018).

We have also made public a mirror of the project bibliography: and we have conducted a series of workshop on Joint Action (

A  discussion  has  been  setup  from  the  beginning  of  the  project between  the  three teams. It has led to a joint paper entitled "Addressing Joint Action Challenges in HRI: Insights from Psychology and Philosophy"co-authored byall the participants of the project in the three labs. The paper has been accepted and will be published in the following months. The abstract of the paper is the following: The vast expansion of research in human-robot interactions (HRI) these last decades has been accompanied by the design of increasingly skilled robots for engaging in joint actions with humans. However, these advances have encountered significant challenges to ensure fluent interactions and sustain human motivation through the different steps of jointaction. After exploring current literature on joint action in HRI, leading to a more precise definition of these challenges, the present article proposes some perspectives borrowed from psychology and philosophy showing the key role of communication in human interactions. From mutual recognition between individuals to the expression of commitment and social expectations, we  argue that communicative cues can facilitate coordination, prediction, and motivation in the context of joint action. The description of several notions thus suggests that some communicative  capacities can be implemented in the context of joint action for HRI, leading to an integrated perspective of robotic communication.




DACOBOT architecture

The project, along with the H2020 MuMMER project leads us to implement a new instance of LAAS Robotic Architecture for interaction in the continuity ofour previous work(Lemaignan etal.2017). We pursued this work started in our lab by expanding the componentsfeatures and refining, consolidating the interactions between these components. Thec omponent names and functions of both architectures are the same but most of the implementations are all new and the way they rely on, interact  with each other, is as well. All the components of the DACOBOT (Deliberative Architecture for COllaborative roBOT) are designed to be human-aware,making the global system human-aware, which is quite rare.

With this architecture, we focus on a given type of Human-Robot Interaction: collaborative tasks, joint actions. In this context, the human and the robot share a common  space and exchange information through multiple modalities (speech, gesture, gaze). The robot should be able to act on its environment, by manipulating objects and navigate among humans. This function is assured by the Motion Planners and Excutors. In order to be aware of its environment, the robot needs perception modalities which are handled by the sensorimotorlayer, it can be cameras, lasers, motion capture, force sensors, etc. To avoid each component having to process the data itself  in order to be able to use them, the Situation Assessment, converts them from geometric data to symbolic data. Moreover, it endows the robot’s visual perspective-taking.  Then, these data are stored in Knowledge Bases, one for the robot and another for the human. Finally, the heart of the architecture, the decision-making process is located in the Supervisor. In order to make its decisions, the Supervisorrelies on the KBs, the communication  through  the  Natural  Language  Processing(NLP)  and  especially the Task Planners. Once the decision made, it controls the robot through the Motion Planners and Executors and the NLP.

This part of the work have benefit  from the work of Amandine Mayima and Kathleen Belhassein as well as Guillaume Sarthou, Guilhem Buisan, Phani Teja, Yoan Sallami, Jules Waldhart in the framework of the H2020 MuMMER project



We proposed and implemented The Director Task: a Psychology-Inspired Task to Assess Cognitive andInteractive Robot Architectures.

Assessing robotic architecture  for  Human-Robot  Interaction  can  be  challenging  due  to  the number of features a robot has to endow to perform an acceptable interaction. While everyday-inspired tasks are interesting as reflecting a realistic use of such robots, they often contain a lot of unknown and uncontrolled conditions and specific robot behavior can be hard to test. With this task, we propose a new psychology-inspired task, gathering perspective-taking, planning, knowledge  representation  with  theory  of  mind,  manipulation,  and  communication.  Along with  a  precise  description  of  the  task  allowing  its  replication,  we  present  a  cognitive  robot architecture able to perform it in its nominal cases. We finally suggest some challenges and evaluations for the Human-Robot Interaction research community, all derived from this easy-to-replicate task.This work has been appreciated by the communauty since it has been selected in the Finalists for Best Paper Award at RO-MAN 2021. This part of the work have benefit from the work of Amandine Mayima and Kathleen Belhassein as well as Guillaume Sarthou and  Guilhem Buisan (and Yannick Riou for the technical aspects).


An important tradition in philosophy holds that in order to successfully perform a joint action, the participants must be capable of establishing commitments on joint goals and shared plans.  We  have  studied  how  commitments  could  facilitate  coordination  and  cooperation  by stabilizing  expectations,  reducing  the  uncertainty  of  the  interaction,  providing  reasons  to cooperate or improving action coordination in a human-robot settings.

However, to serve these functions must be credible in the first place. In the paper "Joint actions, commitments and the need to belong", Fernandez-Castro and Pacherie have offered an account of what motivates humans to abide by their commitments. We first argue  that  while  reputation  management  and  social  emotions  certainly  playa  role  in motivating us to act as committed, there are both theoretical and empirical reasons to think that neither  captures  the  most  basic  motivational  force  at work  in  sustaining  commitments. We  propose  instead  that  the  need  to  belong,  i.e.,  the  need  toaffiliate  with  others  and  form long-lasting bonds with them, is what primarily motivates us to interact and engage with those around us and act so as to preserve and reinforce the bonds we have forged with them.

In  addition,  in a submitted paper on Commitments and the sense of Joint Agency, Fernandez-Castro and Pacherie have explored  three  complementary  ways  in  which commitments may shape the sense of joint agency and its main characteristics. First, they can generate  reliable  higher-level  expectations  about  the  behavior  of  co-agents,  which  would positively  impact  our  sense  ofjoint  agency  to  the  extent  that  these  expectations  are  met. Second, they enable the recipient to demand that the author behaves in accordance with the commitments  and  the  expectations  they  generate  and  to  sanction,  reprimand  or  express dissatisfaction when that is not the case, allowing the recipient to exert more control over the joint action in general. Finally, expressing commitments implicitly or explicitly can modify the affective dimension of collective action, and therefore, the way we experience it.

Consideration  of  the  crucial  role  of  commitments  in  human-human  cooperative  interaction suggests that social robotics should endow robots with similar competences for commitment management  in  order  to  achieve  the  objective  of  performing  joint  tasks  in  human-robot interactions. In this poster and in a submitted paper, Fernandez-Castro and Clodic considered these two philosophical approaches to commitments. These  approaches,  we  argued,  emphasize  different  behavioural  and  cognitive  aspects  of commitments  that  give  roboticists  a  way  to  give  meaning  to  monitoringand  pro-active signalling  in  joint  action  with  human  partners.  While  we  have  not  been  able  to  go  to  an implementation,  we  have  identified  ways  in  which  commitments  (and  esp.  expectations) management could benefit a robotic architecture for joint action.

In a joint paper entitled "What does it take to be a social agent?", Fernandez-Castro, Hakli and Clodic discuss ways in which the commitment credibility problem may be solved  in  the  context  of  human-robot  interactions.After  first  arguing  in  favor  of  the fundamental  role  of  the  notion  of  commitment  in  mutual  recognition,  we  show  how  the attribution  and  maintenance  of  commitment  requires  fundamental  affective  states  such  as social  emotions  or  the  prosocial  motivation  of  the  need  to  belong.  Finally,  we  survey  three proposals  on  how  social  robotics  could  implement  an  architecture  of  commitment  by addressing the centrality of these emotions and exposing their weaknesses and strengths.