This article investigates a blossoming research field: face-to-face communication. The performance and the robustness of the technological components that are necessary to the implementation of face-to-face interaction systems between a human being and a conversational agent – vocal technologies, computer vision, image synthesis, dialogue comprehension and generation, etc. – have now come to maturity. We draw a sketch for a research program centred on the modelling of the many perception-action loops that are involved in the interaction processing and on the dynamic settings of these loops by the many comprehension levels concerning the scene in which human beings, robots and animated conversational agents will inevitably be immersed.