As presented in the introduction of this book, three telepresence devices were used in the seminar: the telepresence robots Beam and Kubi as well as the web conferencing software program Adobe Connect. In this chapter, in the light of work in interactive multimodal communication and more specifically in robot-mediated communication (Herring 2013 ; Takayama and Go 2012 ; Takayama and Harris 2013 ; Neustaedter et al. 2016 ; Sirkin et al. 2011 ; Gaver 1992), we interrogate the notion of artefactual presence through a comparative study of the affordances of these devices. We ask how the effects of presence related to each device define an artefactual presence or an interactional presence, depending on the interactional co-construction implemented by the participants. To what extent do effects of presence vary according to the artefact or a particular device and in the co-construction of its use by the different members? Our study is based on interviews with participants (Amélie, Jean-François, Samira, Christelle) which we cross-reference with the analysis of certain critical moments in the video corpus of the sessions.

Theoretical framework

Several studies in multimodal communication and human-computer communication have demonstrated the characteristics of the different telepresence devices used and their effects on communication.

The researchers point to parameters such as rotation or field of view, but also to characteristics of the mediated space (within which the interaction takes place), such as the lack of symmetry in the transmission and reception of sound and image between the different participants.

An approach focused on interaction and not on geographical location

In studies on meetings involving co-located and remotely located participants, the geographical location is often emphasised. The meeting with the co-located participants is defined as a “hub” while the remote location, present in the form of a “proxy” or artefact enabling the remote participant to participate (screen, video camera, speaker, microphone), is a “satellite” (Sirkin et al. 2011, 163). This “hub/satellite” view thus focuses on the technical device and not on the experience of the interaction.

In our study, in contrast, we do not consider one place as the “hub” and another as a “satellite”. This is because there were sometimes fewer participants in the seminar room than online, and because there were different “satellite locations” (as opposed to a single satellite individual in the case of Sirkin et al.). Thus, in order to correspond to the actual lived experience, the present/remote dualism must be overcome in favour of an approach centred on interaction and not on geographical location.

Engagement and mobility

Other researchers have studied telepresence devices from the perspective of interactional engagement and movement effects. Herring (2013, 1), for example, points to difficulties with Adobe Connect-type devices, including sound and visualisation issues, participant fatigue, difficulty in feeling engaged in the interaction, and frustrations with speaking difficulties:

However, currently popular teleconferencing tools (e.g., Skype, Adobe) are limited in various respects. Even when video is added to audio communication, remote participants often cannot see or hear everyone at the remote location, may feel disengaged and fatigued (because more effort is required to pay attention), and may experience interactional frustration due to difficulty getting the floor and identifying who is speaking (e.g., Egido 1990 ; Sirkin et al. 2011).

This difficulty in feeling engaged is partly due to the lack of possibility of movement. According to Gaver (1992, 21), the possibility of exploring the environment through movement is not a constitutive characteristic of mediated spaces since the cameras and microphones are fixed and controlled by the people on site. According to the author, being online feels more like watching television than having control over a perceptual exploration.

Conversely, “kinetic proxies” (which can be set in motion, such as the Beam or Kubi) allow for a hybrid approach (Sirkin et al. 2011, 166) by combining motion and video image, as opposed to a robot which would be a simple avatar of the remote person. The artefact thus represents the remote participant and reminds the group of their presence through movement or rotation. According to the study led by Sirkin et al., the quality of conversational engagement is higher when motion is possible:

The motorized action brought the remote person to life. Hub participants were able to perceive the satellite’s attention in motion through the swivelling of the display (2011, 176).

Compared to a video-conferencing device, the Kubi’s rotational mobility creates an additional effect of presence (Herring 2013, 3). In addition to its small size, its screen can be rotated to follow the conversation. However, the Kubi cannot be entirely piloted in the same way as the Beam (Herring 2013, 3) since it has to be carried from one place to another by a human agent.

Sirkin et al. (2011) have also revealed undesirable effects of motion. Rotation can be interpreted as a disruption, and as an interruption for the remote participant who must operate this rotation. When the participant rotates the device to face an interlocutor, it can be perceived as though they are “turning their back” on other participants:

screen motion towards one person is more akin to turning one’s back (rather than one’s head) towards someone else (Sirkin et al. 2011, 164).

Another difficulty is that head movements and rotational movements of artefacts are not always interpreted in the same way, as participants seek to attribute intelligibility to these movements, even though some may be mere incidents lacking any communicative purpose.

We study these various movement effects in our context, and analyse below how they combine with characteristics of autonomy (see “Analysis and results”).

Disrupted reciprocity of perceptions in the mediated space

In the field of human-computer interaction, Gaver (1992, 17) compares the affordances of an unmediated situation with a media space defined as a space created by “computer-controllable networks of audio and video equipment used to support synchronous collaboration”. He identifies the following characteristics of the latter space: distant collaboration, restricted field of vision, impossible detailed inspection, limited peripheral awareness, biased sound transmission, limited perceptual exploration, and discontinuity of spaces that make speech turns and communicative behaviours more difficult. These characteristics apply to the situation we analyse in this study.

Media spaces are also characterised by anisotropy, i.e., the non-reciprocity of perceptions in the mediated spaceSee chapter “Attentional affordances in an instrumented seminar”.↩︎, in contrast to air (i.e., face-to-face communication). This term comes from physics and is applied when the properties of an object vary according to direction. As Gaver (1992, 23) explains, air is isotropic and allows for reciprocity of perceptionsSee chapter “Attentional affordances in an instrumented seminar”.↩︎:

Air is isotropic with respect to light and – unless it is moving – with respect to sound as well. This means that air affords reciprocal communication, that people can predict what their partners will see and hear by what they themselves see and hear.

Screen mediation therefore disrupts this reciprocity by making the exchanges anisotropic. This is reflected, for example, in the difficulty for participants in the seminar room to know precisely what a remote participant is looking at. Sirkin et al. (2011) point out the importance for the remote participant to have an broad view of a space, so as to be able to follow which attentional fociSee chapter “Attentional affordances in an instrumented seminar”.↩︎ are being activated. Many parameters may be unknown to the participants in the seminar room (Sirkin et al. 2011, 164): this is the case, for example, of the angle of view of the remote participant’s camera or the size of their computer screen. Furthermore, according to these authors, the “TV presenter effect” makes face-to-face participants feel as though they are all being looked at simultaneously if the satellite is looking at the camera, or all neglected if it looks away. They also note the “skip-over effect”, whereby the remote participant tends to be neglected despite the presence of the face-to-face artefact that represents them.

At the end of this brief literature review, we note that the complexity of the situation is due in particular to the anisotropy of the mediated space, as well as to the affordances of the telepresence devices used, depending on the possibilities of movement or rotation, but also depending on the way these movements happen and are interpreted by the participants. On the basis of these elements, we can now analyse the interactional characteristics of our corpus and the effects of presence they generate.

Analysis and results

Remote communication devices present different potentialities of movement, vision and hearing, which have impact in terms of presence effects, around issues of transmission/reception, and visibility/invisibility or presence/absence: one can be present and invisible to others or visible and absent. These effects of presence define an artefactual presence or an interactional presence, depending on the interactional co-construction implemented by the participants. We define artefactual presence as the presence of the object with a reduced possibility of interaction, as opposed to interactional presence which allows an individual to take their place in the interaction without hindrance. As we will see below, it is mainly the issues of autonomy of movement and visual and sound adjustment that determine the objectal or interactional status of the pilot and her artefact. These presence effects bring into play conviviality, stealthWe mean stealth in the military sense of being designed to avoid detection by using a variety of technologies that reduce signature.↩︎, reactions to solicitation or orders as well as temporality (beginning/end).

Autonomy of rotation and movement

Rotations and movements are characterised by parameters of autonomy: are movements possible or impossible, autonomously driven or dependent on others? The movement must also be considered in terms of the starting position: was this chosen by the user? Additionally, is the movement translational and/or rotational (is it a head or artefact rotation; slow or fast rotation; discreet or noisy rotation)? A distinction will therefore be made between movement in production and the effect of movement in reception.

Movement range of the Beam

Our experimentation showed us that the Beam offered its user only a relative autonomy of movement.

In Session 2, we find another illustration of the limitations of the Beam’s possibilities, when the presence of the artefact seems to be spatially related to group presence.

When the speakers move their table to get closer to the microphone that is transmitting the sound to the remote participants, Amélie, the Beam user, has situated the Beam between this table and the tables behind her, where the participants in Lyon are seated.

Even if she experiences this change of configuration as uncomfortable, she cannot move because she is hampered by the tables which limit her movements and make it difficult to move to another part of the room:

It’s true that, wow, when they came closer, I felt that the normal distance, the interpersonal distance between people, was completely disrespected, they were practically up against me […] it wasn’t very comfortable.

In this case, the comfort of the remote person is replaced by that of the participants in the seminar room.

In other cases, the Beam pilot can sometimes exercise autonomy in piloting the artefact, as we explain now.

Effect of presence of the Beam’s autonomous piloting

In Session 3 (group work), we see an example of the effect of presence of the Beam’s movement, which is also commented on by the pilot in an interview.

During the group discussion, the Beam starts to rotate slightly on its wheels and then suddenly moves towards the centre of the room. Christine exclaims, “Oh, she scared me!” and Morgane, putting her hand over her heart, says “Oh my goodness!” However, the interruption does not last more than 5 seconds.

From our analyses, artefactual or interactional presence depends on perception by others as much as oneself. In an anisotropic mediated space, participants rely as much on the perception of others as on their own perception in an attempt to reconstruct a comprehensive perception and allow the interaction to function.

Artefactual presence of the Kubi

Our analyses identify different situations in which the presence of the artefact pilot is reduced to artefactual presence.

Artefactual presence can take over, for example, through a request for permission to move the artefact by someone else. This request points to a potential interactional presence, even if sometimes the artefact is rotated even though the pilot has not responded to a request for permission.

So although the Kubi does have a certain rotational autonomy, it still needs to be mastered and must correspond to the configuration at a given moment and in a given space. In some cases, such as above, artefactual presence is brought to the fore.

Audio-visual autonomy

In addition to the effects of movement, the angle of vision is also a determining parameter. As a whole, the artefactual communication situation at stake is complex. Firstly, the Kubi user has to choose a setting that allows her to adjust the view on the computer screen. Secondly, the camera within the seminar room that transmits the image to Adobe Connect must also be adjusted to produce suitable visual content. The combination of these two actions allows the Kubi user to feel like they are part of the situation. While face-to-face participants can visually scan a space, remote artefact participants may not see the technical solutions deployed in the room, for example, or certain actions, such as drawing on the board. For the purposes of this paper, we distinguish between audio-visual autonomy and autonomy of movement, but audio-visual perception and autonomous or provoked movement are of course interdependent, because if the angle of vision is a barrier to interactional felicity, then remediation involves positional adjustment, either autonomous or assisted.

Limitations in the field of vision and rotation of the Kubi

The Kubi’s field of vision is limited and must be set by the pilot, which has implications in terms of artefactual presence.

Interactional presence implies at times an artefactual presence; in order to be or become present, one has to transiently pass through moments of artefactual presence, for instance,  when one is assisted by another participant and is momentarily recategorised, not as a pilot, but as an object.

Adjusting the Kubi’s field of vision

The Kubi’s limited field of vision deprives the pilot of some of the interactions that take place in Lyon.

Example during Session 3

In Session 3, while Christelle is making her positioning adjustments, Morgane goes over to the Beam. Christelle misses this action because she is readjusting her field of vision at the time and does not see the participants in the seminar room.

2020-12-01 en field of vision, limitation, adjustment, Kubi IMPEC seminar

Adjustment of the camera’s field of vision in the seminar room

The Kubi pilot depends on the help of the participants in Lyon to adjust her angle of vision, which moderates her interactional power. In contrast, the Beam pilot can adjust her field of view autonomously. She has greater agency in this respect.

The “back” of the Beam

Even though the Beam pilot can move the device autonomously, she still has a limited field of vision of what is happening behind the artefact. Our analysis of the video demonstrates this limitation, and the Beam pilot confirms it in an interview.

The choices made about the issues of transmission and reception, such as adapting the Adobe Connect software for hybrid use (a group in a room/individuals in separate locations), had consequences for audio reception (e.g., participants on Adobe Connect could not hear participants in the room) or visual reception (e.g., participants in the room could not tell apart the silhouettes of participants on Adobe Connect.) who were against the light. These different types of perceptual (motion and audio-visual) autonomy influence the availability and participation regimes allowing the interaction to function.

Participatory autonomy

The technical choices and the potentialities of each device have an impact on participation. By participatory autonomy we mean the regulation initiated by the individual of their involvement in the interaction.

Issues of availability for being spoken to can appear according to whether the artefact allows a participant to participate in the interaction in reception or in transmission.

Difficulty in calling the Beam

The participants found it easier to address the Adobe Connect users than the Beam pilot. Morgane explains in an interview that she tried several times and through several channels (signs, chat, email, SMS) to contact Amélie, the Beam pilot, without success.

Difficulty in giving a strong visual signal when using the Kubi or Adobe Connect

The Kubi or Adobe Connect users also found it difficult to speak up. In the Kubi, unless you speak up and impose your voice, you cannot address the rest of the group.

In the Adobe Connect chat function it is equally difficult to send a strong signal.

More broadly, we can thus question whether participation in the interaction is subject to regimes that could be described as artefactual, in the sense that they are dependent on the artefact, or the telepresence device used.

The chat as a space for autonomous or relayed communication

Still from the point of view of participatory autonomy, we see in this section that the communication space of the chat is ambivalent, in that it sometimes allows participation in the overall interaction, not in an autonomous way but through the on-site participants, and sometimes generates a separate space for autonomous communication.

The remote participants using Adobe Connect all have access to the chat. Some participants in the room are also connected to Adobe Connect and use the chat, which is projected, but despite this, the content of the chat sometimes has to be relayedSee chapter “Attentional affordances in an instrumented seminar”.↩︎.

The social aspect of the chat is pointed out in interviews by various members of the team.

On the contrary, Amélie, the Beam pilot, indicates in an interview that it was difficult for her to access the communication space created by the chat function (unless she manipulated both the Beam and Adobe Connect interfaces simultaneously). Jean-François and Samira both point out the compartmentalisation aspects of the chat.

The chat is sometimes a space for autonomous exchanges, whose participants no longer want to be part of the general group interaction.

Below we explore constraints or orders in terms of participation in the interaction from the point of view of autonomy and intention. These elements involve screen presence, the necessity of thinking about framing of one’s own picture from the viewer’s point of view, or hyper-exposure when speaking.

Over-ratification and over-exposure

Based on the concept of ratification (Goffman 1981), we analyse examples of what we call “over-ratification”: an exposed and undesirable ratification from the point of view of remote participantsSee chapter “New norms of politeness in digital contexts”.↩︎; and of what we call “hyper-exposure”, i.e., taking a conversational turn without wanting to.

Unwanted exposure sometimes occurs via an incidental audio hyper-exposure, which takes over a speech turn without the person intending to. For example, when the Beam user sneezes at homeSee chapters “New norms of politeness in digital contexts” and “Artefacted intercorporeality, between reification and personification”.↩︎, due to the anisotropy of the mediated space, she does not realise that the sound effect is dramatically amplified in the room.

Voluntary hypo-exposure

The anisotropy of the mediated space (Gaver 1992, 234), i.e., the fact that the space has different characteristics depending on the orientation, makes it possible to spy on others, or to be artefactually present while being absent, or on the contrary to “arrive” unnoticed by the participants in the room.

The Beam’s “zoom” function in particular can give the user a stealthy presence, unbeknownst to the other participants.

But while the robot’s movement creates an effect, its immobility is not synonymous with inactivity, even if the participants in the room do not realise it.

Generally speaking, affordances are negotiated by taking into account a set of parameters, such as the delayed adjustments of the Beam in this case.

We have seen through this study that the effect of presences linked to each device define an artefactual presence or an interactional presence, depending on the interactional co-construction implemented by the participants. The issues of autonomy of movement and visual and sound adjustment determine the objectal or interactional status of the user and the device they are using.

The duality between objectal and interactional status does not presume interactional felicity. Artefactual presence sometimes corresponds to the user’s intention in that it allows discretion, just as interactional presence can then go against the user’s intention in that it sometimes creates situations of over-ratification. Artefactual presence can be experienced against one’s will (e.g., the Beam is moved during the break and when the pilot reconnects it has no cues) or can be taken advantage of (e.g., the Beam pilot uses the zoom function unobtrusively while the participants in the room are unaware that the pilot is connected, or the Beam is moved by others, which facilitates the interaction). Similarly, interactional presence can be experienced against one’s will (e.g., the floor is explicitly given to participants online when they have nothing in particular to say at that moment) or can be taken advantage of (e.g., when a request in the chat is relayed to the room verbally).

To conclude this analysis, we propose the following table to sum up these categories:

In addition to an analysis centred around the characteristics of the telepresence devices used, the regulations involving the regimes of autonomy are played out in the intentions of each person and their interpretation, which allow interactions to be co-constructed in a polyartefacted hybrid context, within an artefacted-interactional community.