Artefacted intercorporeality, between reification and personification
Version
française >
Samira Ibnelkaïd, Dorothée Furnon, « Artefacted intercorporeality,
between reification and personification », Interactions and
Screens in Research and Education (enhanced edition), Les Ateliers
de [sens
public], Montreal, 2023, isbn:978-2-924925-25-6, http://ateliers.sens-public.org/interactions-and-screens-in-research-and-education/chapter3.html.
version:0, 11/15/2023
Creative
Commons Attribution-ShareAlike 4.0 International (CC BY-SA 4.0)
And to say that things are structures, frameworks, the stars of our life: not before us, laid out as perspective spectacles, but gravitating around us (Merleau-Ponty 1960, 269).
The new technobodily modalities of onscreen presence invite us to reconceptualise the foundations of the Goffmanian definition of interaction as “the reciprocal influence of individuals upon one another’s actions when in one another’s immediate physical presence” (Goffman 1973). It follows from this definition that notions of presence, intersubjectivity (“reciprocal influence”), corporeality (“physical”) and agency (“respective actions”) ought to be revisited in light of new sociodigital practices.
During their onscreen interactions – here, notably, using a telepresence robot – individuals undertake actions whose authors are difficult to identify. Agency (Butler 2002) is distributed between subject and tekhnê and produces a potential indeterminacy in the assignability of the ethical responsibility of action. Between the locutor, the artefact and the interlocutor, the question arises as to the attribution of the interactional gesture: from its intention to its emission and its perception. All the more so since, although the interaction may be symmetrical in nature (Maingueneau 1996), interactivity (Weissberg 2002) appears to be unilateral. Thus, the pilot of the robot can undertake a series of actions and create a meaningful context that the interlocutor cannot technically regulate (interactivity of the software in transmission and non-interactivity of the hardware in receptionOn the Beam, the user can, for example, raise or lower the volume of their microphone on the interface, whereas the interlocutor, facing the robot, cannot control the output volume (unlike on Skype, for example, which allows the volume to be controlled “from both sides”).↩︎). This asymmetry of the interactivity with the artefact introduces an illusionary confusion between subject and object, which are mistaken for one another, assimilated to one and another, or substituted for one another. As a consequence, the intersubjective, technical, and corporeal modalities of presence are affected.
The definition of the body – the interface of interactions – is thus at stake. The body, in fact, acts as a mediator. During online interactions, one’s body is sometimes visible, often partially perceptible and in all cases elusive. It should therefore be noted that:
What happens with the advent of digitalisation is a complex hybridisation between thought and gesture, between the computer-object and the user-subject who deploys a skilful and unsystematic know-how, which falls under vagueness, under approximation, under creative improvisation as much as under routines (Frias 2004, 10).
It is thus essential to explore the new practical technobodily modalities enacted in a hybrid and polyartefacted presence. Our aim is to analyse, from an interactionist and phenomenological perspective, the epistemological and ethical stakes involved in a form of extended presence that entails both the reification of living subjects and the personification of communicational artefacts. This is a process that is involved in the co-construction of an intercorporeality, of an interworld.
The technobodily modalities of (onscreen) interaction
Interaction and subjects of action
During their interaction with others, individuals produce multimodal actions for which they are held responsible. This capacity of a subject to act on their environment, on objects and on others, as well as the subject’s perception of this faculty, is subsumed under the notion of agency (Butler 2002).
Agency (our acts, our thoughts, our desires are ours and we are relatively conscious of causing and controlling them) combined with resonance (the automatic, non-conscious capacity that pushes us to make the emotions of others resonate in ourselves) and empathy (the active process of understanding the cause of the emotional state of others and displaying understanding and acknowledgement of their emotions) constitute the three dimensions of interaction that allow interactants to build social ties and ensure group cohesion, as well as the autonomy of each individual with respect to others (Nadel and Decety 2006). Individuals are indeed connected to one another by resonance and empathy: resonance makes them automatically reflect the attitudes and expressions of others, and empathy allows them to partially feel what others are experiencing (Tisseron et al. 2013), so that they can, for example, provide help. On the other hand, agency ensures that every individual is the master of their actions, thus avoiding all confusion between self and other.
Mirror neurons
At the cognitive level, a set of regions of the brain are activated when a subject performs an action and when they observe this action produced by someone else. The neurons, called “mirror neurons”, induce this motor resonance (Rizzolatti, Sinigaglia, and Raiola 2008 ; Keysers and Gazzola 2010). Experiments have shown that motor resonance is only activated when the observed model is a human being and not a robot, however (Nadel and Decety 2006). The neurons thus distinguish between biological agents and objects: they are only activated when the agent is a living being. Moreover, one region of the brain plays a pivotal role in the experience of agency: it compares signals emitted by the self to those emitted by the environment. This makes it possible for the individual to distinguish between the consequences of an action that they have initiated and those linked to an external event. Individuals are thus led cognitively and intuitively to acknowledge that they are the authors of their own actions and responsible for them: that they are agents (Nadel and Decety 2006).
Yet this distinction is more difficult to make when a form of mediation between the production of an action and its perception by others comes into play. During an onscreen interaction, for instance, the locutor physically produces multimodal activities that a digital artefact partially retransmits to the interlocutor. The elements appearing on the interlocutor’s screen can be defined as “indexes” of action as described by Charles Sanders Peirce (1960). The restitution of the elements of action is jointly accomplished by the digital tool and the user and is subject to media affordances. These affordances “come into play during an instrumented activity and are defined as the set of possibilities and constraints of the environment that give agents different options to act” (Lamy 2010, 3).
Affordances
Technology is to be understood “in terms of its affordances, as they reveal themselves in and through humans’ attempts to interact with the artefact” (Hutchby 2001, 146). The technical object is thus to be regarded as “a partner acting in a relationship that is as much enabling as restricting” to the degree that it acts as “an interface between the project of action that has been entrusted to it and the subject who is active in their usage of such technical interfaces” (Voirol 2013, 149). Olivier Voirol introduces a new dimension into the notion of intersubjectivity by highlighting the way in which the user interacts with an apparatus that acts with them as a partner in the interaction. As such, the user must show inventiveness and creativity in their interaction with the apparatus.
Individuals and their communicational artefacts have recourse to both natural and non-natural language as a tool of mediation and intercession by mobilising all the semiotic resources at their disposal to act on one another (De Fornel 2013). The notion of agency in the field of anthropology also sheds valuable light on the distinction in the attribution of responsibility for an action between subject and object. Anthropology brings about a decentring of the intentional human subject in favour of a multiplicity of agents, whether human or non-human (De Fornel 2013).
Anthropology and agency
Based primarily on studies of the ritual practices of indigenous peoples in the Americas and as a way of rejecting an objectivist point of view, anthropology allows for a dual promotion of agency inasmuch as “living beings are conceived as humans and objects are treated as having the status of living beings or even human beings who participate as such in social reality” (De Fornel 2013, 39). These two orientations take opposing directions. The first considers agency as stable and adopts an ontological perspective. The second, on the contrary, defines agency as unstable and contextual: animals, plants or artefacts are only conceived as persons within specific practices (a ritual, for instance). For these entities to become agents, humans must undertake “work on perception and categorisation… within everyday or ritual activities” (De Fornel 2013, 39). This latter approach to the notion of agency as artefactual and contextualised raises the question of the responsibility of actions carried out in digital interaction: in particular, of the role played by the tekhnê in communicative actions.
The corporeality of (inter)action
Whatever the nature of agency, carrying out an action necessarily involves a prior corporeal activity, whether the latter is synchronous or asynchronous, self-initiated or other-initiated. Social interactions are by nature body-to-body interactions (Cosnier 2004), since in fact “if the locutor thinks and speaks with their body, the interlocutor also perceives and interprets with their body” (Cosnier 2004). Here we propose to rethink corporeality in social interactions from a phenomenological approach insofar as the etymological origin of the term “phenomenon” is found in the Greek verb φαινεσθαι: to appear, to be shown. Appearing prefigures the moment when the subject and the object or the others come into contact: their decisive “taking cognizance” (Heidegger 1985). As Maurice Merleau-Ponty explains in his Phenomenology of Perception (1945), the body cannot be regarded as an object of the world, but rather as the means for our communication with it.
Corporeal schema
One’s body is defined as “the latent horizon of our experience, itself ceaselessly present prior to all determining thought” (Merleau-Ponty 1945, 109). The contour of one’s body constitutes a border that ordinary spatial relations cannot cross (Merleau-Ponty 1945, 114). An individual apprehends their body and the position of each of its limbs by way of a “corporeal schema” (Merleau-Ponty 1945, 114). This corporeal schema is not a matter of a simple summary of corporeal experiences, but rather an “overall awareness of my posture in the intersensory world, a ‘form’” (Merleau-Ponty 1945, 116). Apart from being a form, the corporeal schema is also a dynamic in the sense that the body appears to the subject as a posture toward accomplishing a certain actual or possible task. The spatiality of the body can thus be regarded differently than that of external objects. Objects have a “positional spatiality”, whereas the body has a “situational spatiality” (Merleau-Ponty 1945, 116).
The polarised body
Applying a deictic element like “here” to the position of the body does not designate a determinate position in relation to other positions, like for an external object, but rather the anchoring of the active body to an object: the body’s situation vis-à-vis its tasks. The body appears to be “polarised by its tasks, as it exists toward them, it coils up upon itself in order to reach its goal, and the ‘corporeal schema’ is, in the end, a manner of expressing that my body is in and towards the world” (Merleau-Ponty 1945, 117). The body tends to incorporate the actions in which it is engaged to the extent that these actions participate in the structure of the body. An agreement comes into being between what the subject is aiming at and what is given: between intention and effectuation. The body is thus shown to be the subject’s anchoring to the world. It follows that the body is neither “in space, nor ... in time. It inhabits space and time” (Merleau-Ponty 1945, 162). According to Merleau-Ponty, “Insofar as I have a body ... I am of space and of time, my body fits itself to them and embraces them” (1945, 164).
Despite these considerations, phenomenological studies have focused their attention so far on the relationship between the Self and the Other and between the Self and the object, neglecting the technical mediation of interindividual relationships. However, as Stéphane Vial notes, “in itself, every phenomenon is phenomenotechnological. There is a transcendental technicality of appearing: i.e., an a priori technical dimension in every phenomenal manifestation or ‘phany’” (2013, 152). For an Other or an object to appear to a Self and vice versa, a technical mediation is indeed required, whatever its form. Gaston Bachelard’s phenomenotechnique reminds us that an “ontophany” – the manifestation of a being – requires a technology as much to occur as to be observed.
Perceptive Umwelt
Technology turns out to be itself a bearer of phenomenality as far as it allows phenomenality to be created: namely, “the possibility of appearing or appearing as real” (Vial 2013, 16). The idea behind the concept of phenomenotechnique is that technologies do not only consist of tools: they turn out to be structures of perception. Technological systems are described by Vial as technoperceptive systems inasmuch as they “structure our possible experience of the world on the phenomenological level by creating a perceptual UmweltThe notion of Umwelt refers to the sensory environment that is specific to a species or individual.↩︎ in which all our perceptions are immersed” (Vial 2013, 31). Consequently, the subject’s perceptions are a function of technotranscendental structures, which themselves depend on the technology of the time (printing, telephone, internet, etc.) (Vial 2013, 31).
With the advent of new technologies, some perceive the body as “an unworthy archaeological vestige that is destined to disappear” (Le Breton 2001, 20). The relationship between the body and technologies attracts fascination and concern insofar as it evokes the myth of a spirit that is separated from the body, of an artificial being that the savant would be able to create, of perfect communication without misunderstanding (Flichy 2009, 11). This crisis of sensibility of the body involves a tension between two different ways of relating to the body: namely, the tension between a biomechanical view inherited from modernity and a virtual view of the body that comes from postmodernity (Casilli 2012, 6). However, the fear of the body disappearing, “swallowed up by a computer screen is less a real risk than a paradoxical reaction to its imaginary hypertrophy, its omnipresence” and this is because our society elevates the body to the status of the ultimate referent (Casilli 2009, 3). Inasmuch as digital ontophany has a broad impact on the phenomenological presence of the things themselves, this is not a matter of the body disappearing but of new forms of corporeal appearance (Vial 2013, 239).
Analysis of sequences of personification and reification
Scientific positioning
Our analytical approach to technobodily modalities of enacting onscreen intersubjectivity is based on an interdisciplinary methodology drawing on visual ethnography (Banks and Morphy 1997 ; Ruby 2000 ; Pink 2007 ; Dion 2007, etc.), multimodal interaction analysis (Goffman 1973 ; Cosnier 2004 ; Mondada 2008 ; Kerbrat-Orecchioni 2010 ; Traverso 2012, etc.) and phenomenology (Husserl 1929 ; Merleau-Ponty 1945 ; Le Breton 2001 ; Vial 2013, etc.). The aim is to analyse sequences of social interactions from the participants’ subjective experience: from their perception and their corporeal action both on and off screen. Our focus is on the participants’ use of multimodal resources related to corporeality (verbal resources, gesture, expressions, gaze, postures) and the multisemiotic resources related to the media (pictures, graphics, videos, links, techno-discursivity). Our audiovisual analysis of interactions is based on recordings that constitute both the medium and the object of an intrinsic analysis (embedded transcription and semiotic enrichmentSemiotic enrichment consists of denoting the participants’ technobodily activities by embedding signs in the audio-visual document. It participates in the visual ethnographic approach which treats the image as an intrinsic element of the research process.↩︎). The aim is to study the participants’ technobodily behaviour in polyartefacted interactions both on and off screen.
States of mediation
Here we focus on problematic interactional sequences occurring during the seminar sessions. These sequences were subject to faulty mediation or faulty apprehension of this mediation that lead to a suspension of the ordinary course of action.
We identify several stages in the mediation process in a problematic situation. Firstly, there is demediation when an incident occurs: communication is no longer ensured, and the medium no longer fulfils its role and fails to serve its purposes. Secondly, remediation attempts are made, wherein the incident is dealt with, mediation is repaired, and the medium can be relaunched, replaced, or supplemented. Finally, immediation occurs as soon as the incident has been resolved: the medium once again fulfils its purposes, and communication is transparent and creates the illusion of immediate communication.
Actions are thus undertaken by the interactants to provide technobodily mediation using multimodal and multisemiotic resources. Moreover, these actions are instantiated during the mediation process by the subjects, the latter embodying particular functions at specific moments in the interaction that we identify as follows. Sentinels ensure participants’ extended presence and flag mediation incidents to the procurators. They are on the lookout for signs of demediation. The procurators are at the heart of the mediation process and ensure the presence of the interactants in a technobodily manner. They bring about remediation. Witnesses attend the interactional event and contribute to extended presence without directly intervening. They benefit from immediation.
The identification of these instances and states of mediation enables us to better understand the practical technobodily modalities of onscreen presence in the sequences that follow and, more specifically, the effects of personification and reification that they induce. Phenomena of the personification of artefacts refer to situations in which subjects attribute human properties to the concrete objects of mediation, endowing them with volition, power, the capacity to act, etc. Phenomena of reification of living beings, on the other hand, consist of regarding the other as an object, removing their human qualities and running the risk of negating all forms of empathy. These phenomena can be conscious or unconscious, intentional or unintentional.
Interactional sequences of personification of the artefact
In this section, we analyse three interactional sequences in which a personification of artefacts occurs:
- Sequence 1 – Humorous attribution of a participant’s sneezing to her robot
- Sequence 2 – Conferring agency to the robot during the distribution of speaking turns
- Sequence 3 – Simulation of an act of physical closing (a kiss on the cheek) addressed to artefacted participants
Sequence 1: “The robot sneezes”
In this sequence, an exchange is taking place following Morgane’s data session; it is Jean-François’s turn to speak. While he is speaking, Amélie (present via the Beam robot) sneezes. Although this would probably be an interactional non-event in a face-to-face setting, it gives rise to specific reactions in this hybrid setting.
The attention of most of the participants is directed towards Jean-François. It can be noted in this regard that Dorothée turns the webcam towards the speaker, at his request, thus allowing the ex-situ participants to see him in their onscreen field of vision. This is thus a situation of remediation in which Jean-François embodies the sentinel function and Dorothée that of procurator.
Attention is focused on the speaker and the sudden sneeze of Amélie, who is piloting the robot, is even more of a surprise because it is not preceded by any corporeal index foreshadowing it. The absence of any index is due to the spatial positioning of the robot: its screen is only facing Morgane and is not visible to the rest of the group. Consequently, there is not any element that would allow them to expect Amélie to sneeze.
Moreover, the sound of the sneeze is not that of Amélie, but rather that of her retransmission by the robot (cf. the notions of primary and secondary fronts, (Ibnelkaïd 2016)). Hence, Morgane, who has visual access to Amélie’s face, can discreetly, spontaneously and quickly tell her “bless you”, while Christine begins to echo the same utterance, but does not complete it, and starts to laugh instead. Christine, in the absence of any corporeal index and in the presence of the artefactual sound, attributes the occurrence to the artefact, stressing the comic effect – “the robot sneezes” – while laughing. This laughter spreads to the participants around her, who can only concretely perceive the object and not the human making use of it.
But as the speaker who has the floor is still speaking and the main interlocutor is paying attention to him, the laughter fades and Amélie takes the initiative to turn the screen and the robot’s camera towards Jean-François, so that her attention is also on the ongoing interactional activity.
Here, the personification of the artefact is intentional and meant to be humorous, and it is based on the absence of any visible index of corporeal activity for part of the audience.
Sequence 2: “The robot is turning its back on me”
In this sequence, the guest lecture is ending, giving way to the discussion. Christine wants to explain the preferred method to claim speaking time: namely, to raise one’s hand to “reserve” one’s turn to speak, which Christine will then be in charge of distributing.
First, Christine says, “who wants oh I think we’ll do the same as the other time eh (.) meaning you raise your hand, by raising your hand” and adds “and then for those of you who are remote eh the same thing you wave and I’ll give you a turn to speak distributed in advance”. The expression “raise your hand” vis-à-vis the in-situ participants gives way to the term “wave” for the ex-situ subjects, allowing the spectrum of semantic signals to be enlarged.
However, this proposal for organising speaking turns comes up against its limits in the case of robot mediation in the current spatial configuration. This risk of a faulty mediation is pointed out by Françoise, who then plays the role of preventive sentinel: “except the robot you won’t see it/him” (the original pronoun in French could both refer to an animate or inanimate subject). Christine confirms this faulty technobodily mediation and provides an explanation for it: “ah yes the robot, because it/he’s turning its/his back on me”. The personification of the robot is expressed here as an agency that is attributed to what is discursively represented as its artefactual rather than corporeal schema.
Then Christine proposes an indirect form of remediation by continuing her personification of the artefact in the third person and in attributing an intentional and reflective agency to it: “but uh it/he’s going to turn around because in any case it/he’s going to follow the others at some point”. The course of action to be followed is presented as a form of routine practice on the part of the telepresence robot, which would appear to have developed its own form of mediation vis-à-vis other artefacts and mastery of its artefactual schema, or its spatial posture in accordance with the situation of interaction. The robot is its own independent procurator.
However, the actions of moving the robot are still to be attributed to its pilot who initiates them. The pilot herself, ex-situ, has developed a routine practice of appropriating the interactional space via the extension of corporeality that the in-situ artefact represents. When Amélie hears Christine’s utterance, she makes her robot move so that it is no longer only facing the guest speakers, but sharing a reciprocal view with Christine and the other members of the group in order to facilitate the discussion. This movement, which is initiated by Amélie and acted upon by the robot, is verbally punctuated by Christine’s “that’s it”, which validates her professed confidence in how the mediation is self-managed by the pilot and her robot who are perceived as being in symbiosis. Christine’s validation makes Amélie smile.
Sequence 3: “So, I’ll give you a kiss, eh” (originally “faire la bise” in French which is a common interactional opening or closing gesture)
This interactional sequence constitutes the interactional closing of a seminar focusing on a work session with a small group. During this closing sequence, the postural-facial-gestural elements of informal face-to-face interactions are used in this hybrid setting.
Christine, an in-situ participant, gets up from her seat and announces the projected action out loud: “So, I’ll give you a kiss, eh”.
A typically French closing follows: Christine gives a “bise”, or kiss on the cheek, to the Beam robot, the computer and the Kubi, one after another. The personification of the artefacts is again intentional here, simultaneously inducing a comic effect (cf. Sequence 1), a personalisation and an individualisation of the closing sequence. In digital interactions, the closing sequence can easily become a face threatening act to the extent that it is not gradual, but rather abrupt (a break in the direct face-to-face interaction) (Ibnelkaïd 2016), all the more so in a polylogal, polyartefacted configuration that entails a dual communitarisation (“on-site”/“remote”) (or membership categorisation) (Sacks 1992). The personification of artefacts in a closing sequence serves as a mitigator or even a face-flattering act (Kerbrat-Orecchioni 2005).
Interactional sequences of reification of the animate
In this section, we look at three interactional sequences in which living beings are reified:
- Sequence 4 – Polymorphic reification of the perceptual organs of an artefacted participant
- Sequence 5 – Assimilation of a subject to its artefact in a remediation proposal
- Sequence 6 – “Mise en abyme” of responsibility for the mobility of an artefacted participant when being asked to move the robot
Séquence 4 : “She’s there, her eye”
In this sequence, the guest lecturer is participating in the seminar via the Beam telepresence robot. This excerpt shows an exchange between the lecturer (Susan Herring) and Christelle. Susan pauses in the midst of answering a question from Christelle to check on the state of the mediation. Christelle is also participating ex-situ via the video conferencing platform Adobe Connect.
Assuming the role of sentinel, Susan asks the group indirectly, “I don’t know if Christelle can hear me”, to which Christelle responds by specifying the nature of the faulty mediation, which is not a matter of hearing, but rather seeing: “Yes, but I can’t SEE you”. This is due to the fact that Susan has positioned her telepresence robot to face the video-projection of the Adobe Connect image of Christelle on the wall and not the webcam on one of the seminar tables.
It appears that, for Christelle, two different senses of her body schema (seeing and hearing) are dispersed in the process of reception and that, for Susan, a distinction is made between the artefact presenting the image, the projection of the self, and the artefact receiving communication, captured by the webcam. The two modes are usually united: both in a physical face-to-face setting and remotely with computers, tablets and mobile phones, in which case the webcam is placed as close to the image as possible in order to avoid this kind of confusing hiatus.
Christelle’s response elicits attempts at clarification by the participants who are spatially closest to the webcam. After Samira, Morgane and Christine have verbally relayed Christelle’s words, Morgane and Christine explain “she’s there” while pointing at and touching the webcam. And Morgane adds “this is her” and Christine “her eye”. This reification of the animate is thus polymorphous. Christelle is perceived in turn as being in the material object, being inside of it – “she’s there” – then as being the object itself – “this is her” – and as being a part of the object, an organ dispersed in the material artefactual space: “her eye”. Susan, who is used to mobile artefacted interactions, then understands: “alright alright I was talking to the screen”. The polymorphism of the reification via successive reformulations is based on the desire to be as explicit as possible about the dispersion of the “organs” of communication and to facilitate their hybridisation and integration in the ongoing interaction.
Séquence 5 : “Do you want me to turn you?”
During the group work session, attention is supposed to be directed at the activity taking place on the whiteboard on which Caroline is illustrating what she is discussing. There is thus a need to make sure that everyone can see the board and what is written on it.
Christine embodies the role of sentinel in making sure that the mediation is valid for Christelle: “Do you see something there, Christelle?” To which Christelle responds by explaining that there is indeed mediation but that it is inadequate: “uh yeah but not well” and immediately starts to produce an utterance about remediation. She is not looking for an in-situ procurator insofar as she is already undertaking an act of remediation. Her aim is to replace the artefact used to view the whiteboard (the mobile webcam of the computer connected to Adobe Connect) by the artefact that she was using so far to see her colleagues (the webcam of the Kubi tablet connected to Skype). Christine validates the remediation proposal – “yeah, turn around” – by using the reifying turn of phrase assimilating an object to a subject.
However, the participant who is spatially closest to the Kubi object is strongly tempted to act as procurator, since it only requires her to make a simple mechanical gesture, which is more spontaneous, quicker and more efficient than trying to resolve the digital mobility bug. Being aware, however, of the reification induced by the procedure of manually moving the tablet and the face threatening act that it represents for the Kubi’s pilot, Caroline refrains from acting without permission and asks Christelle: “Do you want me to turn you or not (laughter)? All the more so since the pilot can organise the position of the in-situ participants on the interface to facilitate the Kubi’s movements by simple clicking on the avatar of the person she wants to address. The Kubi then turns in that person’s direction. Manually moving the Kubi thus disrupts the pilot’s organisation.” Caroline does not seem to hear Christelle’s response, which does not validate her offer to act as procurator – “wait no” – and moves the Kubi, adding “I don’t want to be impolite”. Jean-François comments ironically on this affirmation – “too late” – and then laughs.
Caroline’s posture is in fact ambivalent in this situation of demediation: on the one hand, she proceeds physically to undertake remediation; on the other, she is careful to mention that she does not want to do something that would make Christelle lose face. The gesture of moving the Kubi does in fact constitute a form of taking over the subject’s mobility, an infringement on Christelle’s freedom of movement and an appropriation of her corporeality, given that a process of reification has already been set in motion at the start of the sequence. The subject and the object are one; to move the one is to move the other, to interfere in their body schema.
Séquence 6 : “Amélie, can you move, please?”
Amélie is attending the same work session via the Beam robot. The Beam is blocking the view of the camera that is supposed to be recording the interactional data for the research project.
Morgane intervenes to ask Amélie to move her robot by employing a reifying turn of phrase that links the subject and object and by identifying the element blocking the camera’s view as being the subject, Amélie, and not the artefactual object: “Amélie, can you move, please, you’re in the (.) camera’s field of vision?” Since Morgane is not facing the robot’s screen, however, Amélie does not realise that she is being addressed. Christine, who is in Amélie’s field of vision, then takes over the enunciation by attempting to repeat Morgane’s request – “Amélie can you uh (waving for her to come closer)” – and then wonders which space she can occupy, “where can she go”. Having defined the most appropriate place, Christine verbally and gesturally guides Amélie in moving the robot: “that’s it that’s good (.) back up (.) no (.) that’s it keep backing up (.) that’s it now turn (.) that’s it”. It appears that Christine is directing Amélie who is directing the robot. A sort of «mise en abyme» of responsibility for the movement takes place here. A chain of agency is put into action that runs from the in-situ subject, Christine, via the ex-situ pilot subject, Amélie, to the Beam robot.
A dual movement at work in artefactual intercorporeality
A dual process of personification and reification is thus at work in polyartefacted technobodily mediation. On the one hand, a vocal-gestural attribution of artefactual characteristics to the living subject by others – reification – and on the other, the vocal-gestural attribution of human characteristics to the subject’s artefact – personification. The first movement, reification, is introduced into the interaction when the other prompts the artefact to take action, thereby identifying the artefact, whether intentionally or unintentionally, with the subject piloting it. In the second movement, personification, when the subject acts upon the artefact, the latter retroacts, and this retroaction perceived by the other is assimilated to human activity. The artefact perceived to be the subject of the action is attributed an intentionality, an agency of its own. We can present this process schematically as follows.
Moreover, the dual process of reification and personification seems to be expressed in a differentiated apprehension of matter. In the case of the reification of the animate, there is an effect of “materialisation”: a process that involves “using a material in order to give form to an abstraction” (Chatonsky 2015). This involves “instrumentalising matter under the authority of a conceptual enterprise” and its visual presentation aims to “translate digital data into some sort of form (image, sound, etc.)” (Chatonsky 2015), in response to a desire to make an abstraction sensible.
Materialisation
This phenomenon is found in Sequence 4, “She’s there, her eye”, in which the participant’s senses are verbally materialised in the artefacts when Morgane and Christine explain, “she’s there”, as they point at and touch the webcam, with Morgane adding, “this is her”, and Christine, “her eye”. The webcam is identified as the concrete materialisation of the sense of sight and replaces what is organically attributed to the eyes in a face-to-face physical setting.
In the case of the “materiality” effect, here in the situation of the personification of the artefact, we need to conceive of matter as given and observe the networks woven by it (Chatonsky 2015).
Materiality
This is the case in Sequence 2, “The robot is turning (its) back on me”, in which the robot is given an anthropomorphic materiality by one of the participants, who comments on its material presence and its positioning amid the other subjects and artefacts. When the participant adds “but uh it/he’s going to turn around because in any case it/he’s going to follow the others at some point”, she is referring to the robot’s independence amid the other artefacts and subjects and its agentive and material capacity to navigate this network.
Dual process
We can schematically present these effects as follows:
In this dual process, the artefact acquires the form of a persona in the anthropological sense of the term: namely, the potential of everything, whether an object or a human being, to affirm a singular presence. This persona is enacted in a borderline presence: a sort of ambiguous presence that goes through varying degrees of intensity, from the most material to the most invisible.
Enacting an interworld
Interactants’ hybrid presence crosses physical spaces and communication media via multimodal and multisemiotic actions undertaken by subjects, their corporeality and their artefacts. They draw on technobodily resources at their disposal, in order to preserve communication despite the multiple space-time frameworks. These technobodily resources are employed to flag mediation incidents (demediation), as well as to try to resolve them (remediation) and to achieve a state of media transparency (immediation). To this end, the subjects coordinate and cooperate among themselves and with the artefacts, both explicitly and implicitly, by embodying instances such as sentinels who flag mediation incidents, procurators who repair problems in the system of mediation, and witnesses who participate in the interactional event without intervening in the mediation process. Gestures are thus shared. There is an extension of the corporeal schema through the artefacts and the other subjects involved in the interaction and in the extended presence of each of the participants. Intersubjectivity is introduced into the corporeal schema that is extended to the other, and a form of transsubjectivity is enacted insofar as it goes beyond and cuts across technology and distance. The bodies become one, in order to interact in a complex technobodily network.
The transsubjective gesture and its action form part of a chain of distributed agency. Each of the interactants has the opportunity to contribute to the preservation of communication by way of any significant gesture, no matter how small. The communicative affordances and the positioning of the bodies and the artefacts in space entail that the subjects have to cooperate, since they cannot individually undertake the complex and reticular physical-digital mediation. The extended presence exists in the form of a flow running across subjects and artefacts. The gesture is constituted within a chain: It is initiated by some, and continued and completed by others, whether subjects or tekhnê. Distributed agency ensures an enlarged field of action.
Percepaction
There is thus an extension in terms of both action and perception; action is dependent on perception and attention. It is essential to perceive and to be perceived in order to act, just as it is necessary to act in order to perceive and to be perceived in onscreen interactions. This coordination of intrinsically linked perception and action constitutes a single operation of “percepaction” (Roquet 2002): the idea of exiting oneself in order to perceive and be perceived. In his analysis of movement, Hubert Godard (1994) also aims to “consider perception as a gesture”, in the sense of a movement, a meaningful action (1994, 68).
Thus, every operation of percepaction is the result of a subject-artefact-other hybridisation, which gives rise to both the personification of the artefact and the reification of the animate. Indeed, corporeality is artefacted and artefacts are embodied; there is intercorporeality in this percepactive extension.
Intercorporeality
Merleau-Ponty outlined the phenomenon of intercorporeality in his last working notes.
There is here no problem of the alter ego because it is not I who sees, not he who sees, because an anonymous visibility inhabits both of us, a vision in general, in virtue of that primordial property that belongs to the flesh, being here and now, of radiating everywhere and forever, being an individual, of being also a dimension and a universal. What is open to us, therefore, with the reversibility of the visible and the tangible, is – if not yet the incorporeal – at least an intercorporeal being, a presumptive domain of the visible and the tangible, which extends further than the things I touch and see at present (Merleau-Ponty 1964, 185).
My two hands “coexist” or are “compresent” because they are one single body’s hands. The other person appears through an extension of that compresence; he and I are like organs of one single intercorporeality (Merleau-Ponty 1960, 274).
If the body of the subject and that of the other form a single intercorporeality, objects are not left out and participate in this network of intersubjective perception to the extent that “our organs of perception are found both inside and outside, they are also to be included among the things through which we gain access to a specific dimension of a being’s radiation” (Penayo 2016, 85).
Screen presence is thus the result of the multisemiotic, multimodal and sensory expression of artefacted subjects and involves the flesh and its extensions. The subjects are sentient beings who co-construct themselves in interaction by equipping themselves with technologies that allow them to overcome physical distance and to manifest themselves in multiple and reticular spatio-corporeal configurations. The subjects thus make themselves present to each other by way of an artefacted intercorporeality that enacts a world they share in common: an “interworld” (Merleau-Ponty 1964, 317).