robot’s situation awareness and are direct clues through which communication partners can infer which aspects of the interaction robots take into consideration.
1.3 Aim and Structure of the Thesis 15
1.3.2 Operationalization of Common Ground
The kind of common ground under investigation is generally personal common ground.
While communal common ground is equally important to social interaction, and to the understanding of context, as personal common ground, robots are still so novel to the most of us that there is no communal common ground to establish or maintain. In other words, communal common ground builds on the assumption that there is a community, but as of yet no ‘community’ of robots exist. Thus I confine the analysis to aspects that are directly relevant for interaction and may be taken for granted in interaction between people.
Within personal common ground, signals that communicate the perceptual and the actional basis, and personal diaries are investigated. The remaining two elements, ‘acquaintedness’
and ‘personal lexicons’ are not investigated as they would require an experimental design that spans several interactions over a relatively long period of time. Although such an investigation is relevant and will at some point become necessary, it is beyond the scope of this thesis.
Context Effect
Common Ground
Displays of awareness
Figure 1.2: Conceptual Model
Context, common ground, and displays of awareness are combined in one conceptual model (visualized inFigure 1.2) that drives and informs the work presented throughout the thesis.
Specifically, displays of awareness of contextual features communicate to partners what the (perceived) common ground is. In turn, the shared common ground between communication partners affects their perception of each other and how they interact. Common ground cannot be measured or analyzed directly, but what can be analyzed are the displays of awareness of context and how these displays are responded to (i.e. the effects). In the following, I describe how the three aspects of common ground are investigated through displays of awareness to contextual features.
Perceptual Basis
The perceptual basis comprises those elements in interaction that people can perceive and share (Clark, 1996, p. 112). This includes information such as the layout of the physical interaction space, the people in it, and events carried out in that space. The perceptual basis is thus a shared context to which all interaction partners can attend to. This is a huge
problem for interactions with robots. People generally do not know how, what, or even if robots perceive anything, because robots do not possess a human nature (Clark, 1996, p. 106) that can guide what assumptions people can make of them. In ethnomethodological terms this means that robots are unlikely to be ‘members’ humans have encountered and dealt with before, and as a direct result thereof people cannot make use of their ‘members’
knowledge’ to understand robots. The perceptual basis is usually not dealt with directly, but it is indicated in actions, for example through talk, gaze or gesture. Therefore, it is not these actions that themselves are part of the context, but they are indicators of a shared perceptual context. The perceptual basis is investigated in Chapter 4 and in Chapter 5, in which a robot produces verbal references to objects in the interaction space and comments on the state these objects are in. Through these actions, the robot signals to communication partners that these objects, and their states, can be considered common ground.
Actional Basis
The actional basis comprises joint actions people engage in (Clark, 1996, p. 114). In contrast to the perceptual basis, actional basis can be accessed directly, for example through talk.
Talk can display to communication partners what aspects of the common ground they take for granted. The actional basis is usually expressed through speech, but can just as well be expressed through gestures or gaze behaviors. The actional basis is investigated through contingency, incrementality, and proactivity. Contingency is understood as a linear temporal relationship between cause and effect; i.e. using the understanding of contingency found in the HRI literature (e.g. Lohan et al. (2011)). Contingency works on multiple modalities that include, but are not limited to speech, gesture and gaze. The effects of contingency as an indicator for situation awareness are investigated using these three modalities. Chapter 3investigates effects of contingent gazes and nods, Chapter 5 investigates the effects of contingent verbal responses and contingent face-tracking, and Chapter 8investigates the effects of contingent responses to repair initiations.
Incrementality is generally seen as a model for speech processing (see e.g. Schlangen and Skantze (2011)), and this is also how I consider incrementality. Incremental speech is therefore speech that is produced in small ‘chunks’ or increments, where each new contribution is updated with information about the situation the speaker is in. Therefore, incrementality is a display of what contextual information a speaker considers to be relevant and thus signals to the communication partner what the speaker considers to be the common ground. Chapter 5 investigates verbal incremental feedback in a physical exercise, andChapter 6 addresses the effects of incremental feedback in an object search task. In both of these studies, the verbal incremental feedback is designed to signal the robot’s awareness of participants’ nonverbal conduct.
Proactivity is an indicator for situation awareness in the sense that it signals to communi-cation partners what is about to happen. Proactivity, as conceptualized and implemented for this thesis, does not include any measures of anticipation or intention recognition of the partner in interaction. Proactivity displays actors’ awareness of their own actions as relevant context in communicating what they consider to be common ground. Thus,
1.3 Aim and Structure of the Thesis 17 proactivity is an indirect signal that interaction partners orient to a joint plan. Proactivity is investigated inChapter 7, in which a robot in a handover scenario proactively signals what it is going to do next using gaze cues.
Personal Diaries
Personal diaries refer to what has gone on previously in the current and in previous interactions already and which is part of the common ground that communication partners establish and maintain throughout the interaction (Clark, 1996, p. 114). Displays of memory show that actors not merely react to stimuli as they are perceived, but that actors can store the information and contextualize it, which is problematic for robots.
For example, Christian (2011) posits that one way to distinguish between humans and machines is that humans can access and make use of past experiences and recontextualize them to new situations, while computer (and thus robots) cannot. Displays of memory therefore work as indicators to communication partners that previous interactions, or previous actions within the current interaction, can be taken as common ground. This use of memory is often taken for granted in interactions between people, but cannot be taken for granted in interactions between people and robots (seeChristian (2011)). This aspect of common ground in HRI is investigated inChapter 4 in which a robot displays an orientation to previous utterances and uses this information in its own productions.
1.3.3 Thesis Structure
The remainder of the thesis is structured as follows: In the following chapter, Methods &
Data, I account for the methods I use and what possible consequences the choices of these methods may have for my results. The chapter also includes an overview of the empirical studies carried out. Following that chapter, I explore my research questions in a series of empirical investigations in Chapters 3 to8. These empirical chapters are followed by a discussion of the results obtained and the insights they provide in relation to my research questions, to previous work, and with respect to what design implications can be drawn from the results. This discussion is followed by a conclusion, in which I attempt to combine all elements of the thesis and set a course for future work.
2. Methods & Data
In this chapter I present and review the methods used in data collection and analysis for the thesis. At the end of the chapter I also provide an overview of the empirical work that lay the foundation for the following six chapters.