Methods in Translation Process Research

In the following, I describe the methods typically applied in TPR; I shall draw on this

description in the literature review in Section 3.4 of previous studies that have relevance for the current thesis.

Reflecting the primary focus on internal processes as mentioned above, studies in TPR have traditionally used methods such as verbal reports (think-aloud protocols and retrospective interviews), keystroke logging, screen recording and eye-tracking (Saldanha & O’Brien 2013).

Saldanha and O’Brien refer to such studies as examples of “process-oriented research”. In the beginning, these methods were primarily used in laboratory settings, but most of them

are now also used in workplace research. With increasing recognition of the context-dependence of the translation process, qualitative methods such as semi-structured interviews and participant observation are increasingly employed to study translation in workplace settings. Saldanha and O’Brien refer to this type of research as “context-oriented”. In TPR, there is general consensus that triangulation is necessary and that the combination of qualitative and quantitative data is fruitful (Alves 2003; Krings 2005, p.352;

Göpferich 2008, pp.66–67; Dam-Jensen & Heine 2009, p.11; Saldanha & O’Brien 2013, p.109; Ehrensberger-Dow 2014, p.366; Jakobsen 2014, p.65). Thus, typically, a number of methods are combined in one study. However, as pointed out by Dam-Jensen and Heine, triangulation cannot remove the disadvantages of the different methods, but it can contribute to obtaining “a rich and more detailed picture” (2009, p.12). In the following, I will first comment briefly on qualitative methods increasingly being employed to take into account the context-dependence of the translation process, and next, I shall comment on more traditional TPR methods which are both qualitative and quantitative.

3.3.1 Context-oriented Translation Process Research methods

A number of scholars have called for more research into translation in its authentic setting to take into account the fact that the translation process is context-dependent. However, this is a quite recent development and no single way of researching this has been

established. As stated by Risku, “[s]cholars of cognitive translation studies have only just started to develop and pilot new research designs that have the potential to cope with the overwhelming changes in research methodology needed to study situated, embodied and extended cognition” (2014b, p.336). Thus, we are still at an early stage of exploring

translation as a situated activity. However, several scholars have suggested the application of ethnographic methods, including interviews, participant observation and document collection (Risku 2010, p.107; Christensen 2011, p.156; Hubscher-Davidson 2011; Olohan 2011, p.353; Risku 2014b, p.336), using qualitative, semi-structured interviews, for example, to inquire into translators’ perspectives on translation processes and their use of technology. Participant observation has also been applied to observe translators at work as well as other actors involved in solving translation tasks, for example project managers, during which the researcher takes notes. Also, collecting documents from a workplace setting may contribute to gaining insight into the translation context. Qualitative methods such as these have sometimes been used alone and sometimes in combination with the more traditional TPR methods described in the next section.

3.3.2 Traditional Translation Process Research methods

Methods which have commonly been used to study internal translation processes are think-aloud protocols, keystroke logging, screen recording, eye-tracking, observation protocols and retrospective interviews. In Krings’ (2005) terms, think aloud protocols, keystroke logging, screen recording, eye-tracking and observation protocols are online methods since they elicit data during the translation process, whereas a retrospective interview is an offline method, since such data are elicited after the translation process has ended. Further, Krings refers to think aloud protocols and retrospective interviews as generating verbal-report data

and to keystroke logging, screen recording, eye-tracking and observation protocols as methods for observation of translator behaviour.

Think aloud protocols provide verbal-report, online data. When thinking aloud, translators are asked to perform a translation task and at the same time, “verbalize whatever crosses their mind during the task performance” (Jääskeläinen 2010, p.371). The transcripts of these verbalizations are referred to as think aloud protocols. Think aloud was applied by pioneers within TPR (Krings 1986; Gerloff 1988; Lörscher 1991) and has been an important research method in TPR (Saldanha & O’Brien 2013, p.124). According to Ericsson and Simon (1980;

1984), the primary advantage of think aloud is that translators verbalize their cognitive processes while the information is still in their short-term memory and verbalization is thus assumed to reflect underlying cognitive processes. In this context, Göpferich states that “es bis heute keine Methode gibt, mit der man mehr Aufschluss über komplexe mentale Problemlösungsprozesse bekommen kann” (2008, p.22) (“until today there is no other method by means of which you can get more information about complex mental problem-solving processes”). However, think aloud has been criticized for a number of reasons. First, a slow-down effect has been confirmed by scholars such as Krings (2001) and Jakobsen (2003). Second, there is the risk that the task of verbalization changes the very processes the researcher seeks to investigate (Saldanha & O’Brien 2013, p.123), a phenomenon referred to as “reactivity” (Jakobsen 2014, p.70). For example, Krings (2001) showed that the translators who were asked to think aloud made twice as many revisions in the target text and

processed texts in smaller units than translators that did not think aloud. Also, think aloud is problematic with regard to ecological validity, particularly in a workplace context, where it would (at least for translators working in open office environments) be unnatural for translators to verbalize their thoughts while translating as well as disturbing to their surroundings.

Keystroke logging is a form of observation of the translator’s behaviour during translation by means of a computer protocol (Krings 2005). Keystroke logging software records all the translator’s keystrokes and mouse actions and the pauses between these, producing a log file which can be presented in different ways after the translation process (for example, as a so-called linear representation). The log file can also be used to replay the translation process at different speeds. A number of keystroke logging programs have been developed (for an overview of keystroke logging software, see, for example, Spelman Miller & Sullivan 2006, p.6 ff.). However, Translog, developed by Jakobsen and Schou at Copenhagen Business School, has, according to Saldanha and O’Brien, “provided an important impetus for the method of keystroke logging” (2013, p.132) in TPR, and Inputlog (Leijten & Van Waes 2006;

Leijten & Van Waes 2013), originally developed for writing research, has since been used in TPR. Both programs are unobtrusive in that they do not interfere with the translation process, but they are quite different in the sense that when using Translog, translators need to work in a special editor, whereas Inputlog can log activities in other environments as well, although with some restrictions (this will be further addressed in Section 4.3.1.1.3.1.4.2.1).

In relation to the use of keystroke logging for investigating translation processes when CAT tools are applied, several researchers have experienced problems due to lack of

compatibility of the logging software with commercially available CAT tools (cf. e.g. Mesa-Lao 2011; Ehrensberger-Dow 2014 and Section 4.3.1.1.3.1.4.2.1 in this thesis).

Screen recording is another type of computer protocol used to observe the translator’s behaviour during translation. Screen recording software records everything or chosen parts of the translator’s screen during the translation process and produces a movie file that can be replayed afterwards at a preferred speed. In addition, depending on the specific choice of software, the translator’s verbal utterances and even facial expressions can be recorded. A particular advantage of applying screen recording is its ecological validity: the software is not visible to the translators and they can go about the translation process as they normally would. However, as mentioned by Göpferich (2008, p.54) and Christensen (2011, p.143), an important shortcoming of this method is that it cannot register which parts of the screen the translators are focusing on.

This problem may be solved by applying eye-tracking, another online method for observing translator behaviour. An eye-tracker records what the translator looks at and for how long, as well as the translator’s eye movements from one part of the screen to another (Göpferich 2008, p.56; Saldanha & O’Brien 2013, p.136). For example, eye-tracking has revealed

interesting results with regard to cognitive effort while translating (cf. e.g. O’Brien 2007).

The use of eye-tracking is based on the so-called eye-mind hypothesis formulated by Just and Carpenter (1980) which posits that there is no appreciable lag between what a person fixates on and what the brain is processing. The most commonly used eye-tracker in TPR resembles a standard computer monitor (Saldanha & O’Brien 2013, p.136). However, eye-tracking can be problematic in a workplace context for several reasons; for example, according to Saldanha & O’Brien (2013, p.138), it can be cumbersome moving eye-trackers around, and the computer connected to the eye-tracker also needs to have tracking software installed as well as a suitable operating system and graphics card. Therefore, Saldanha & O’Brien (2013, p.138) state that the easiest solution is to ask translators to use a specific computer which has the desired properties and to which the eye-tracker is

connected. This, however, reduces ecological validity as translators will not be familiar with the computer, keyboard etc. Furthermore, variation in light intensity is undesirable, sound has an impact on pupil dilation, caffeine and heavy eye make-up can affect pupil size, and translators should be seated at exactly the same distance from the monitor and should not alter their head position significantly during translation (Saldanha & O’Brien 2013, pp.138–

139). Also, only rather short texts can be used when eye-tracking is applied, among other things because of the large amount of data the eye-tracker generates which may slow down the computer and because scrolling is undesirable. Also, for reasons of accuracy, the font size has to be made considerably large (typically a 14-16 point font size) and double line spacing should be used, and typically, these aspects do not correspond to the software usually used by translators (e.g. commercial CAT tools).

Observation protocols are, according to Krings (2005), the simplest form of behaviour observation. The researcher observes the translator during translation and notes observable and audible actions (e.g. the use of hard copy dictionaries or the translator talking to

him/herself or to others). This method is often used in conjunction with application of other online methods, such as screen recording.

Finally, a retrospective interview is a verbal-report method used after a translation process.

In retrospective interviews, translators are asked to give a retrospective verbal report of their cognitive processes during a specific translation task. This interview can be immediate or delayed, depending on whether it is conducted immediately after the translation process or after a longer time-interval. According to Ericsson and Simon (1980; 1984), retrospective interviews have the disadvantage that only information recently attended to can be

verbalized (since it is still in the short-term memory). Thus, it might not be possible for a translator to verbalize his or her cognitive processes because the information needed has to be retrieved from long-term memory. This process entails a risk that the translator might have forgotten some information and/or that the translator infers missing information or generalizes incomplete memories (Ericsson & Simon 1980, p.243; Ericsson & Simon 1984, p.19; Göpferich 2008, p.34). In order to address this potential problem, a retrospective interview is often cued in the sense that the translator is shown a video of his or her process to trigger the memory (Englund Dimitrova 2010, p.407; Hansen 2013b, p.91; Englund Dimitrova & Tiselius 2014, p.180). Such a cue “can provide very powerful reminders to participants about the nature of their engagement in a task” (Saldanha & O’Brien 2013, p.125).⁶ Another risk that has been articulated is that participants might not be able to verbalise their processes if the task has become routine, because overlearned processes operate automatically without leaving a trace in short-term memory, a phenomenon referred to as automaticity (Ericsson & Simon 1980, p.236; Ericsson & Simon 1984, p.15;

Göpferich 2008, p.19; Christensen 2011, p.144; Saldanha & O’Brien 2013, p.123)(Saldanha &

O’Brien 2013, p.123; Ericsson & Simon 1980, p.236; Ericsson & Simon 1984, p.15; Göpferich 2008, p.19; Christensen 2011, p.144). If this is the case, participants may only verbalize the final result of their cognitive processes. In a translation context, this could, for example, be what translation of a specific term they decided on and not their thoughts about potential alternatives. However, an advantage of retrospective interviews is that they do not interfere with the process we seek to investigate because retrospective interviews are carried out after the translation process itself. Furthermore, Ericsson and Simon point out that instructing the participant – in TPR, the translator – to report everything he or she can remember from performing the task, generally reduces the risk that the translator

constructs or infers processes that were not closely related to the actual cognitive processes heeded during the translation process. Along the same lines, Ericsson and Simon

recommend instructing subjects “to only report details that they remember heeding at the time of the original episode” because this may eliminate many people’s “tendency to fill in information that they can’t remember, but “must” have thought” (1984, pp.19–20). In TPR, several researchers point out that retrospective data should not be used as the only source of data on a translator’s cognitive processes; rather they should serve as a qualitative supplement to data collected using other methods (O’Brien 2007, p.200; Christensen 2011, p.144; Englund Dimitrova & Tiselius 2014, p.196; Muñoz Martín 2014, p.63).

6 However, Englund Dimitrova and Tiselius (2014, p.180) point out that there is a risk that the cue used to prompt the translator’s memory might actually ”install” false memories.

Each of the methods described above has its advantages and disadvantages, so the combination of methods must depend on the research question and the research setting (Saldanha & O’Brien 2013, p.22). In any case, as mentioned above, since the cognitive processes in Muñoz Martín’s (2010b) first level are non-observable, the cognitive processes underlying the observable actions can only be inferred by the researcher (Christensen 2011, p.142; Jakobsen 2014, p.65). Also, a number of scholars have argued that through a

combination of traditional TPR methods and the context-oriented qualitative methods described in Section 3.3.1, a more comprehensive picture of the translation process can emerge (Hubscher-Davidson 2011; Olohan 2011; Risku, Windhager, et al. 2013;

Ehrensberger-Dow & Massey 2015). Olohan addresses this specifically with regard to translators’ interaction with technology and states that:

“[i]n order to investigate this interplay as it emerges we must draw on ethnographic methods of inquiry. Direct observation and field notes could be combined with data collected from keystroke logging and/or eye tracking to give a fuller picture of the interaction” (Olohan 2011, p.353).

As noted by Risku et al. (2013, p.172), the miniaturization of observation technology (such as screen recording, keystroke logging and eye-tracking) increases the possibility of carrying out such studies in situ while also improving ecological validity. Furthermore, methods used for studying the translation process may be combined with analysis of the translation product.