F RAMEWORK FOR E VALUATION OF O RGANIZATIONAL I NTERVENTIONS

A framework for evaluation of organizational interventions was needed to move toward the objective of PHASE II, that is to explore how the theory of relational coordination can be used in organizational intervention processes as a tool for improving the interdisciplinary collaboration in surgical units, and the objective of PHASE III, namely to assess whether relational coordination and safety culture in a surgical unit are improved during an organizational intervention process.

A research- and theory-based framework presenting an evaluation model that includes both process evaluation and outcome evaluation was chosen as the primary inspiration of evaluation in this study. Three dimensions highlighted the importance of applying a differentiated evaluation model as a reference framework in order to achieve a detailed understanding of the changes seen in this project. First, the framework captured different elements that reflected a change process, even in an ever-changing context such as the one the surgical unit was embedded in. Second, it incorporated the participants' involvement and engagement in the intervention process during PHASE II, as well as the changes in participants’ attitudes as measured during PHASE III.

Finally, it facilitated an awareness of the contextual conditions surrounding organizational intervention processes.

The framework is based on a process-oriented organization theory describing how best to evaluate organizational interventions (Tsoukas and Chia, 2002; Nielsen and Abildgaard, 2013). What made the chosen framework particularly interesting was the fact that it is based on a system-oriented and dynamic understanding of the processes of change within organizations, rather than on a linear and causal explanatory approach.

Instead of looking at the evaluation of organizational change as changes causing movements from one stage to another, the model proposes the need to also focus on the contextual conditions and evaluate both processes and outcomes, as shown in Figure 7, p. 44)

According to the framework, the evaluation of an organizational intervention process is divided into phases (Nielsen and Abildgaard, 2013). Initiation is the phase at the beginning of an organizational intervention process, in which the intervention and communication plan is developed, and the participants who are going to be involved in the change process are designated. Screening is the phase in which the problem areas are identified and the baseline measurements for evaluating the intervention’s effect are conducted. In the following action-planning phase, the intervention activities are developed, and a description of both the content and the development process is documented.

Finally, there is the implementation phase, in which the intervention initiatives are implemented, and the implementation activities are documented. By evaluating the initiation, screening, action-planning, and implementation phases during an organizational intervention process, valuable knowledge regarding any changes in the organization during the intervention will be gained, resulting in a more differentiated evaluation.

collaboration

These elements are during the intervention process influenced by, what the framework names, the organizational actors’ mental models through the influence the mental models have on participants´ behavior. Nielsen and Abildgaard (2013) use definitions adopted from the science of psychology when they emphasize that actors’ use mental models to make sense of the world. Mental models are individually developed, and they guide the actors’ understanding of the surroundings and how best to react to changes that occur (Johnson-Laird, 1983). Transferring this definition of mental models

Initiation Screening Action-planning

Evaluation

Change in attitudes

Changes in processes

Structural changes

Quality and performance

Implementation

Change mechanisms related to:

Organizational actors´ mental models and behavior

Change mechanism related to the external context:

Structural, political, or economic changes in the organization Interlinked elements relevant to the evaluation The intervention design and processes

Figure 7 Framework for Evaluation of Organizational Interventions, inspired by Nielsen and Abildgaard (2013).

to organizational intervention processes, mental models govern how organizational actors react to the change initiatives, and they may explain the actors’ behaviors.

Insight into the actors’ mental models may be gained by exploring the actors’ attitudes toward issues related to the organizational culture and the everyday performance of tasks (Nielsen and Abildgaard, 2013).

The evaluation model emphasizes that the effect of organizational interventions must be evaluated based on different parameters, including changes in actors’ attitudes, knowledge, and values, changes in organizational procedures and structures, changes in working conditions, and changes in quality and performance outcomes (Nielsen and Abildgaard, 2013). Furthermore, changes that occur concurrently must be anticipated (Nielsen and Abildgaard, 2013). Changes derived from economic challenges and realized in staff and/or production reductions are often unforeseen and, when carried out with immediate effect, are likely to affect the participants’ attitudes toward engaging in organizational changes. Interventions at the meso and micro levels within an organization are influenced by organizational and structural changes at the macro level due to policy decisions and strategic modifications issued by the senior management.

As described in the introduction, surgical units are situated in a context characterized by uncertainty, time pressure, and complexity at the micro and meso levels, and they are embedded within constantly changing hospital organizations. Therefore, it would be unrealistic to evaluate the intervention initiatives that originate from the organizational intervention process in this study as being isolated from the changes, terms, and events surrounding the project. Hence, it seemed essential to choose an evaluation model that embraced both processes and outcomes by tracking and assessing the interlinked elements relevant to the evaluation. These elements, such as the change mechanisms related to the health professionals’ attitudes (relational coordination and safety culture), the intervention processes (initiation, screening, action planning, and implementation), and the external context, were measured and documented during the intervention process in the surgical unit. The framework used for the evaluation of the organizational interventions in the surgical unit in this study is shown in Figure 7, which was inspired by Nielsen and Abildgaard (2013).

However, the practical implications of using a model that include both process and outcome evaluations in this study will be described in Chapter 5. Methods and Procedures, when the data collection methods and procedures applied in PHASE II, PHASE III, and PHASE IV are presented.

4.4 Summary

This mixed methods study was designed with the purpose to create new knowledge about how communication and relationships are practiced in interdisciplinary surgical teams in contexts of variable complexity in DK, guided by the theory of relational

coordination, as well as to offer recommendations on how best to improve the quality of collaboration and safety culture in surgical teams in the future.

A multiphase design was used, in which four phases were conducted sequentially and concurrent over time. These phases are: PHASE I (Exploration), PHASE II (Monitoring

& Development), PHASE III (Assessment), and PHASE IV (Integration &

Interpretation).

In PHASE I, qualitative data were collected from observations, interviews, and focus group interviews, inspired by the ethnographic principles in practice suggested by Atkinson (2015), Spradley (1979, 1980), and Hammersley and Atkinson (2007), in order to form the basis for the planning interventions applied in PHASE II.

In PHASE II, qualitative data were drawn from an organizational intervention process inspired by the Relational Model of Organizational Change (Gittell, 2000b, 2016; Gittell et al., 2010; Gittell et al., 2011), while the Framework for evaluation of organizational interventions (Nielsen and Abildgaard, 2013) provided the basis for the development, implementation, and evaluation of the interventions. Further, in PHASE II, the quantitative data collected from measurements of relational coordination in PHASE III were used as a prioritizing tool during the intervention process (Gittell et al., 2011, Gittell, 2016).

In PHASE III, the quantitative data collected from measurements of relational coordination and safety culture were analyzed separately and then compared (Gittell, 2012a, 2016; Sexton et al., 2002; Kristensen, 2016a, 2016b).

Finally, the qualitative data and findings derived from PHASE I and PHASE II, as well as the quantitative data from PHASE II and PHASE III, were integrated and interpreted during PHASE IV, as inspired by Creswell and Plano Clark (2011), Greene (2007), Fetters, Curry, and Guetterman (Fetters et al., 2013; Guetterman et al., 2015).