Quantitative analysis – hierarchical regression

77

studying English proficiency or evaluating the competence of employees or collective groups.

78

employee peer evaluations across multiple contexts, where English is expected to play a prominent role, is not leveraged in predicting outcomes in other contexts, but rather as indicators of a generative mechanism in the settings of the study, or in other words, the “properties of a structural entity” (Ron, 2002: 120). Moreover, the approach in Paper 3 is particularly suitable for outlining implications for employee behaviour (measured through a questionnaire). The depth of the qualitative case study in Paper 2 uses variations in formulation and argumentation to position employee discourse and ideology as factors behind the role of English as an organisational language. While interviews with the academic respondents in Paper 3 would give insight into the functioning of, and conditions for, how English proficiency influences their evaluation of their expatriate peers, the analysis of the moderating role of the proficiency variable allows for the theorisation of a relationship which the interviewees might not have formulated in an interview situation. In a hierarchical regression, by adding new variables and interaction terms in each model, their explanatory power on the dependent variable may be compared.

Thus, we can compare how various control and independent variables influence the responses. By finding the model with the greatest explanatory power, we find an indication of the explanation which accounts for most of the variance in the dependent variable – perceptions of collaboration performance. However, by analysing the moderating effect of our main independent variable of interest – English language proficiency, we also examine its role in influencing the strength of other relationships between variables. In critical realist terms, framed within the given context, these relationships indicate mechanisms, as the questionnaire constructs are theorised to represent actual employee perceptions and relationships between them (Ron, 2002).

Given the diversity in critical realist thought, there is also a broad spectrum of opinions concerning the scientific value of quantitative methodology. On one end of the spectrum, some scholars have completely dismissed statistical analysis as an

79

independent empirical strategy and basis for outlining mechanisms. Sayer (1992) questions the measurement of socially dependent and contextually varying phenomena with scales, as one does when measuring qualitatively invariant objects.

Thus, the instability found in social science contexts is argued to make interval scales inaccurate and inappropriate for explaining the causality behind a phenomenon. A second issue with quantitative research designs highlighted by Sayer (ibid.) is that these analyses amount to an inference of probabilistic causality.

Such inferences may be critiqued for aiming to establish law-like transferable generalisations on the basis of variable correlations found in one specific, or a selection of, contexts. Critical realist research, on the other hand, focuses on what a phenomenon is and why processes happen, than predicting future events (Ron, 2002). In this perspective, the strengths of quantitative designs do not coincide with the primary objectives of interpreting and explaining the empirical through a continuous iteration between data and theory. While the generalisability of quantitative results to a certain category of contexts is commonly regarded as a quality criterion, the critical realist perspective emphasises the specificity of context to explain a given phenomenon. Thus, while several scholars in this paradigm acknowledge the value of statistical analysis, such studies are often given a complementary role in mixed methods projects in the same empirical context, to find general patterns in data which can be studied in further details through qualitative methods (Hurrell, 2014).

The sample of respondents in the third paper consists of expatriate academics and their most important collaboration partners in predominantly Northern European Universities (Denmark, Sweden, Norway, and Finland). There are several sample characteristics which could explain our results, and must therefore be discussed. In a first round, expatriate academics were identified and asked to provide their nationality and country of residence, as well as their most important local collaboration partner and permission to contact them. Thus, one can expect a

80

potential selectiveness in the initial sample of respondents, as only academic who apparently matched the sample criteria for being an expatriate academic were invited. Furthermore, since only respondents who were willing to participate did so, our sample excludes all potential respondent who are either not inclined to participate in research or do not have the time. Of the 1274 responses (response rate of 32%), 884 expatriates gave consent for the second questionnaire. Thus, a large portion of respondents with potentially different experiences of the importance of English proficiency might have been left out. In addition, from those who responded positively, only a limited number provided contact with a colleague in the dyad. The second round of questionnaire deployment had a response rate of 21.4% and after combination with the initial expatriate dataset yielded 189 pairs of responses from expatriate academics and their local collaboration partners. Thus, it is possible that only those who had a good collaboration relationship with a local colleague, provided contact for participation in the study.

Table 5: Sample descriptive statistics (Paper 3) Sample composition:

Age average: 42.14 years (SD: 10.25)

Gender: Male: 70.8%, Female: 29.2%

Years worked in host country average: 10.55 years (SD: 7.36) Tenure at current employer average: 7.14 years (SD: 5.77)

Most represented nationalities: Germany 8.8%, United States 8.2%, Italy 6.4%

Country of employment: Denmark 29.8%, Finland 29.2%, Sweden 21.6%, Norway 6.4%, Other - East Asian university, e.g. Hong Kong, Taiwan, China 13%

Academic positions: Professorial rank (Full, associate, assistant) 55%, Research/teaching assistant 21.6%, PhD student 14.6%, Post-doctoral (5.8%)

81

The composition of this sample and the contexts which the respondents are situated in have several implications for the theoretical contribution of this thesis.

Most importantly, we have studied expatriates in a predominantly Northern European context, where we have had a theoretical expectation that English plays a central role in the academic organisations (Hultgren et al., 2014). By studying the potential relationship between perceptions of language proficiency and perceptions of individual competence and organisational status, we have aimed to explore what the individual level and interpersonal implications are for academics in this professional and linguistic context.

The suitability of quantitative methodology in critical realist research depends on how the researcher utilises the instrument and analysis to make conclusions based on questionnaire responses on the empirical layer of reality.

Therefore, a discussion of how questionnaires capture our target concepts on the empirical level is warranted. While each questionnaire item deserves separate attention, the general notion of using a digital questionnaire to measure employee opinions and attitudes is not without challenges. Firstly, a single instance of an employee response on a scale does not necessarily mean that the respondent actually carries the preconceptions we interpret them to have and, more importantly, enacts them in interaction with their peers. Secondly, despite using a validated scale which has been statistically tested for consistency across contexts, it is not possible to guarantee that we have full knowledge of its interpretation in our focus context.

Elements, such as respondent interpretation of the vocabulary applied in questions, or the numerical scale intervals may both vary. Thus, caution is needed when interpreting an employee’s responses. Finally, the questionnaire only presents a static relationship between variables at the time of the respondents’ replies, and thus provide limited insight into the temporal unfolding of relationships or processes.

However, a critical realist framing of our instrument collectively strengthens the interpretation of relationships between items on the questionnaire.

82

The key variables in the third paper are collaboration performance, academic reputation and English/local language proficiency. In order to generate data for these variables, the following instruments have been applied in the questionnaire.

The collaboration partners were asked to rate “Your collaboration partner’s level of performance quality” through Early’s (1987) scale with ratings on a seven-point Likert-scale ranging from (1) “Very poor” to (7) “Very good”. This instrument has aimed to generate values for the study’s dependent variable. By asking the collaboration partners to rate the quality of their peers’ collaboration, we have aimed to capture the individual-level perception of the expatriate’s ability to perform the tasks expected of them, when engaging in professional academic collaboration.

Individual responses here are likely to reflect the peer’s impression of the expatriate’s ability to adhere to the quality criteria in socially shared norms associated with academic conventions and values (Bauder, 2006), or academic habitus (Bourdieu, 1988). However, “performance quality” could carry ambiguity, as it does not indicate what kind of performance, or within which area. Therefore, the peer’s interpretation of performance will influence which areas and types of collaboration activities and quality criteria they weigh.

We have gauged expatriate academic reputation, by posing the following question to their peers: “Within your immediate work group's research field how would you rate this person's [expatriate academic] academic reputation?”.

Responses may span from (1) “Among the highest 1 percent of the field's researchers” to (7) “Lower than 86 percent of the field’s researchers”. The instrument functions thus as a proxy for peer assessment of the construct ‘academic reputation’. The terms ‘academic reputation’ and ‘immediate work group’s research field’ are broad enough to include sentiments concerning the expatriates’ social standing. The questionnaire items aim to capture to what extent the expatriate’s peer has registered that the expatriate receives positive evaluations and is known for their work within a wider academic community. While this proxy also relies on the peer’s

83

access to information and experience concerning the construct, ratings are interpreted to indicate the expatriate’s reputation.

We have measured language proficiency through a four-item seven-point scale. The peers have been asked to rate expatriate academics’ language proficiency on a seven-point Likert-scale, ranging from (1) “Very poor” to (7) “Very good”.

The items are: “His/her general English language skills are”, “His/her written English skills are”, “His/her oral English language skills are” and “His/her English language skills compared to others at this workplace are”. We have used the same for local language proficiency, where the items refer to the local language – e.g.

“His/her oral local language skills are”. While peer assessments of language proficiency are not treated as precise measurements of objective language proficiency, the items indicate how the expatriates’ language skills are registered by their collaboration partner, and thus evaluated. Since the present study aims to study interpersonal implications of English as a key organisational language, peer perceptions of language proficiency are a suitable instrument for this purpose.

While the values of the items alone may be attributed to a wide range of other factors, the paper makes its theoretical inference based on an analysis which reveals relationships and interaction between variables. Regression analysis operates on the basis of a mathematical function, where the value of the dependent variable Y, varies according to the independent variables X. The formula represents a linear relationship based on the scatter plot of values along a scale. The linear function may thus predict the value of Y for values of X (Ringdal, 2013). While the aim of this method here is not to predict the value of Y outside the studied context, the function resulting from the analysis is based on a statistical analysis of the existing data distribution, and may thus provide the basis for inferencing a mechanism.

Hierarchical regression analysis allows the researcher to assess the cumulative effect of independent variables on the variation of dependent variables.

This is done through block regression, where the explanatory power of models may

84

be compared (Ringdal, 2013). In the first step, we gauged the effect of our control variables: tenure, collaboration duration, whether English is the home language, work adjustment, gender similarity, age difference. Thus, we could control that our chosen control variables were not more powerful in explaining the phenomenon of interest than the selected independent variables. In the second step, we gauge the introduction of our baseline hypothesis variable – perception of academic status.

After finding a significant positive relationship confirming our baseline hypothesis, we introduce the English and local language proficiency variables in the third step of the regression. Based on the increased R²-value, we can infer that, together, status and English language proficiency evaluations play a key role in how the peers evaluate the expatriate colleagues’ performance. As these two variables play prominent roles in explaining the variation of our dependent variable, it was necessary to explore the details of these relationships further by testing for interaction. We have done this in the fourth step, where we introduce the interaction term of status and English proficiency evaluations.

The final step in the hierarchical regression is the most significant building block of the critical realist theorisation of the results. As opposed to only analysing correlations through regression analysis, the analysis of interaction highlights the interrelated manner in which the independent variables influence the dependent variable (Ringdal, 2013). This relation reflects the characteristics of a mechanism within our sample. In Paper 3, we also outline which contextual conditions are likely to contribute to the formation of this interrelation. While some critical realist readings of quantitative results tend to downplay the role of regression analysis in theorisation (Sayer, 1992), scholars have argued that finding a relationship between variables within a sample is a substantial basis for theorising how and why the given variables influence each other as found in the analysis (Downward, Finch and Ramsay, 2002; Hurrell, 2014; Ron, 2002; Zachariadis, Scott and Barrett, 2013). For instance, a statistical analysis which outlines the moderating role of a variable is an

85

indicator that the construct measured plays a modifying role in a mechanism.

Furthermore, the researcher may analyse the contextual conditions of the sample and variables as probable enabling factors for the illustrated interaction. Thus, we have moved beyond more static interpretations of regression analysis, where we show that higher evaluations of status and English proficiency each, separately, correlate with higher performance evaluations. We also show that these perceptions depend on one another, as the strength of their impact on the dependent variable varies according to the value of the moderating variable.

The data for Paper 3 was collected before the EU’s GDPR Directive.

However, participants were still asked to confirm the voluntary nature of their participation and that all responses and data collected concerning the identity of the respondent would be anonymised. The final dataset is stored at Aarhus University servers in an anonymised format. A further ethical consideration is the impact of a questionnaire on the field. Like qualitative field work, the collection of quantitative data is also a form of scholarly intervention which can leave a mark on the empirical site (Tharenou, Donohue and Cooper, 2007). Since the key variables in Paper 3 relate to individual status and language proficiency, the potential effect of asking such questions is considered here. Although the link between status, or other positive personal attributes, and language proficiency is not apparent on the user interface of the questionnaire, asking peers to evaluate such attributes could induce stronger focus on peer qualities. In turn, this could have a negative consequence for the relationship between the peers, if tension emanating from negative attribute perception is exacerbated. However, the varied nature of topics covered in the questionnaire directs attention away from specific attributes and minimises the potential increased attention on single variable constructs.

86