The reports we have sampled were chosen for their political relevance (all three have been used as foundation for policy), public appeal (all three were presented in the media), and quality as professional work (all three were written by lead-ing agencies within the field). However, we can conclude that even though the sampled reports make heavy use of both qualitative and quantitative methods, they do not provide evidence according to the established academic standards
according to which concepts are defined, claims are based on critical investiga-tion, arguments are theoretically scrutinised, and knowledge is considered to be a contribution to common understanding only after it has passed through a peer review process.
Thus we need to distinguish between academic and strategic evidence. The latter is evidence that employs scientific methods and language, but does so in order to provide answers to political agendas; furthermore, the very production of evidence should be interpreted as the outcome of a network of actors, including politicians and administrators in need of evidence to back up policy, research institutes providing ways of acquiring this evidence, and professional media.
In connection with this point, the methodological issues communicated to the scientific reader, e.g. the use of disclaimers and mentioned biases, function to make the evidence valid. However, these methodological issues remain unrelated to the critical parts of the reports communicated to the political reader, and when the reports are cited in professional media, they address the professional reader.
The experts contribute to the confusion, allowing themselves to be cited as experts in something they have not actually studied.
Notes
1 We would like to thank Jane Due for valuable research assistance.
2 In turn, the municipalities need to deliver the results that the Ministry of Education expects from them, and the achievements of the national government are measured against the results of other countries in PISA and TIMMS.
3 There are a number of other research institutes, most notably the Danish National Centre for Social Research (SFI), which also produce relevant reports. We chose to concentrate on the three institutes that, in our opinion, most often deliver reports on topics related to education.
4 Cohen, Manion, & Morrison (2000, pp. 254), for example, define triangulation as “an attempt to map out, or explain more fully, the richness and complexity of human behavior by studying it from more than one standpoint.”
5 Defining, or even discussing, the issue of causality in more depth is beyond the scope of this paper (but see, for example, Winship and Morgan, 1999; Gangl, 2010; Pearl, 2009).
6 Institutional response rates for the sampled university colleges (90%) and high schools (92%) are reported. However, the possibly more important response rates for the realised college student sample (N=1189) or high school student sample (N=1926) are not reported (see Epinion pp. 14).
This constitutes a critical omission of essential information about these surveys.
7 As pointed out by many survey specialists, great care is needed in the wording of survey questions (Schnell, 2012) – and just because respondents are willing and able to answer a question it does not mean that the answer can be used to address the original research question of a report.
8 Each question is worded so that the respondent has to decide if they agree (or strongly agree), disagree (or strongly disagree), or do not know or find it irrelevant.
9 While many of the pedagogues will find a job in the pre-school sector, their education also qualifies them to work in other pedagogical contexts with older children or even adults.
10 http://www.folkeskolen.dk/515186/laereruddannelsen-skal-taettere-paa-folkeskolen (10.10.14).
References
Aisinger, P. (2013, March 22). Manglende praksiskobling giver stort frafald i lærerud-dannelsen. Retrieved June 22 2014, from http://www.folkeskolen.dk/525602/
manglende-praksiskobling-giver-stort-frafald-i-laereruddannelsen Bryman, A. (2007). Barriers to Integrating Quantitative and Qualitative
Research. Journal of Mixed Methods Research, 1(1), 8-22.
Callon, M. (1986). Power, action and belief: a new sociology of knowledge? In:
J. Law (ed), Some Elements of a Sociology of Translation: Domestication of the Scallops and the Fishermen of St Brieuc Bay (pp. 196-223). London: Routledge.
Cohen, L., Manion, L., & Morrison, K. (2000). Research Methods in Education.
London: Routledge/Falmer.
Clarke, K. (2005). The Phantom Menace: Omitted Variable Bias in Econometric Research. Conflict Management and Peace Science, 22(4), 341-352.
Epinion (2007). Valg og fravalg af lærer-, pædagog-, sygeplejerske- og socialrådgiver-uddannelsen. København: Epinion for Undervisningsministeriet.
Epinion (2014). Who are we? Retrieved October 3, 2014, from http://epinion-global.com/about/who-are-we/
EVA (2013). Frafald på læreruddannelsen. En undersøgelse af årsager til frafald.
København: Danmarks Evalueringsinstitut.
Gangl, M. (2010). Causal Inference in Sociological Research. Annual Review of Sociology, 36(1), 21-47.
Groves, R.M. (2006). Non-response Rates and Non-response Bias in Household Surveys. Public Opinion Quarterly, 70(5), pp. 646-675.
Hagens, T.G. (2006). Conscience Collective or False Consciousness?: Adorno’s Critique of Durkheim’s Sociology of Morals. Journal of Classical Sociology (6), 215-236.
Hartung, D., Nuthmann, R., & Teichler, U. (1981). Qualifikation und Beruf. In:
D. Hartung, R. Nuthmann, & U. Teichler (eds), Bildung und Beschäftigung, (pp. 157-204). München: KG Saur.
Howe, K.R. (1988). Against the Quantitative-Qualitative Incompatibility Thesis or Dogmas Die Hard. Educational Researcher, 17(8), pp. 10-16.
KORA (2008). Jensen, T.P., Kamstrup A.K., & Haselmann, S. Professionsbachelor-uddannelserne - de studerendes vurdering af studiemiljø, studieform og motivation for at gennemføre. København: AKF.
KORA (2010). Jensen, T.P., & Haselmann, S. Studerendes vurdering af teori og praksis på professionsbacheloruddannelserne. København: AKF.
KORA (2012). Jensen, T.P., Olesen, L.S., & Dahlgaard, J.O. Professionshøjskolernes udviklingsforpligtelse, videnbasering og videncenterfunktion – Kortlægning og kvalitative studier. København: AKF.
Latour, B. (2005). Reassembling the Social: An Introduction to Actor-Network-Theory.
Oxford: Oxford University Press.
Nørby, J. (2012, August 8). Læreruddannelsen skal tættere på folkeskolen. Retrieved October 10 2014, from http://www.folkeskolen.dk/515186/laereruddannels-en-skal-taettere-paa-folkeskolen
Payne, G., Williams, M., & Chamberlain, S. (2004). Methodological Pluralism in British Sociology. Sociology, 38(1), pp. 153-163.
Pearl, J. (2009). Causality: models, reasoning, and inference (2nd ed.). Cambridge:
Cambridge University Press.
Regeringen (2013, March 8). Bekendtgørelse om uddannelsen til professionsbachelor som lærer i folkeskolen, BEK nr. 231 af 8.3.2013. Retrieved October 10 2014, from https://www.retsinformation.dk/Forms/R0710.aspx?id=145748&exp=1 Sale, J.E., Lohfeld, L.H., & Brazil, K. (2002). Revisiting the
Quantitative-Qualita-tive Debate: Implications for Mixed-Methods Research. Quality & Quantity, 36(1), pp. 43-53.
Schnell, R. (2012). Survey-Interviews. Methoden standardisierter Befragungen.
Wiesbaden: VS Verlag für Sozialwissenschaften.
Shavit, Y., & Müller, W. (eds). (1998). From School to Work - a Comparative Study of Educational Qualifications and Occupational Destinations. Oxford: Clarendon Press.
Tracy, S.J. (2010). Qualitative Quality: Eight ‘’Big-Tent’’ Criteria for Excellent Qualitative Research. Qualitative Inquiry, 16(10), pp. 837-51.
Weirsøe, M. & Holm-Pedersen, P. (2011). Stordriftspædagogik øger frafaldet.
Forskning (pp. 4-7). Bupl.dk. Retrieved October 10, from http://www.bupl.dk/
iwfile/BALG-8LFJ5D/$file/12stordrift.pdf
Winship, C., & Morgan, S.L. (1999). The estimation of causal effects from observational data. Annual Review of Sociology, 25, pp. 659-706.
Dansk abstract
Den strategiske brug af evidens i læreruddannelse: en undersøgelse af forskningsrapportgenren
Denne artikel analyserer tre forskellige rapporter. Alle rapporter omhandler læreruddannelse, er produceret af forskellige analyseinstitutter i Danmark og er brugt som evidens indenfor uddannelsessektoren. Analysen identificerer kritiske metodologiske og konceptuelle forhold, der er relateret til produktionen af forsk-ningsrapporter, og relaterer disse til debatten om evidens i uddannelser. Artiklen afsluttes med refleksioner over forskelle mellem akademisk og strategisk evidens.
Nøgleord: evidens, læreruddannelse, forskningsrapporter, mixed methods, epistemologi.