The third element of the synthesis consists of going through the factors running across the studies to find the ones that can explain differences in the direction and strength of the studied effect. In this connection the question is also addressed as to why a phenomenon does or does not have an effect, and whether there are special circumstances that play a part and can explain why an effect is strengthened or weakened
4.4.1 Direction and strength of the influence
It is not easy to decide the direction of the influence. And it cannot be excluded that causality, at least par‐
tially, may run from pupil achievement to school inputs. Schools and administrations may respond to low pupil achievement by changing school inputs or teachers. The mechanism will act as a suppressor effect, and will lower the observed correlation between school input and pupil achievement compared to the true school input effects. Conversely, teachers and leaders might self‐select into schools on the basis of pupil achievement, so that the best teachers and school managers select themselves into better schools, thus generating a spurious correlation between teacher and school management quality.
For example, as mentioned on page 71, Educational Leadership as a subcategory of the school factor Man‐
agement and Leadership was found to be negatively related to effectiveness, meaning that less effective schools manifested more educational leadership. This result could be interpreted to say that less effective
sis of the quantitative studies is complemented by a counting of significances. In consequence we cannot rank the relative importance of the various a school factors and subcategories.
4.4.2 The significance of context
Research into effective schools is based on a theory that the results achieved by a school are derived from (a) the individual abilities of the pupils, (b) the cultural, socio‐economic and family background of the pupils and (c) what the pupil experiences at the school. Effective schools research seeks information about factor (c), and attempts to control and correct any influences arising from the other two factors, cf. p. 24 .
In the research mapping (Section 3.1 and 3.2) possible contexts have been noted that could be considered to have significance for an assessment of the direction and strength of the effect. Table 3.1, Table 3.2, Table 3.3, Table 3.5, and Table 3.6 respectively show the studies’ distribution by country, language used in the research reports, the educational setting of the studies, the curriculum area of the studies, and the specific group of pupils that have been examined. In this context it is worth mentioning that it has not been possi‐
ble to show that studies that include data from Nordic countries, give conclusions that differ from studies based on data from non‐Nordic countries.3 Added to these are the contexts that have been brought to light by the reading of the mapped studies. As mentioned in Table 3.12, most studies (97 out of 109) give an adequate description of the context.
It must also be noted that the very concept of the ‘good school’ from the outset should be considered to be both politically controversial and dependent on cultural context. However, as shown above (page 60), the studies employ only two different definitions of the ’good school’: firstly a school with ‘high academic achievements’ and secondly a school with ‘high non‐academic achievements’. However, since the two defi‐
nitions in most cases relate to the same studies, it would seem reasonable to suppose that the two defini‐
tions are not mutually exclusive, but rather supplement each other, i.e. the ‘good school’ is in general un‐
derstood in terms of ‘high pupil achievements’.
Can something be said about the significance of context for the present systematic review? One of the pri‐
mary studies included in the systematic review (see Rumberger & Palardy fig. 1, p. 11) provides a helpful figure about the relationships between context, school processes and pupil experience, and school outputs and pupil outcomes, by creating a conceptual framework for analysing school performance, cf. Figure 4.3 .
3 A similar, resently published Swedish study reaches the same conclusion: ”In broad outline, the same factors appear
as significant when we compare results from Swedish and international studies” (Skolverket, 2009, 33).
97
Figure 4.3: Conceptual Framework for Analysing School Performance
(Source: Rumberger & Palardy, Am. Educ. Research J. 2005, fig. 1, p.11)
If we apply this conceptual framework to the present systematic review, we get Figure 4.4. In the figures, the yellow boxes represent Pupil Background and School Inputs, the content of which shapes the empirical environment for both School Processes and Pupil Experiences, represented by white boxes, resulting in school outputs and pupil outcomes, represented by red boxes.
Figure 4.4: Conceptual framework for the present systematic review
It has been stated in the systematic review that factors relating to background, such as the teacher’s gen‐
der, age, socio‐economic background and ethnicity, by and large play a part in the mapped studies. Some studies especially bring up teacher’s gender and ethnicity as important for specific groups of pupils. In this context we call attention to the fact that one of the reasons for the exclusion of a study was the following screening criterion: ‘When none of these three criteria are part of the study design: 1. Control is present for differences in pupils' socioeconomic background; 2. Control is present for differences in pupils' scholastic aptitude; 3. A pre(‐post) is present. When one criterion is found the study must be included’, cf. Table 2.2,
p. 31. As it appears from Figure 4.4, we have in fact included the following contextual factors, if possible, in the systematic review:
Low, middle, and high SES Gender
Ethnicity
Pupil’s scholastic aptitude
We have applied this information in the studies focusing on the pupils’ socioeconomic background to carry out the syntheses for pupils belonging to separate SES groups, cf. the analyses in Sections 4.3.1.2 and 4.3.2.2. It has not been possible to show in this systematic review that these contexts call for special educa‐
tional treatment for specified Pupil Groups.