• Ingen resultater fundet

3.3 Theme two: Professional development

3.5.1 The role of fidelity

In a quasi-experimental study by Caven et al. (2012) of three schools in Massachusetts, learning time was extended. The study examines the effect of the extra time on schools, tea-chers, and students over three and four years of implementation. Participating schools were required to expand learning time by at least 300 hours per academic year, with the aim of improving student outcomes in core academic subjects, broadening enrichment opportunities, and improving instruction by adding more planning and professional development time for teachers. Schools were able to draw upon state resources as well as technical assistance and support from Massachusetts 2020 and Focus on Results to implement expanded lear-ning time in their schools. Participating schools received an additional $1,300 per student to lengthen the day and/or year.

The effects were studied using qualitative ethnographic field studies of classroom acti-vities, individual interviews with school administrators and teachers, and focus-group interviews with students. Results were also assessed quantitatively by means of teacher and student surveys on attitudes to and perceptions of school, including relationships with teachers and participation in extracurricular activities. These data were supplemented by data from administrative public registers on student-level achievement in reading/English language, mathematics, and science exams as well as student-level characteristics (such as special education status). The data also contains information from the end-of-year student information management system files (such as student-level demographic variables and behaviour variables, including attendance rates, truancy rates, in-school suspension rates, and out-of-school suspension rates).

The study shows that the three participating schools allocated their expanded learning time with differing degrees of fidelity. Across the schools, use of time, approaches to academic support and enrichment, and engagement of community partners varied considerably.

All three schools worked with the Massachusetts Expanded Learning Time (ELT) implemen-tation index. The Expanded Learning Time index worked as a guide for both implemenimplemen-tation and evaluation of implementation. The index focuses on the following factors: school-wide academic focus, support, and instructional practice; enrichment activities; teacher leadership and collaboration; school leadership; and stakeholder satisfaction. Higher implementing

schools showed significantly greater effects on students’ academic and social outcomes than in low implementing schools after both three and four years of implementation.

High levels of student engagement were observed in classrooms where teachers had care-fully planned and executed lessons. Additionally, in those classrooms where students were more engaged, there appeared to be positive relationships between students and teachers.

Students did not appear to be more engaged in one subject within or across schools or one type of class. Nor was student engagement sensitive to time of day. At all three schools, stu-dents articulated that enrichment classes were fun and enabled them to build connections to the school and staff.

The effect of extended-learning time was significantly positive in science and maths. A signifi-cantly higher proportion of teachers in the extended-learning schools reported that the length of the school day allowed them to accomplish their teaching goals and cover the amount of instructional material their students need to learn than might be expected in the absence of longer learning time. A significantly higher proportion of teachers in extended-learning schools reported that they were satisfied with the amount of time available for instruction in English language, mathematics, and science than expected in the absence of longer time. At all three schools, observers noticed evidence that teachers throughout the school were using data to inform instruction. At one of the schools, a bulletin board in the principal’s office depicted each student‘s achievement test results over time. Students, each represented by a post-it note, advanced from red to yellow to green as their scores on recurring assessments improved.

The PhD thesis by Mayer (2012) examined the impact of implementation adherence moni-toring and group feedback procedures when teachers implemented the Levelled Literacy Intervention (LLI) among first- and third-grade pupils. The study investigated changes in implementation adherence levels as teachers participated in problem-solving feedback sessions. Pre- and post-surveys were also employed to examine changes in teacher percep-tions of implementation monitoring and feedback procedures, as well as the relapercep-tionship between various background variables (e.g. previous teaching experience using LLI teaching ) and implementation adherence levels. This was a literacy intervention for a small group of students (typically three to four students) who were below grade-level in early reading and writing skills. The overall goal of the intervention was to bring pupils up to grade-level within 14 to 18 weeks. Each thirty-minute intervention session was systematic and sequen-ced, including explicit instruction using a variety of levelled books and other materials for teaching word structure and sound.

Fifteen teachers from seven elementary schools in a district in the south-eastern United States agreed to participate in the study, after attending an introductory meeting at their school. All teachers had 34 school days to implement LLI, and it was recommended that they hold four to five sessions per week. In this period there were three feedback sessions, separated by two to three weeks. An introductory meeting took place before these sessions.

The procedure was conducted in the autumn of 2010 with one teacher group, and in the spring 2011 with two teacher groups. Each teacher group comprised four to six teachers.

The research design was quantitative, and involved repeated measures. The implementa-tion adherence level was measured by a checklist of implementaimplementa-tion adherence to LLI. The author developed this checklist because no such adherence monitoring tool had previously been developed, and its validity was ensured by expert feedback.

The implementation adherence checklist was completed by teacher self-reports, the primary researcher, and two research assistants. The fifteen teachers completed an implementation adherence checklist after every LLI session they conducted. Between one and two sessions were video-recorded every week, and a total of 88 usable videos were analysed. The resear-cher viewed all video records while completing the implementation adherence checklist. The two research assistants viewed approximately 30 per cent of the videos while completing the implementation adherence checklist. Overall, agreement between the teachers’ self-reports and the video-recording (researcher/ research assistant) was good. The agreement between researcher and research assistant was excellent.

During group feedback sessions, the primary researcher presented implementation adherence data to the teachers. In other words, she presented the group’s average implementation levels and per cent agreement between the self-reporting checklists and the researcher-completed checklists. In the group feedback sessions, teachers also had the opportunity to share common barriers that interfered with implementing the intervention and to share recommendations for overcoming those barriers. After each session teachers received a written summary of their shared recommendations.

The study did not demonstrate a significant functional relationship between group feedback sessions and implementation adherence levels. However, teachers generally maintained high-implementation adherence levels throughout the study. In other studies, by contrast, implementation adherence often appears high in the beginning of the implementation and then declines. It is likely that the self-reporting checklists, video observations, and group feedback sessions were influential in maintaining implementation adherence in this study.

Other factors were also assessed that might have affected implementation adherence. These included previous training and experience, current LLI responsibility, and primary teaching assignment. The only factor that had significant effect on LLI implementation adherence was the number of previous LLI student groups taught. Teachers who had taught more student groups had higher implementation adherence levels.

The author noted that the recommended intervention frequency was not followed. Four to five LLI sessions had been recommended conducted four to five times a week, but only two teachers followed this frequency rate, and other teachers had a lower frequency rate. Teachers held between 16 and 31 intervention sessions during the LLI period. Teachers often indicated reasons why the intervention was cancelled, and common reasons given were often related to school planning, for example for end-of-grade testing, teacher workshops, field trips, or teacher meetings. The author indicated that it was difficult to coordinate scheduling of LLI groups with the classroom teachers’ schedules, and that time for daily uninterrupted LLI teaching was a challenge.

Perceptions about LLI monitoring and feedback procedures and understanding of essential steps were also assessed in pre- and post-ratings. A pre-feedback session survey was completed by teachers at the introductory meeting before teachers experienced involvement with LLI adherence monitoring and feedback procedures. A post-feedback session survey was com-pleted at the end of the last feedback session. The findings indicated that after participating in these, teachers had positive perceptions of implementation monitoring and feedback methods. Teachers also reported significantly greater understanding of the essential LLI steps in post-ratings than pre-ratings. The author noted that it was very possible that the sample of teachers in the study were more committed to implementing LLI and more open to having their own behaviour assessed and discussed than the average teacher, as they had agreed to participate in the study after attending an introductory meeting of LLI. Thus it was important to be aware of this particular sample of teachers when considering the results.

The aim of the study by Korkeamaki & Dreher (2011) was to determine what kind of literacy instruction took place in grades one and two in one Finnish school, and to find out how the teachers were able to interpret the core curriculum and implement it into their classrooms.

In 2004 the Finnish National Board of Education introduced a new curriculum framework that aimed to make students active participants in their own learning. Accordingly, the

learning environment was to encourage students’ curiosity, interest, and motivation. Mo-reover, the learning environment was to promote students’ abilities to interpret, question, and formulate arguments. These abilities were of particular importance because research showed that Finland’s students were not particularly strong in these areas.

The study focused specifically on the core curriculum of the subject “mother tongue and literature.” This curriculum reflected the visions outlined above and involved a list of content to be included. It provided guiding principles (for instance,. that students’ interaction skills should improve), then more specific objectives (for instance, that pupils should “become accustomed to interactive situations at school”). The list of content was neither detailed nor complete, reflecting one of the characteristics of the Finnish school system, that it is based on trust that teachers will interpret the curriculum to fit their own classroom context. The Finnish core curriculum is in fact one of the least prescriptive curricula among European countries. The teachers were to structure the content according to the context and level of students’ learning. Finnish research suggested that this curricular approach, although in-teresting, was demanding, as well as challenging for teachers.

The study is a case-study of one Finnish school in 2006, two years after the implementation of the core curriculum. Twelve teachers from first- and second-grade literature classes were observed during a two-month period. Classroom observations were conducted by groups of between two and four pre-service teachers as part of their literature course. A total of 44 pre-service observations were involved. Multiple observers allowed for cross-checking of the observations. In addition, the findings were also presented to the teachers who had been observed so as to validate the data.

The qualitative data was analysed by comparing, first, the observed behaviour of the tea-chers and, second, the principles of the core curriculum of mother tongue and literature.

The results revealed that teacher practice in the classroom did not correspond fully with the requirements of the core curriculum. The authors distinguished between shortcomings in the teachers’ teaching style and shortcomings in the content of the lesson. Many areas of content were covered, including language awareness and students’ reading. However, there was no observation of students spending time sharing their reading with peers or teachers, and no observation of creative writing. These activities were all specific objectives outlined in the core curriculum.

Most teachers followed a traditional teaching style of “teacher-led instruction”: for instance, a teacher asked a question and the student replied. Thus teachers followed a rigid structure

which left little time for reflective discussion, which was the intention of the core curricu-lum. The classroom teachers did not promote the students’ ability to interpret, question, or develop argument, and there was no observation of spontaneous shifts in the structure of the class to cater for student interest or motivation. The teachers often followed prescribed activities such as textbooks which had teacher guides on how to work with the text. This however had not been the intention of the core curriculum.

The authors of the study have not addressed how the school as a whole worked with the core curriculum to make teachers aware of the requirements during the two years. Thus it is unclear how the process of implementation of the core curriculum was unfolding in the school. The authors assume that the lack of fidelity between the core curriculum and the teachers’ behaviour was due to the fact that the core curriculum was demanding and challenging for teachers. They describe how teachers were positive towards the curriculum.

However, the core curriculum provides only principles and objectives, and this requires not only considerable content and pedagogical knowledge, but also a significant commitment of time among teachers.

The authors conclude that the intangible nature of the Finnish core curriculum was challen-ging. They therefore suggested giving teachers more specific training or guidelines. However, providing more tangible guidelines can run counter to the idea of creative and interpretive teaching. Overall, the authors argued that the fact that the core curriculum was intangible and non-prescriptive hindered the implementation process.

The aim of the study by de Kock & Harskamp (2014) was to investigate whether students’

performance and ability in specific mathematics tasks could be improved by an eviden-ce-based intervention, a metacognitive computer programme. The metacognitive computer programme assisted students in Word Problem Solving (WPS) in mathematics classes by providing specific hints. WPS are mathematics problems in written form: for instance, “Peter has eight apples and buys five more. How many apples does Peter have now?” In the compu-ter programme, students could choose hints when they did not know how to continue their solution process in WPS. The steps of hints were presented with a visual representation of the main features of the mathematics problem, without giving away the answer.

A quasi-experimental design was carried out on eighteen grade-five classes in twelve mid-dle-sized elementary schools in the Netherlands which were assigned either to the experi-mental or the control condition. In total, eighteen teachers and 390 students took part. Twelve

teachers and their 280 students were assigned the experimental condition, and six teachers and 110 students the control condition.

The experimental condition involved teachers implementing the metacognitive computer programme in their mathematics classes, while the control condition involved using the usual mathematics textbook. Teachers in both control and experimental condition participated in workshops preparing them for (respectively) using the usual mathematics textbook for assisting students in WPS or using the metacognitive computer programme. Thus measures were taken to encourage both groups to believe that they were part of an evidence-based intervention. Moreover, both groups were given the same evidence-based information about WPS. However, teachers in the experimental group had to use the metacognitive computer programme to support the students in their process of problem-solving, but the control group had to use the mathematics textbook. In the workshops, both groups were given time to plan their ten-week implementation period, so that their new knowledge and the computer programme were integrated in the mathematics curriculum that fitted their schedule.

In the experimental condition, students were given a pre-trial in the programme before the ten-week implementation period started. During the ten weeks, they worked with the computer programme for twenty minutes each week. Teachers were instructed to check the students’ performance every week, and they were instructed to give feedback to students who failed to use hints or did not manage to solve particular word problem tasks. To check if the teachers tracked their students sufficiently, teachers had to fill in a logbook every week.

In the control condition, teachers followed the mathematics textbook and used training materials with word problem samples. Teachers organised four to five mathematics lessons of 45–50 minutes each on the topic during the ten-week period.

Overall results (multivariate multilevel analysis) showed that students assigned to the me-tacognitive computer programme significantly outperformed the control group in solving and analysing word problems. They also scored significantly higher on self-monitoring, a metacognitive ability necessary for WPS in mathematics. These results controlled for dif-ferences in the pre-test scores of the two groups. Analysis of students’ computer log files during the ten weeks showed that students in the experimental condition maintained their performance in solving problems correctly during this time.

The computer programme gave students immediate standardised hints or metacognitive prompts relevant for WPS. In the control condition, students received hints or prompts when the teacher had the time. Moreover, the results of teacher self-reported questionnaires

showed that the six teachers in the control condition used the instruction in WPS they had received only sparsely, and that their use of it did not increase over the ten-week period. It thus appeared that students in the control condition received inadequate training in WPS compared to the experimental condition. In other words, the study indicated that a metacog-nitive computer programme guided by a teacher is more effective in providing immediate and relevant WPS feedback than a typical teaching setting in which one teacher caters for all students.

The authors highlighted that the results of this study, in which teachers had implemented, organised and supervised the intervention, were less strong than previous studies (controlled experiments) with the same computer programme in which researchers had implemented, organised, and supervised the intervention.

The results of the students’ computer log files indicated that students were given enough time to work with the computer programme on a regular basis. Moreover, the log files in-dicated that teachers were capable of organising their timetable and combining their daily tuition with the computer programme. In other words, the implementation appeared to be well managed in terms of planning and incorporating it in the mathematics curriculum.

From the teachers’ logbooks it was clear that teachers in the experimental group did not use the computer programme to the maximum. They did not, for example, use the computer data to analyse which individual students could improve their problem-solving techniques.

Moreover, teachers gave students hardly any individual feedback, nor did they help them improve their hint usage. The authors recommended that teachers should in future be trained in the specific skills mentioned above to improve the implementation of the metacognitive computer programme.

The results of the study indicate that success in both the experimental and the control condition were somewhat compromised because teachers in both conditions did not ade-quately follow the instructions given. The authors indicate that teachers’ fidelity to the instructions of the implementation has a significant impact on enhanced student learning.

The study by Cross et al. (2015) was conducted in order to address gaps in fidelity measure-ment and to contribute to understanding the relationship between fidelity and outcomes in the context of a randomised trial of a school-based preventive intervention delivered by paraprofessionals. Two aims are presented in the study: first, to develop reliable observational