The role of support systems

The PhD project by Barker (2011) is a qualitative comparative study examining key compo-nents of the implementation of RtI² (an extension of the original Response to Intervention (RtI) framework, using a data-driven problem-solving model to identify specific student needs) at two elementary schools in a county of southern California.

The study investigates the contribution of the following three areas to successful implemen-tation of RtI²: (1) leadership attributes, skills, and practices; (2) professional development practices; and (3) new roles for general education teachers, special education teachers, and support staff (for a more thorough description of the study see theme one).

One of the study’s conclusions is that professional development practices that encourage collaboration through teams allow teachers and support staff to focus on student achieve-ment and create opportunities to share experiences of instructional practice.

The new and expanding roles and responsibilities for all staff members continue to be re-defined and changed over the course of the RtI² implementation. For many teachers in the study, their roles expanded to include individual or small-group instruction, collaboration with other staff members, monitoring progress, collecting data, analysing data, and modi-fying instruction. The new roles required a change in how both teachers and support staff conducted assessment and intervention practices for struggling students, as well as students with disabilities. These changes in roles and responsibilities for teachers and support staff are very different from what was required in the past.

The final conclusion is that staff members’ new roles and responsibilities as existing resour-ces undergo a redistribution in order to provide support for the implementation of RtI². Resources including staffing and release time must be made available or adjusted both for initial and for continuous implementation of RtI². Because roles and responsibilities of staff members changed with the implementation of RtI², release time for professional development or collaboration is crucial.

The study by Berger et al. (2014) was part of an experimental research project evaluating the impact of a school-level problem-solving team model entitled instructional consultation

teams (IC teams) on student and teacher outcomes conducted as a partnership between university researchers and school-district personnel. The implementation of IC teams was measured in terms of teacher utilisation in seventeen schools located in a suburban school district in a mid-Atlantic state in the United States. These schools received extensive training and ongoing support to implement the programme for four years from 2005/6 to 2008/9.

The IC teams model is a problem-solving intervention model promoting the use of eviden-ce-based instructional and behavioural practices. It is a school-level intervention that requires systemic change, extensive professional development, and external support. It is directed at improving teaching practices and student outcomes as well as addressing teachers’ belief systems and practices. When implementing the IC teams model, each school has a team leader and trainer (referred to in the study as the IC teams facilitator). The facilitator trains and recruits team members, promotes the programme among school personnel, takes on record-keeping duties, and works directly with teachers as a case manager. IC teams are interdisciplinary and often include general educators, administrators, specialists, and spe-cial educators. Team members serve as case managers, working individually with teachers who have requested support. In addition to one-to-one support, the IC team serves multiple purposes: it assigns team members to individual cases, monitors case progress, assists with problem-solving on specific cases, provides continuing professional development to team members, addresses school-level problems, and takes part in evaluation activities.

As part of the implementation of IC teams, extensive professional development efforts take place at schools during multiple years in order to train facilitators, principals, and IC team members. In the current study, this training was provided to schools in three phases over a three-year period. The first phase of training was implemented during the 2005/2006 school year: facilitators and principals, together with one team member from each school, were trained. This training included introductory training as well as seven monthly skills sessi-ons (including practice assignments) and coaching by more experienced facilitators. In the second phase of training, completed during the second year, full teams from each received introductory training and training in consultation skills and processes. In the third phase of training, during the third year, facilitators, principals and team members were provided with technical support training, whose focus was on assisting them to integrate IC teams into existing school structures and to align resources.

After three full years of implementation, programme records indicated that at least 70 per cent of schools in the sample had met the criterion of having 67 per cent of teachers in their school utilise the IC team according to programme prevalence rates. Teachers’ self-reported

data suggested that 82 per cent of schools had met this goal by the second full year of imple-mentation, and that all schools had reached the goal by the final year. There was substantial variation in utilisation rates across schools, ranging from 32 per cent to 93 per cent according to programme records and from 67 per cent to 100 per cent according to teacher self-repor-ting. The annual prevalence rate was more variable, rising in seven schools, falling in five, and mixed in five more. Despite these differences, most schools showed a jump in utilisation rate by the second year and a levelling and more gradual increase over the remaining years.

It must be noted however that high teacher attrition rates (personnel turnover) added com-plexity to the picture and probably influenced the results, because teachers who might have used the programme but then left the school would be missing from the data. However, the overall picture seems to be one of solid progress for some IC teams after three years.

The pattern for teachers’ self-reported membership of IC teams showed a 16 per cent reported membership rate during the baseline year, rising to 23 per cent in the second year, remaining high and then slowly increasing to 28 per cent by the final year. The percentage of teachers approaching an IC team for help with a pupil with academic difficulties increased from 39 per cent during the first year to 60 per cent by the fourth year. Similar patterns were observed for requests for help with behavioural difficulties.

A positive relationship between utilisation rate and facilitator stability was found across both measures of utilisation. In only eight of the seventeen participating schools did the same facilitator remain in place over the three years of the study, and a few schools had a different facilitator in each year. The case illustrations provided some insights as to the role of the facilitator and the skills required to perform it. Overall, it was demonstrated that facilitators need to be active, effective, and skilled in the intervention, should have the confidence of the staff, and should be able to cooperate well with the school principal.

The support of the principal was hypothesised to be of importance to the effectiveness of facilitators and IC teams. Study data suggested that principals taking an active role at the beginning of the implementation process were associated with a higher percentage of teachers using the team. There were no significant relationships between principal attendance at team meetings and utilisation in the first year, and principal attendance at meetings and serving as a case manager in the following year were negatively associated with teacher utilisation.

No apparent patterns were observed between administrator personnel stability and teacher utilisation of IC teams. There was a significant relationship between the principal taking a case in the second year of the study and facilitator stability, although this relationship was

not replicated in the third year of the study. There were no significant associations between principal attendance at meetings and facilitator stability.

One of the major findings of the study was that high levels of teacher turnover limited the impact of the programme. Over two years, one-third of the general education teachers working in the schools at the time the project began had left, and by the fourth year, this number was at 48 per cent. The authors conclude that high turnover rates can potentially impede interventions that rely on training and personnel development, making the goal of obtaining school-level effects and extensive culture change hard to reach. It is speculated that IC teams and similar programmes directed at the practices or attitudes of teachers may have a lower chance of successful implementation in schools with high teacher turnover.

These in turn may be the schools that most need an improvement in teacher practices.

The purpose of the study by Bradshaw & Pas (2011) is twofold: (1) to describe the process by which the state of Maryland scaled up a school-wide prevention model called Positive Behavioural Interventions and Support (PBIS), which aims to improve school climate and student behaviour; and (2) to evaluate contextual factors at the school and district levels that are associated with training, adoption, and implementation.

An increased adoption of prevention models at the district and state levels can be seen in most school settings across states in the United States. These scale-up efforts require con-siderable coordination and resources to ensure high-quality programme implementation across multiple schools. Several models for widespread programme dissemination and implementation have been proposed; however, there are few published empirical examples of the application of these models to the successful implementation of statewide prevention efforts. Bradshaw and Pas argue that further information is needed on how to develop the systems in school settings so as to support their implementation.

The implementation of PBIS is built upon the public health model of three-tiered prevention.

Attention is focused on creating and sustaining primary (school-wide), secondary (targeted/

small group), and tertiary (individual) systems of support that improve the outcomes for all children by reducing problem behaviours and making schools more effective, efficient, and more positive work environments for both students and staff.

Each district in Maryland is required to have some type of district-level PBIS coordinator who provides local leadership for the PBIS effort, participates on the PBIS state leadership

team, and coordinates local and state PBIS training events.

The study is a cross-sectional study with multilevel analyses. Data came from a statewide evaluation of school-wide PBIS, collected during the 2006/7 and 2007/8 school years. For the initial PBIS training analyses, 810 traditional elementary schools across 23 districts in the state were eligible for inclusion. In this particular study seventeen districts (i.e. clusters) were included in the adoption and implementation analyses. All the traditional elementary schools in these seventeen districts (764 schools) were included in the adoption analyses. An examination of adoption among the trained schools only was also undertaken, including the subset of 298 trained schools across the seventeen districts. Lastly, the 227 schools (76 per cent of the trained sample) in the seventeen districts that submitted implementation were included in the implementation analyses.

The implementation of PBIS in Maryland comprises four phases. Phase one is creating rea-diness, by holding a conference to carefully review state data on school safety and to review and discuss different prevention models that might be a good fit for the state’s priorities and resources. Phase two is the initial implementation. The Maryland PBIS initiative has developed a relatively extensive multilevel infrastructure or “support system” to promote dissemination of the PBIS model. The consortium includes various stakeholders (educators, practitioners, researchers, policymakers) who jointly coordinate, train, and support schools and districts in the implementation of school-wide PBIS.

All 24 Maryland school districts collaborate with the state to provide the initial two-day sum-mer PBIS training, annual one- or two-day regional booster training sessions, and ongoing support for schools and implementation coaches. Much of the state- and district-coordinated training occurs for a core set of PBIS team members (typically four to six members attend the training events, including an administrator). It is a requirement in Maryland that each school has a building-specific coach, who, together with the district leadership, provides technical assistance, aids in problem-solving, and maintains momentum and enthusiasm for the programme within the school. It is advantageous if the coach is a local expert and has prior experience with the programme. The role of coach was originally conceptualised as someone external to the school (so that school psychologists or counsellors might coach a school they were not otherwise assigned to), although this external model proved difficult to sustain when districts went to scale. Most PBIS schools now have internal coaches.

Some districts have written PBIS coaching responsibilities into the school psychologists’

job descriptions. The coaches and core members of the PBIS team attending the district and state training events then lead the training of the other school staff back at the school. This

training model, whereby the core PBIS team trains the rest of the school staff, is one element of PBIS that has facilitated the state’s ability to rapidly scale the model.

Phase three is institutionalisation, and here the implementation of PBIS was initially financed, managed, and led by the state team, including funding, coordination, and staffing by the Maryland State Department of Education and in-kind staffing, coordination, and evaluation support provided by Sheppard Pratt Health System. Johns Hopkins University became a partner in 2001 and has provided in-kind training, coordination, and evaluation support.

The national PBIS technical assistance centre also provided funding for a full-time staff member to help coordinate the PBIS Maryland initiative. However, as the initiative expan-ded to include more than half the schools in the state, it was no longer feasible to maintain centralised leadership for the effort at state level; the 24 local school districts therefore took on considerably greater responsibility for sustaining previously trained schools, while the state-level team focused on expansion to new schools and to more advanced tiers. This type of multilevel leadership and coordination structure is considered a key component of sca-ling-up efforts. Thus approximately three to five years into the Maryland initiative, there was an intentional shift towards creating greater capacity, coordination, and resource allocation at the district level. Each district is now required to have some type of district-level PBIS coordinator, although there is considerable variation in the amount of full-time effort that this person specifically dedicates to PBIS. The coordinator provides local leadership for the PBIS effort, participates on the PBIS state leadership team, and coordinates local and state PBIS training events. Many of the districts now have their own PBIS leadership team, a budget for PBIS, and other resources allocated for local support of the initiative.

Phase four was an ongoing evolution and renewal. A critical element of the PBIS Maryland initiative was developing and maintaining a comprehensive data system to monitor and evaluate PBIS statewide. There has been a focus on monitoring both implementation fidelity and student and staff outcomes. The National PBIS Technical Assistance Center and other researchers have created a series of validated, research-based measures of PBIS fidelity which are freely available for use by schools, districts, and states.

The conclusion of the study was that the model appeared to be effective, as evidenced by the high concentration – over half – of schools trained in PBIS throughout the state. It is important to note, however, that the process was not linear. There were often times when the collaboration looped back to an earlier stage to integrate with emerging concerns, programmes, and priorities. Similarly, the evaluation and monitoring activities were ongoing, and played an important role in all phases of the implementation process. As

hypothesised, schools with greater need were more likely to receive training. Specifically, higher rates of suspensions as well as mobility were positively associated with training, while a higher rate of academic achievement was inversely associated with training. Accor-ding to the authors, this suggests that the lower-performing schools were more likely than other schools to access PBIS.

It is important to note that schools in Maryland self-identify for PBIS training, and thus it appears that many of the lower-performing schools were seeking training as a way of im-proving the school. At the district level, the number of schools (or district size) was inversely related with training, such that schools within larger districts were less likely to be trained.

The finding that training and adoption were associated with district-level predictors is con-sistent with literature suggesting that the district has the most influence in determining the involvement of schools in a particular initiative.

As hypothesised, the number of years since a school had first received training in PBIS was positively associated with implementation, as measured by all three IPI scales. Similarly, the concentration of qualified teachers also predicted implementation quality, suggesting that the better prepared and coached the teachers, the more effective the programme imple-mentation. It is also stressed that support given by practising school psychologists and other professionals within the school should be ongoing, since implementation takes time.

The study by Becker et al. (2013) explores the association between a two-phase coaching model and the implementation of the PAX Good Behaviour Game (PAX GBG) by elementary school teachers in a large urban school district. The two main goals of the paper are (1) to examine how coaches tailor their practices according to teacher implementation quality, and (2) whether coaching is associated with improved implementation quality.

The PAX Good Behaviour Game functions as a group-based token economy in which groups are rewarded for their collective success in preventing aggressive/disruptive and of-task behaviours. In addition, verbal and visual cues are used to promote attentive and prosocial behaviours and a positive classroom environment. The implementation of the PAX Good Behaviour Game unfolds over the course of an entire academic year of 31 weeks. In connection with this process, a two-phase coaching model is employed to support teacher implementation. This model includes a one-day training workshop, followed by two phases of coaching: a universal coaching phase lasting four to six weeks, in which coaches use the same strategies with all teachers, and a tailored coaching phase, during which coaches apply

an adaptive approach tailored to fit the needs of individual teachers. Ongoing coaching efforts take place throughout the implementation period, with intensity and content varying after phase one according to the needs of each teacher. Coaching is carried out by three former school teachers, who are employed by the research team and given intensive training and supervision.

The authors use quantitative measurements based on three data sources: coach logs, teacher logs, and observations of teacher implementation quality. Observations are completed by independent observers using the PAX Good Behaviour Game implementation rubric at four time-points spaced over the academic year. The implementation rubric includes items on seven dimensions reflecting core components of the intervention, including preparation of students, choice of activity, and use of timer, and observers rate teachers on each dimension using a five-point scale. These ratings are then averaged in order to build a mean imple-mentation rating score for each teacher. Rubric scores for the autumn semester (round one) serve as an initial measure of implementation quality, while final rubric scores occurring in May (round four) are used as an outcome variable reflecting teacher implementation proficiency following coaching. Additionally, rubric scores are used to categorise teachers into two groups: low-quality and high-quality implementers.

Overall, study findings suggest that the two-phase coaching model is associated with improved implementation quality of the PAX Good Behaviour Game. This association highlights the importance of coaching as a support system for securing implementation quality. Out of a total of 129 participating teachers, 55 per cent were categorised as high-qu-ality implementers based on round-one rubric scores, while 45 per cent were categorised as low-quality implementers. As round-one scores were from a time-point at which teachers had received only one to two days of training and about a month of coaching, this suggests that models like the PAX one can be implemented with good quality by many teachers after a relatively modest level of training and coaching.

Furthermore, results show that round-four implementation scores were significantly higher than round-one scores, meaning that teacher implementation quality improved over the course of the year. Low-quality implementers showed the greatest improvements, but did not reach the same rubric scores in round four as teachers who started out as high-quality implementers. This is taken to indicate that assessments of teacher implementation made early in the process are to an extent predictive of future implementation quality, and can therefore be useful in determining which teachers are in need of additional support.