Gérard Bonnet
Head of European and International cooperation,
Direction de l’évaluation et de la prospective, Ministère de la jeunesse, de l’éducation nationale et de la recherche, France & Visiting Fellow at the Institute of Education, University of London Using the concepts defined for the Anthology, and leaving aside the question of the informal assessment of pupils by teachers, the question of examinations, as well as the question of indi-vidual inspection of teachers, which do not fall within the remit of this survey, evaluation of school education in France can be described as applying to several areas, e.g. schools and pu-pils, teaching practices. It covers the school education system as a whole, at national, regional and local levels. Evaluation methods and results are also assessed (meta-evaluation). The value and purpose of evaluation are well defined and understood at a conceptual level, but their practical use is an area of concern.
Evaluation procedures are carried out by three different institutions within the school education sector. Two of these are responsible for actually organising and conducting evaluations (the inspectorate and the education ministry’s evaluation division), while the third reviews evalua-tion findings and methodology (the high council for evaluaevalua-tion).
At a national level, the general inspectors, based in the education ministry, participate in the supervision, training and recruitment of some types of teachers. They are also more generally involved in evaluating the school system, either at the national or regional level. For instance, they are called upon to give their assessment of the overall way in which an education region is performing in terms of different focus areas, e.g. its management processes, its pedagogical provisions, the results of its pupils, etc.
Counselling or assistance for schools is mainly the responsibility of regional inspectors, a group whose main task is to inspect, supervise and assess teaching staff. These inspectors have a re-gional remit but retain close links with the general inspectors at the education ministry. There are 3,000 regional inspectors, a figure which should be compared with the 880,000 teachers in primary and secondary education (an inspector in the primary sector supervises an average of 300 teachers, while in the secondary sector, inspectors are each responsible for about 750 teachers).
The evaluation division of the education ministry (direction de l’évaluation et de la prospective) is in charge of the coordination of the evaluation and forecasting functions of the ministry. This division has overall responsibility for the production and analysis of both educational statistics and qualitative and outcome studies. This implies close collaboration with other divisions of the ministry, the academies and other actors in the field, including numerous institutions and re-search teams. Its remit exceeds what is discussed here, as it is in charge of aspects of evaluation for the higher education and research sectors, in particular with respect to the production of certain statistics and indicators.
The high council for evaluation was set up by the education minister in 2002. Although its members are formally appointed by the education minister, its work is, in practice, independent from ministerial interference. Its remit is to assess evaluation procedures and methodologies, not the education system. To do this, it reviews the different assessments of the education system carried out and produces an annual report in which it makes recommendations concerning the state of educational evaluation. It comments on the assessments carried out and published by the education ministry and by public or private bodies. It may also
commission specific assessments from public or private organisations in areas where evaluation
Educational Evaluation around the World 54
specific assessments from public or private organisations in areas where evaluation may be deficient.
Values and purposes
The status of evaluation was clarified in the 1989 Education Reform Act, which states that evaluation is an integral part of the education system. It is mandatory for educational institu-tions to take part in surveys organised by the ministry with a view to providing statistics, organ-ising national testing, etc. Regional or local evaluation initiatives are also prescribed for the school education sector. However, the Act does not define the term “evaluation” which is seen in the French context as a blanket term covering what might also be described in other coun-tries as quality assurance or quality development.
At first, educational evaluation was mostly quantitative, i.e. derived from basic statistics, such as teacher-pupil ratio, average spending, etc. In the 1960’s the education ministry began set-ting up a statistical information system which became increasingly sophisticated and compre-hensive over the years. Nowadays this system encompasses all school education statistics. From a technical point of view, it makes use of up-to-date technology and relies on a unified defini-tion of statistical concepts so that data are comparable throughout the country. One of its characteristics is that it is based on the data collected for management purposes at all levels of the education system and enables the construction of statistics which can be used for general monitoring and steering purposes.
Later, assessment and evaluation surveys were introduced to give some account of the quality of education. The combination of a powerful statistical information system and of quality as-sessment has created an evaluation system which describes the state of the education system with rigour and accuracy and allows for the construction of indicators. This concept of evalua-tion is rather broad since it encompasses all the data available to measure and assess the sys-tem. It is used as a tool to help with the management of the system but also as a way of mak-ing stakeholders aware of the necessary changes in their professional practices.
Evaluation is also a major tool in the current context of greater power and responsibility being transferred from the centre towards the regional and local levels, since this creates the need for a strong monitoring and steering system.
For well over a century, the French education system was organised centrally (nationally). The education system consists of three main levels of responsibility: (i) the national education minis-try, (ii) regional education administration (deceptively known as “académie”), which also de-volves responsibility for primary education to the district level, and (iii) the school. An OECD study conducted a few years ago in France showed that each of these levels made roughly one third of the decisions.
In addition, elected regional, district and communal assemblies have specific responsibilities for lower and upper secondary schools (collèges and lycées) and primary schools respectively, mostly in the areas of building maintenance and construction, as well as the employment of some ancillary staff.
Central government retains, at each of the three main levels, overall responsibility for teaching staff and all pedagogical aspects of education, even if some of it is devolved to the lower ad-ministrative levels. In other words, the crucial, if subtle, difference is between political transfer of responsibility from national government to local elected assemblies, and executive devolu-tion of parts of the power of central government (the educadevolu-tion ministry in this case) to na-tional government representatives at regional, district and communal level. It should be appar-ent from the above description that the real power still remains with cappar-entral governmappar-ent through its local representatives, with local political bodies enjoying only partial responsibilities.
The question of devolution of power to regional and local levels has been one of the main or-ganisational concerns of administrative reform in education in France over the past couple of
Educational Evaluation around the World 55
decades. One of the main thrusts has been the transfer of more and more responsibility from the ministry to regional administration (académie). Although it is clearly understood that educa-tion in France will remain “naeduca-tional”, it has also become apparent that more initiative has to be entrusted to the académies. The head of each académie (there are 30 in all, including overseas territories) is appointed by the education minister and is his representative in each region (this must not be confused with the notion of a locally elected politician responsible for education, which does not exist in France).
In order to facilitate this process, indicators are needed to allow central government to allocate resources equitably throughout the country and to allow the académie level to do the same within a given region.
At the regional level, education indicators are also used to make decisions on employment policies (which are partly the responsibility of the regionally elected politicians) based on the output of the education system (number of school graduates, including vocational training, number of university graduates; qualitative information on the qualifications; etc.). The regional level may also establish its own evaluation procedures in order to make decisions and shape policies.
Next to this role in monitoring and steering the education system, evaluation has other pur-poses. In a country where it is official policy that education should be of comparable standards wherever you happen to be, it is necessary to measure quality, and evaluation fulfils this role in terms of quality assurance.
However, evaluation also has a quality development component, and it is also used as a tool for change. As such, the emphasis is not so much on the summative aspect as on quality develop-ment. The French evaluation system includes procedures to encourage educational change and to promote the dissemination of a culture of evaluation among stakeholders throughout the system. This is based on the belief that such stakeholders, among whom teachers are promi-nent, will improve their professional practices only if they are shown, as in a mirror, the conse-quences of their actions. For instance, one form of pupil assessment is conducted in such a way as to involve teachers and schools. It is an educational tool used to initiate changes in profes-sional practices. Other tools are also provided to schools to improve their managerial and pedagogical practices. These mechanisms will be fully discussed below.
Evaluation is understood to encompass all these different aspects. The concepts of quality as-surance and quality development are closely intertwined within the concept of evaluation. At least this is how it is understood by the vast majority of stakeholders.
The greater part of evaluation initiatives comes from the ministry, where a specific division was set up for the purpose. The lower levels are under an obligation to participate in the processes put in place. They are also encouraged to have their own arrangements. For instance schools are encouraged to have their own assessment tools to monitor their progress and to make use of them to devise their compulsory “school plans”.
Objects
As explained above evaluation in France is meant to cover the whole of the school sector, at all levels and in all its components. It embraces both the public and the private sector, at least that part of the private sector which is government funded (i.e. 90% of the private sector).
All types of evaluation can be found: pupils’ achievements and attitudes, institutions, pro-grammes, subjects, themes, professional practices, etc. There are specific arrangements in place for meta-evaluation, which is mostly conducted by the high council for evaluation.
Institutions are subject to evaluation in one-way or another. For instance, schools are assessed from the point of view of their administrative and financial policies but also from the point of view of their performance, in particular of their ability to cater for all types of pupils and bring
Educational Evaluation around the World 56
them to the end of schooling, and of their examinations results in the context of their socio-economic environment. Even whole administrative areas, such as an académie, can be subject to evaluation in term of their educational provisions or, sometimes, even their results (percent-age of early school leavers, of school graduates, etc.).
Pupils’ achievements are assessed nationally through various testing procedures, either to iden-tify their strengths and weaknesses from a teaching perspective or to give a picture of national or regional performance. The results of the development of academic subjects in schools, na-tional policies and innovations are evaluated (for instance the implementation of reforms such as the introduction of foreign language instruction at primary school or of team work for teachers). Themes like literacy, numeracy, ICT policies, provisions and results are looked at.
Sensitive issues, such as early school leavers, or violence in schools, are subject to scrutiny. Lon-gitudinal studies are used to understand specific mechanisms or effects of schooling (selection, pupil counselling, dropping out, etc.).
Teachers’ professional practices and motivations are investigated through questionnaires or classroom observation of representative samples (this is distinct from individual inspection which also exists).
Whenever a specific type of evaluation is decided upon at the national level, the subject to be assessed is always very clearly identified and defined after consultation with the relevant bodies interested in the results of the work. When evaluation is decided at regional or local level, there is more flexibility so as to tailor the study to specific needs, which may be different from those of the national level. Likewise, the methods used locally may differ from national ones (except for the collection of statistical data, for which specifications are national) and may produce results that cannot be compared outside the area where they were applied.
Overall it is fair to say that the assessment of pupils’ achievements and skills is one of the pre-dominant types of evaluation in France. Pupils are consistently targeted through the whole of school education. The reason for this is that evaluation is largely aimed at providing help for teaching and learning. Next to them, schools and educational administration units are the sec-ond preferred evaluation targets. This is because is it felt that there is plenty of room for im-provement in the way they operate; and their importance in the process of delivering education is also becoming more and more recognised in a country were there is not much in the way of a tradition for school effectiveness.
In most cases the evaluation subject is chosen on the basis of its relevance to policy and to national or local priorities, due to the need to be very selective when deciding what to evaluate.
Evaluation is costly in terms of resources and is an added burden on whoever happens to be the subject. In order to be accepted and properly carried out, it needs to be perceived as serv-ing a specific purpose. Hence, besides the need to obtain information on issues under scrutiny, there is an attempt, as far as possible, to target areas which are directly connected to local and national policy imperatives.
Stakeholders
Because evaluation in France covers such a large domain and variety of purposes, it is actually aimed at a whole range of stakeholders, according to the various aspects of education being dealt with.
Evaluation data are available for policy makers (Parliament, ministers, etc.), for educational administrators at local, regional and national level, and for all the stakeholders of the system (parents, teachers, etc.). At the same time, these data are also made available to the press and the general public in order to facilitate discussion about education as part of the normal de-mocratic process. With the creation of the high council for evaluation to be occupied with meta-evaluation, it should be clear that evaluators themselves are concerned with the assess-ment of their work. The following are examples of the way specific evaluation is aimed at a particular audience and of the impact on that audience.
Educational Evaluation around the World 57
Diagnostic national testing (which currently exist in three grades) is meant as a tool for teachers to assess the competence of their own pupils in specific subjects. In order to help them place their pupils in relation to others, the results of a representative sample are analysed by the min-istry and then fed back to all the teachers, together with pedagogical comments, so that they can compare the ability of their pupils with the national or regional average pupil in the same grade. This is done in printed and electronic form. It is complemented with a bank of assess-ment items which teachers can use at will for further diagnosis and which is freely available on the Internet.
In the past the diagnostic testing programme has had positive results on teachers. For instance, following the 1989 primary school mathematics tests, the national sample showed a bias in favour of arithmetic as opposed to geometry even though both subjects were given equal weight in the national curriculum. As a result, when teachers realised that this was a general problem they altered the structure of their teaching, and it was observed the following year that competence in geometry had greatly improved without any outside interference.
At the same time, it has also been observed that teachers, particularly at the secondary school level, make insufficient use of the diagnostic testing programme in their teaching. In fact one particular test for upper secondary school had to be discontinued after a few years because not enough teachers used it.
School indicators have been constructed at ministerial level as tools for head teachers to man-age their schools. A procedure similar to that described above enables them to compare their processes and results with those of other comparable schools. Three indicators (performance indicators based on national final examination results) are published each year (and made avail-able on the Internet) to give some idea about the value added to upper secondary schools.
A set of regional indicators is published each year by the ministry to show how académies compare. Although this publication is national and widely available, it is also intended to help the recteurs (the civil servants who represent the minister in the region) and their staff, as well as regional politicians, to formulate relevant policies.
A similar set of national indicators is published annually and is widely used by national policy makers, administrators, parliamentarians and the press to discuss the state of the education system, and even as a basis for drafting national reforms and policies. These national indicators are also one of the elements which the Treasury takes into account in deciding on the share of the national budget that will be allocated to the education ministry.
It is fair to say that the educational world and French society at large have a clear understand-ing of the aims and audience for each type of evaluation. The resultunderstand-ing data are eagerly awaited and accepted as useful information by those concerned, in particular the press. Conse-quently, there is no debate about the way existing evaluation is conducted. What is currently being discussed, in particular through the reports and recommendations of the high council for evaluation, is the extent to which evaluation findings are sufficiently used by policy makers, and what new subjects should be targeted, e.g. teachers, in particular.
It is fair to say that the educational world and French society at large have a clear understand-ing of the aims and audience for each type of evaluation. The resultunderstand-ing data are eagerly awaited and accepted as useful information by those concerned, in particular the press. Conse-quently, there is no debate about the way existing evaluation is conducted. What is currently being discussed, in particular through the reports and recommendations of the high council for evaluation, is the extent to which evaluation findings are sufficiently used by policy makers, and what new subjects should be targeted, e.g. teachers, in particular.