Jacques L’Écuyer Chairman Dany Rondeau Project coordinator
Commission d’évaluation de l’enseignement collégial du Québec (CEEC)
For the past ten to fifteen years in Canada, evaluation of universities’ and colleges’ educational programs has been at the center of continuous debates. In various sectors of society, there have been calls for better accountability, more information on the quality and effectiveness of the institutions, performance measurements, quality assurance and monitoring bodies. At the source of these debates, one finds in particular the increasing cost of higher education, its competitiveness in the context of globalization, changes in the needs and behaviour of stu-dents and the necessity to increase accessibility without lowering quality.
This context has lead to major changes in the higher education policies and regulation mecha-nisms in many parts of Canada. Governments have reinforced the need for clear and measur-able objectives and effective performance and tracking reporting systems. Alberta and British Columbia legislation on higher education is currently being thoroughly revised. Ontario has recently developed key performance indicators and established a Post-secondary Education Quality Assessment Board. Quebec’s universities have had to sign performance contracts with the Ministry of Education.
The present paper attempts to present the situation of the evaluation of higher education in Canada as it is in 2003. It describes the quality assurance mechanisms in the various provinces and analyses their main characteristics. Recent changes have been taken into account.
Overview of Higher Education in Canada
Canada is a federation of ten provinces and three territories. It has two official languages, Eng-lish and French. Education falls under the jurisdiction of the provinces. Each province and terri-tory has accordingly established its own system of education with its particular structure and regulation mechanisms. Differences between each system can be quite important. In Quebec for instance, the primary and secondary schooling lasts 11 years as compared with 12 in other provinces. But on leaving secondary school, the Quebec student must first receive a two-year college education before entering the university. In other provinces, this obligation does not exist.
The basic structures of the education systems are however quite similar, and the first post-secondary degree, the baccalaureate, is normally obtained after 16 years of schooling. Degrees are usually granted by universities or university-colleges. Colleges of all types, institutes and grant diplomas, generally after two or three years of post-secondary schooling, and certificates or attestations in the case of shorter programs.
Each province has its own quality assurance system and has developed a series of measures to ensure the quality of its post-secondary education system. The legislation has been used exten-sively to protect the credibility of the degrees granted by institutions and to control the devel-opment of the system in a context where the number of would be providers is increasing. In most provinces, the power to grant degree is restricted to institutions formally recognized ac-cording to the legislation and regulations in force. Even more importantly, in nearly all prov-inces, new program proposals and important changes in programs must be approved by the
Educational Evaluation around the World 86
Ministry of Education or by a mandated intermediate regulatory body. In fact, presently in Can-ada, nearly all new degree programs are submitted to some kind of review before being of-fered.
New Program Proposals:
The procedure used to review new program proposals varies according to the province and the type of institution. In Alberta, all new program proposals coming from public institutions, uni-versities or colleges are reviewed by the department of education. The department focuses on how the program reflects the mandate of the institution and balances with other programs in the region. In the case of private institutions, degree program proposals are reviewed in depth and, if found adequate, are accredited for a period of time, normally 6 or 7 years. This accredi-tation is granted by The Private Colleges Accrediaccredi-tation Board (PCAB), an agency set up by the Government. Ontario and British Columbia have procedures which are quite similar. The partment of education reviews all new programs for duplication. When the proposal of a de-gree program originates from an institution which is not already authorized to offer dede-gree programs, it is reviewed by an advisory body, the Degree Quality Assessment Board (DQAB) in British Columbia and the Post-secondary Education Quality Assessment Board (PEQAB) in On-tario. These agencies have been established by the Government.
In Manitoba, all new university and college programs must be approved by the Council on Post-secondary Education (COPSE). Programs are reviewed from quality, organizational and financial perspectives. In the Maritime Provinces (New Brunswick, Nova Scotia and Prince Edward Island), new university programs are reviewed by the Maritime Provinces Higher Education Commission (MPHEC) to ascertain their suitability given their objectives, structure, resources and relevance.
New college programs must be approved by the education department in New Brunswick and Nova Scotia. Both COPSE and MPHEC are government agencies.
In Quebec, new university programs are reviewed from a quality perspective by the
« Commission d’évaluation des projets de programmes (CEP) » under the aegis of the
« Conférence des recteurs et des principaux des universités du Québec (CREPUQ) » and their relevance is assessed for financing by the Ministry of Education. New college diploma programs are approved by the Ministry of Education.
There are no external reviews of new programs in Saskatchewan, Newfoundland and the Terri-tories.
Provincial Continuous Quality Assurance Practices:
Apart from new program assessments, some provinces have continuous quality assurance mechanisms. Such mechanisms exist in Alberta for the private institutions, as the Private Col-lege Accreditation Board must periodically review the operations of the private colCol-leges with accredited programs. A similar provision exists for the British Columbia and Ontario quality assessment boards, DQAB and PEQAB, but it has not yet been tested, as these boards are still very young.
At the university level, Ontario, Quebec and the Maritime Provinces have in operation mecha-nisms to audit the internal quality processes used by universities for the assessment of their existing undergraduate programs. In Ontario, this is done on a voluntary basis by the Under-graduate Program Review Audit Committee (UPRAC), under the aegis of the Ontario Council of Academic Vice-presidents. In Quebec, each university is required to have a periodic program assessment policy. These periodic assessment processes are reviewed by the « Commission de verification de l’évaluation des programmes (CVEP) » under the aegis of the Conference of Rec-tors and Principals (CREPUQ). Finally, in the Maritime Provinces, the Maritime Provinces Higher Education Commission (MPHEC) has the mandate to monitor the universities’ quality assurance mechanisms.
Educational Evaluation around the World 87
Ontario is the only Canadian province that has an external quality assessment agency at gradu-ate level. In this province, gradugradu-ate programs offered or proposed by public universities are appraised on a voluntary basis by the Ontario Council on Graduate Studies (OCGS), under the aegis of the Council of Ontario Universities.
At the college level, Quebec is the only province that has an extensive quality assurance system.
In this province, the Commission d’évaluation de l’enseignement collegial (CEEC), a government agency, has the mandate to assess programs as well as public and private institutions.
Professional Accreditation:
At the provincial and national levels, some professional programs are accredited by professional accreditation agencies. In some fields, for instance health sciences, engineering, etc., it is re-quired that applicants for professional registration have graduated from an accredited program.
These accreditations are valid for a certain period. Hence the accredited programs are periodi-cally reviewed according to standards and procedures that are frequently modelled on those used by similar organizations in other countries. The Association of Accrediting Agencies of Canada (AAAC) counts 21 such members.
Other Mechanisms:
Other mechanisms are in place to ensure quality in Canadian post-secondary institutions. For instance, the Association of Universities and Colleges of Canada (AUCC) and the Association of Canadian Community Colleges (ACCC) do not accredit or evaluate programs or institutions.
However, they require of their members that they deliver high quality programs consistent with the academic standards set out by appropriate jurisdictional authorities. As such they do give some assurance of quality.
Finally, it must be added that a number of ministries of education have established a series of performance indicators. These indicators are published and, in some cases, used for adjusting the level of public funding. Ontario is a leader in this trend. It has launched a Key Performance Indicators Project to account for the performance of the higher education institutions of this province. Employment rates, graduate satisfaction, student satisfaction, employer satisfaction and graduation rates are published on a yearly basis.
The previous overview shows how diverse are the regulatory systems in Canada. Except for a few cases, it is, therefore, not possible to speak of a Canadian higher education quality assur-ance system. Nevertheless, a few trends or characteristics emerge from this diversity when we try to answer the questions why (values and purposes), what (objects), for whom (stakeholders) and how (method) evaluation is done.
Values and purposes
In Canada, the reasons for doing evaluation vary according to the agencies. Some do it for control reasons, some for quality assurance, some for quality improvement. The distinctions are not always clear between these reasons and one has to look at the objectives and practices of the agencies to determine into which category they fall. For the purpose of this article, we will classify under “control” those evaluations that are done to verify if a program should be of-fered or if an institution should operate. Quality assurance, on the other hand, means an evaluation that is done to determine if a program or an institution meets predetermined stan-dards. Finally, quality improvement is an evaluation done for the explicit purpose of improving the quality of a program or an institution. This last type of evaluation usually has the following characteristics: it implies that the agency resorts to peers, that it makes recommendations to the institutions themselves and that the evaluation has some kind of follow-up either directly or through periodic evaluations.
Educational Evaluation around the World 88
Control
In Canada, evaluation is quite often done with the purpose of controlling the development of the higher education system and assuring its quality. As was pointed out, in the major prov-inces, new degree programs are reviewed in one way or another by government-controlled agencies with a view to:
avoiding duplication;
making sure that they are relevant given the needs of the society;
making sure that they reflect the mandate of the provider;
making sure that they meet generally agreed standards.
The first three reasons are given in nearly all provinces where evaluation of new programs is done. They are obviously reasons of control coming from the fact that in Canada, the govern-ments heavily subsidize higher education and want to avoid wasting money. They apply mainly, but not exclusively, to public institutions, as even private institutions get government funding in some provinces.
The fourth reason is of a different nature and, in many provinces, concerns only colleges or private institutions that wish to offer a degree granting program but are not recognized as a degree granting institution by a charter or by the provincial legislation. In such cases, permis-sion to offer the program will not be given unless it has been evaluated and shown to be of good quality. Consumer protection and the defense of the credibility of the national and pro-vincial degrees (Baccalaureate, Master, Doctorate) are the main reasons for requiring this evaluation which is, at the same time, a quality assurance and control measure.
Improvement
Evaluation is carried out for the improvement of quality in five of the ten provinces: Ontario, Quebec and the three Maritimes Provinces. In these provinces, an agency has the mandate to periodically audit the internal quality assurance process of each public university. In Ontario, this is done by the Undergraduate Program Review Audit Committee (UPRAC), in Quebec by the
« Commission de vérification de l’évaluation des programmes (CVEP) » and in the Maritimes, by the Maritimes Provinces Higher Education Committee (MPHEC). In all three cases, the audit is periodic, formative and the recommendations of the agency are directed to the institutions themselves with a view to providing assistance and advice.
Program evaluation by the Ontario Council on Graduate Study (OCGS) is also done in the per-spective of quality improvement. It is periodic, and recommendations are made to the institu-tions themselves with a view to promoting best practices in graduate education.
The work of the « Commission d’évaluation de l’enseignement collégial (CEEC) » of Quebec will be reviewed in the case chapter. As will be seen, it is done with the explicit purpose of improv-ing the quality of Quebec college education.
Quality Assurance
The evaluations realized by the professional Boards for the purpose of program accreditation usually contain a dimension of improvement superimposed on specific quality assurance objec-tives. These evaluations are indeed done to verify that programs meet the criteria of the agency and, as such, fall into the category of quality assurance mechanisms. However, these evalua-tions are done on a periodic basis, and the criteria often contain particular exigencies related to continuous quality improvement. Furthermore, the criteria are themselves periodically reviewed to reflect the state of the art, and, finally, recommendations are addressed to the institutions.
All that contributes to the continuous improvement of the accredited programs.
Educational Evaluation around the World 89
It is interesting to realize that whenever the evaluation is done by a government owned or con-trolled agency, it is usually done as part of a government policy, and, for this reason, institu-tions have to submit themselves to the process. In the other cases, Ontario’s UPRAC and OCGS
and Quebec’s CVEP, owned by their respective Conference of Rectors, the process is voluntary, but all members of these Conferences have accepted to comply with it. As for the evaluations done by the professional Boards, they are usually a near obligation, since, otherwise, the graduates from these programs would have difficulties in registering in their profession.
Objects
Generally speaking, Canadian evaluation agencies evaluate programs, not institutions. This is because the authority to grant academic credentials is given by governments through charters or legislation and reserved for institutions with adequate mechanisms for assuring institutional and program quality. These institutions, mostly public, are subsidized and supervised by their respective governments and, therefore, are not subjected to institutional accreditation. There are, however, a few exceptions to this situation. Alberta, British Columbia, Ontario and New Brunswick have opened the field of higher education to private institutions that meet the stan-dards of quality for degree granting institutions. These private institutions are submitted to a periodic institutional evaluation to make sure that these standards are maintained.
As previously mentioned, public universities in Ontario, Quebec and the Maritimes Provinces are audited by an external agency. This audit concerns their quality assurance mechanisms. It is formative and is realized to provide assistance to the universities and, at least in the Quebec case, to testify to the efficiency of their internal quality assurance practices.
At the college level, institutions and programs are usually « registered » (or « licensed ») by governments. This registration is done for consumer protection only. Institution or program quality is not evaluated. There is, however, a major exception to this situation, namely, Quebec where institutional as well as program quality is extensively evaluated by the « Commission d’évaluation de l’enseignement collégial (CEEC) ». This evaluation is done primarily for quality improvement, but the Commission can recommend that a program or an institution be closed if it is of a poor quality.
In all other cases, agencies evaluate programs, not institutions. Agencies like PCAB in Alberta,
DQAB in British Columbia, COPSE in Manitoba, PEQAB in Ontario, CEP in Quebec and MPHEC in the Maritimes Provinces evaluate projects of new programs to assess their quality and suitability.
Such aspects as the relevance of their objectives and content, the coherence of the curriculum and the adequateness of the resources are assessed.
Professional programs are evaluated periodically by the professional accrediting agencies. This evaluation is realized to ensure that these programs comply with the standards of the agency.
For instance, the Canadian Engineering Accreditation Board declares in its documentation that accredited programs « will meet or exceed minimum educational standards acceptable for pro-fessional engineering registration in Canada ». These standards cover a wide range of ques-tions: governance, program objectives and content, pedagogical methods, student assessment, student support, student success, resources, etc.
The Ontario Council on Graduate Studies (OCGS) periodically evaluates graduate programs. This evaluation is for improvement of the graduate education. Criteria depend on the type of pro-grams – master, doctorate, professional – and cover aspects such as objectives, content, breadth, research component, analytical skills development, etc.
Educational Evaluation around the World 90
Stakeholders
Accrediting agencies and evaluation bodies consider that the evaluation may benefit many stakeholders. There are the universities, which receive support during the elaboration of a new program; the colleges, which gain experience in evaluation and improve the quality of their programs; members of accrediting agencies; the students, who benefit from a high quality education; and the public, the consumers, the clients and the employers, who can be assured of the competences of graduates and professionals.
However in any evaluation, there are always stakeholders who benefit more directly than oth-ers from the evaluation process. In the case of the evaluation of proposals of new programs, governments appear to be the major beneficiary since the result of the evaluation is used to determine if a program should be offered and funded. It is clear, however, that the consumers, who are protected from receiving an education of poor quality, also benefit from this evalua-tion. The institutions themselves greatly gain from the comments of the review team.
When the evaluation is done for the purpose of a professional accreditation, the professions themselves are the main beneficiaries. The professional agencies evaluate the programs accord-ing to their own criteria, which are certainly in accordance with the needs or the objectives of the profession, but do not necessarily take into account the general context and the resources of the institution. Obviously the society benefits from better-trained professionals, but in the absence of public discussions there is always the danger that the accreditation criteria reflect more the interest of the profession than the true needs of the society. Nevertheless, profes-sional accreditation can be, and usually is, a source of important benefits for the society and the institutions, as well as for the profession.
When quality improvement is the main objective of the agency, as is the case for the Maritimes Provinces Higher Education Commission (MPHEC), the Quebec « Commission d’évaluation de
When quality improvement is the main objective of the agency, as is the case for the Maritimes Provinces Higher Education Commission (MPHEC), the Quebec « Commission d’évaluation de