Government Gazette

Vol. 407, No. 20050, 7 May 1999

GOVERNMENT NOTICE

Notice 595 of 1999

DEPARTMENT OF EDUCATION

NATIONAL EDUCATION POLICY ACT, 1996 (ACT NO. 27 OF 1996)

PUBLICATION OF THE REPORT OF THE MINISTERIAL COMMITTEE ON THE INVESTIGATION INTO THE SENIOR CERTIFICATE EXAMINATION FOR PUBLIC INFORMATION

In view of the poor examination results in the 1997 Senior Certificate Examination severe criticism was received from several sectors of the South African community.

In response to these criticisms I instituted several initiatives to address the problems. One such initiative was the appointment of the Ministerial Committee on the Senior Certificate Examination.

The terms of reference of the Committee was to address the following aspects related to the Senior Certificate Examination:

• Quality assurance of Senior Certificate assessment in terms of the South African Certification Council Act, 1986 (Act No. 85 of 1986);
• the capacity of the provincial departments of education to analyse and use the Senior Certificate results; and
• the examination and computer system currently used by the Provincial Examining Bodies with a view to determining the suitability, efficiency and effectiveness of the system.

Now, therefore, I Sibusiso Mandlenkosi Emmanuel Bengu, Minister of Education, hereby in terms of Section 3(4)(1) of the National Education Policy Act, 1996 (Act No. 2 7 of 1996), request any person, society or organisation to submit to me, before Friday 28 May 1999, comments on the afore-mentioned draft document.

Comments should be forwarded to:

REPORT

INVESTIGATION INTO THE SENIOR CERTIFICATE EXAMINATION
BY THE MINISTERIAL COMMITTEE

THE HONOURABLE MINISTER OF EDUCATION PROF S.M.E. BENGU

The committee has pleasure in submitting its report on the Senior Certificate Examination

Dr Morgan Naidoo (Chairperson)
Mr Louis Farrell (Member)
Dr Mabu Mateme (Member)
Ms. Meg Pahad (Member)
Prof Francois Steffens (Member)
Mr John Tyers (Member)
Ms Nan Yeld (Member)
Mr Jim Yeomans (Member)

ACKNOWLEDGEMENTS

Members of the committee express appreciation for their appointment to serve on the Ministerial Committee on the Senior Certificate Examinations. It is hoped that the results of the investigation will go a long way in contributing towards quality performance for both educators and learners.

Special thanks go to Heads of the Provincial Departments of Education, who made our visits to their departments very smooth. Not only did their heads of examinations and those of other sections welcome us with open arms, but they also assisted in bringing together identified units and stakeholders, who gave very generously of their time to have in depth discussions with Committee members. Our thanks also go to the following for their time:

They unreservedly gave advice, opinions and suggestions that we found invaluable.

We wish to express our sincere gratitude and appreciation to all the senior officials and their support staff at the national Department of Education.

CONTENTS

SECTION ONE

1.1 Preface
1.2 Glossary
1.3 Terms of Reference of Committee
1.4 Nature of Investigation
1.5 Limitations of the Investigation
1.6 Executive Summary

SECTION TWO

2.1 The Senior Certificate

2.1.1 Problems related to the current role of the Senior Certificate
2.1.2 Improving the relevance of the Senior Certificate using the critical outcomes
2.1.3 Improving overall achievement levels in the Senior Certificate by improving language teaching
2.1.4 Improving the relevance of the Senior Certificate by introducing coherent rules of combination for all Senior Certificate/FETC qualifications
2.1.5 Reviewing the utility of Higher and Standard Grade papers in the Senior Certificate Examination
2.1.6 Piloting a public examination at GETC level
2.1.7 A changing conception of 'recovery' and 'second chance' programmes
2.1.8 Piloting new FETC learning programmes and qualifications to supplement the current Senior Certificate subjects
2.1.9 The impact of reaming and teaching on the Senior Certificate Examinations
2.1.10 Consolidated Recommendations

2.2 Quality Assurance of the Senior Certificate Examination

2.2.1 The role of SAFCERT in assuring the quality of the Senior Certificate Examinations
2.2.2 Moderation of standards across bodies by moderating examination papers
2.2.3 Moderation of standards across bodies by using common questions/papers
2.2.4 The maintenance of consistent standards from year to year within each examining body
2.2.5 The question of second language as medium of instruction
2.2.6 Reporting the results of repeaters separately
2.2.7 The levying of charges for quality assurance
2.2.8 Lines of accountability and reporting for SAFCERT
2.2.9 The role of examination boards/ assessment agencies
2.2.10 Consolidated Recommendations

2.3 The Senior Certificate and Access to Post-Secondary Opportunities

2.3.1 The role of the Matriculation Board of the CUP
2.3.2 Access to Higher Education
2.3.3 The accreditation of state, private and foreign examination boards/ assessment 44 agencies offering qualifications at Senior Certificate level
2.3.4 Consolidated Recommendations

2.4 The Capacity of the Provincial Education Departments to analyse the Senior Certificate Results, and to use these to Enhance Teaching and Learning.

2.4.1 Criteria for an Effective Feedback Mechanism
2.4.2 Human Resource Issues

2.4.2.1 Personnel Vacancies
2.4.2.2 Working Relationships between Curriculum and Examination Sections
2.4.2.3 Understanding of Standardisation and Normalisation Procedures

2.4.3 Statistical Reports
2.4.4 In-School/Internal Assessment

2.4.4.1 Continuous Assessment
2.4.4.2 Orals and Practical Examinations

2.4.5 Examiners

2.4.5.1 Selection of Examiners
2.4.5.2 Training of Examiners
2.4.5.3 The Functions of Examiners

2.4.6 Examiners' Reports

2.4.6.1 The Format and Function of Examiners' Reports
2.4.6.2 Other Issues Arising from Examiners' Reports: the Examining of Languages

2.4.7 Examinations and Public Accountability: Harnessing the Media
2.4.8 Consolidated Recommendations
2.4.9 Summary of Visits to Provinces

2.5 Information Technology Systems Used for the Senior Certificate Examinations

2.5.1 Features of the Present Examination Computer System

2.5.1.1 Overview
2.5.1.2 Programme Changes
2.5.1.3 The Service Offered by Q-Data
2.5.1.4 The Help Desk
2.5.1.5 The Systems Administrator
2.5.1.6 Provincial Understanding of the System

2.5.2 Shortcomings of the Present System noted in Visits to Provinces

2.5.2.1 Documentation
2.5.2.2 Management Information
2.5.2.3 Duplication of Requests
2.5.2.4 Two Sub-Systems
2.5.2.5 Alternatives to the Existing System

2.5.3 Recommendations and Action Plan

2.5.3.1 Documentation
2.5.3.2 Management Information
2.5.3.3 Duplication of Requests
2.5.3.4 Central Source Code
2.5.3.5 Role of Q-Data
2.5.3.6 The Help Desk
2.5.3.7 Fiscal Policy
2.5.3.8 Impact of Present Examination System on Other Systems
2.5.3.9 Tender for Maintenance
2.5.3.10 Examination Fees
2.5.3.11 SAFCERT
2.5.3.12 Capacity Building in Provinces
2.5.3.13 Systems Administrators
2.5.3.14 Trainee Consultants
2.5.3.15 The History Database
2.5.3.16 SAQA and the IEB
2.5.3.17 Strategy to Retain IT Professionals
2.5.3.18 Colleges of Education Computer System
2.5.3.19 Year 2000 (Y2K)
2.5.3.20 ITC Meetings

2.5.4 Provincial Feedback: Summaries of Visits to Provinces

SECTION THREE

3.1 Synthesis and Major Observations
3.2 Recommendations and Framework for Action
3.3 Appendices

SECTION ONE

Examinations - and more particularly examination results - are of concern to many deferent groups, and as such they demand careful scrutiny. For pupils, they can generate a feeling of achievement and a means by which career aspirations are likely to be satisfied. For parents, they reflect the success or otherwise of their and their children's efforts. For teachers, examinations are a means by which to measure their effectiveness as professionals. For the community, examinations are of profound importance as the results have the psychological effect of indicating the competency, achievement and worth of any nation. This is particularly so in the South African context, where years of ethnic provision of education and deprivation have distorted examination performance. Although the notion exists that examinations reflect mainly on cognitive achievement, it is essential that we note that they also provide important information on sociological, psychological and emotional aspects of a society. Good examination performance can have a positive effect on the self-image of a nation - likewise, poor performance can have a demoralising and destabilising effect.

For the system planners and administrators, examination performance can serve as an important guide to the educational health of a nation. It can be used as a warning when things go wrong as well as an indication of the success or failure of curriculum reforms. A further and very powerful use of examinations, however, is their washback effect on schooling - indeed, the 'assessment-led reform' school of thought holds that the prominent place occupied by high-stakes examinations can be used extremely effectively to drive educational reforms. Thus, educationally sound changes in the examination process and papers are likely to promote improved teaching and learning practices in the preparation for the examinations -conversely, of course, poor or out-of-date assessment practices will derail or severely constrain curriculum innovations.

The Senior Certificate examination, in its present form, plays a critical role in the current education system. It is the culmination of twelve years of schooling and serves as the entrance into Higher Education. In view of this it has become the critical concern of everybody in this country.

In terms of the Constitution of South Africa, the Minister of Education is responsible for norms and standards in education, and the provincial authorities are responsible for the actual provision of education. The provincial authorities are also responsible for matters related to the running of institutions in the General and Further Education bands and conducting examinations is therefore a provincial responsibility. The national Department of Education is not directly involved in the administration of examinations. It does, however, have a monitoring function to ensure that national standards are maintained.

The National Education Policy Act (Act No. 27 of 1996), states that The Minister shall direct that the standards of education provision, delivery and performance throughout the Republic be monitored and evaluated by the Department annually or at other specified intervals, with the object of assessing progress in complying with the provision of the Constitution and with national education policy". The monitoring of examinations, however, must be carried out in terms of the South African Certification Council Act, (Act No. 85 of 1986), which "ensure(s) that the certificates issued by the Council at a point of withdrawal represent the same standard of education and examination".

The first non-ethnic provincial Senior Certificate examination was written in 1996. The number of candidates who wrote the examination was 518 032. Of these, 278 958 candidates passed the examination, the national pass rate being 63,9%. In the 1997 Senior Certificate examination, the number of candidates who wrote the examination was 559 233. In comparison with 1996, there was an overall increase in the performance of candidates of 7,9% (41 201 candidates). The 1997 examination results, however, show a significant decline in the performance of candidates. The pass rate was 47,4%, a decrease of 6,5% when compared with the 1996 results. In total, 14 692 fewer candidates passed the examination. This prompted a national outcry from several sectors of the South African community. The nation began to question the quality of schooling and the examination process.

The Minister of Education, in reaction to the criticism received about the quality of teaching and learning in schools, instituted several initiatives to address the problems. One such initiative was the appointment of the Ministerial Committee on the Senior Certificate examination.

The terms of reference of the committee were as gazetted by the Minister of Education on 25 February 1998 (Government Gazette No. 18708 Vol. 392). They are reproduced as Appendix A.

During its first sitting the Committee discussed the terms of reference. The Committee divided itself into three sub-committees to look into the following focus areas: Quality Assurance of the Senior Certificate examination; the capacity of the provincial education departments to analyse and use the Senior Certificate results to enhance teaching and learning; and the Information Technology systems used for the Senior Certificate examination.

The expanded terms of reference were approved as follows:

1.3.1 Quality Assurance of Senior Certificate Assessment

The SA Certification Act (1986), as amended, will be analysed, with a view to:

• Establishing what, if any, short -term improvements might come into place during 1998;
• Advising on any transitional arrangements that could be made usefully;
• Recommending any longer term re-design of FETC level assessment practices to meet the requirements of current OBET-based policy reforms; and
• Determining the capacity of provinces in respect of the standardisation process.

The following issues will form part of the investigation in respect of the above:

• Forms of moderation across examining bodies;
• The role of the SAFCERT Council;
• Moderation strategies, including: external moderation; statistical adjustment; external instruments; QA criteria; and QA process evaluation;
• The use of national standards and integrative assessments to transform qualifications;
• SAFCERT in relation to other bodies, including SAGA, the Matriculation Board, the Department of Education, NSA, etc.; and
• The role of the Department of Education in relation to all examining bodies, including non-provincial bodies.

The following will be undertaken:

• An assessment of the extent to which the provincial education departments have the capacity to undertake the kinds of analysis (in respect of Grade 12 examinations, including technical examinations at that level) which would enable them to -

• arrive at appropriate conclusions about the state of teaching and learning in the provinces;
• report effectively on the problems responsible for poor performance in the examination; and
• put in place appropriate measures to improve learner and learning site performance.

• The development of recommendations about the kind and scope of analysis and/or procedures which would most likely yield the information listed above, and the way in which such an analysis could best be conducted (including who should most appropriately conduct it). It is envisaged that these recommendations would include procedures for the effective dissemination of these findings, and for the best way in which they could be acted upon to improve learner and learner site performance.

1.3.3 The Examination Computer System

The following will be undertaken:

• An analysis of the examination system currently used by the Provincial Examining Bodies, to determine the suitability, efficiency and effectiveness of the system.
• An investigation whether all programme changes to the system (i.e. pass and endorsement requirements as stipulated in Report 550), have been effected accurately and timeously.
• An assessment of the level of understanding of the computer system by personnel currently working within the system.
• A review of the technical capacity of the provinces to capture, process and report on examination data accurately and timeously.
• An evaluation of the nature, quality and financial implications of the service offered by Q Data Consulting Services in terms of maintaining the computer system.
• An evaluation of the nature, quality and financial implications of the system utilised by the Western Cape Education Department prior to Q Data Consulting Services.
• An investigation of the current and future role of the Help Desk services currently offered by the Q Data Help Desk.
• An assessment of the progress made by the provincial departments in securing a new contract for either the development of a new computer system or the maintenance of the existing system.
• An investigation (in the case of provinces which have decided to develop a new system) of the feasibility of maintaining a parallel run so as to ensure the successful implementation of a new system.
• An investigation, in conjunction with the Gauteng Department of Education, of the future location of the Historical Data and the Help Desk Services.

The investigation had to be conducted within a period of 30 days and within a limited budget. This required the committee to develop strategies to collect, analyse and interpret salient data within these limits.

The committee divided itself into three sub-committees, viz.:

Sub-committee 1 undertook a study of the relevant policy legislation, a literature survey of quality assurance, a study of policy directives presented by the Department of Education and interviews with key role players and stakeholders.

Sub-committee 2 visited the education departments of all nine provinces, with a view to determining the status of examinations and the type of information that was submitted to schools, as well as to test various ideas and options. Discussions were held with various role players such as teacher organisations, subject advisors, examination personnel and curriculum specialists. In addition, examiners' reports were gathered and studied in order to assess the role they played, or could play, in improving teaching and learning.

Sub-committee 3 interviewed key personnel involved with Information Technology. The committee visited all the provinces to determine provincial capacity in respect of the 1998 Senior Certificate examinations.

The full committee discussed the findings of the sub-committees with a view to presenting a holistic and co-ordinated report.

This comprehensive Ministerial Committee Report covers all of the areas as originally gazetted and as per the revised terms of reference that were agreed upon. There are, however, a number of factors that should be taken into account which prevent the report from being as complete as the Ministerial Committee would have liked. These are as follows:

• The time frame given to the project was too short to allow an in-depth investigation of all the issues. Wide consultation was not possible and most contacts were at the first level.

• There was not enough time to test the responses given.

• The lack of availability of many reports and statistics from the provinces made it difficult to form a comprehensive picture of conditions in the provinces.

• Lack of the availability of syllabi, both within the provinces and the Departmental Head Office, was a limiting factor.

• The committee is concerned about the lack of resources, both financial and skills, within both the provinces and the national Department, in order to implement its recommendations. Many of the recommendations take cognisance of this fact, rather than simply making recommendations that could be implemented under ideal circumstances.

• There was no time to study the impact of the South African Qualifications Authority and the National Qualifications Framework on General and Further Education within the school system.

• The committee was only able to look at the IT aspects of the Senior Certificate examinations' Help Desk and associated functions, and the History Section. Because of time-constraints, the committee was unable to review the Formal Technical Colleges and Teacher Examination sections of the Examination Systems as covered by National Tender RTS 1021SA. This tender for the maintenance of all four sections was awarded to Q Data and terminates in September 1999. All four sections are interdependent.

Despite these limitations, the committee believes that its findings and recommendations could make a major contribution to the improvement of the quality of education in the country.

Background

Examination results as a barometer of learner performance in any nation are of great importance, impacting even on their self-image. In this era of "assessment-led reform" it is recognised that high-stake examinations have a powerful effect on teaching and learning in the classroom. Recent poor performance in the first non-ethnic provincial Senior Certificate examinations has caused the nation to question the quality of schooling and the examination processes, and led to the appointment of this committee.

Terms of Reference

The Committee was requested to investigate three aspects of the examination process. Firstly, the quality assurance of the assessment was examined. Secondly, the use of Senior Certificate information by provinces in the process of improving teaching and learning was investigated. Finally, the Information Technology systems used for the Senior Certificate were considered.

Findings

The Committee found that whilst SAFCERT was doing a reasonable job in terms of the statistical adjustment, its work falls short of good quality assurance. Its approach to the moderation of papers is in itself poor, but the fact that this is where the Quality assurance" ends is most worrying. Consequently it is recommended that SAFCERT be absorbed into a Further Education and Training Quality Assurers (FETQA) to be constituted to perform the full quality assurance function as contained in the SAGA regulations. This FETQA should work with the envisaged Learning Site Effectiveness Units (LSEU) to ensure a "hands-on" approach to quality management at classroom and school as well as systems levels.

It is further suggested that attention be given to all aspects that affect learning and teaching in the classroom negatively, but that a single cost-effective assessment remains the best measurement at the end of Grade 12. The Further Education and Training Certificate (FETC) must be designed to perform this function, whilst a properly moderated General Education and Training Certificate (GETC) would measure the exit from free and compulsory schooling at the end of Grade 9.

The Committee also found serious constraints regarding provinces' use of information gained from previous Senior Certificate examinations to improve learning and teaching. The examinations and curriculum sections in provinces should move much closer to enhance and promote the formative value of the Senior Certificate. Closer attention needs to be paid to the selection and training of examiners and moderators and the subsequent use of examination reports to improve what happens in practice needs to be strengthened. In addition, the post structures in the examination sections of provinces need further investigation.

The examination of all first languages needs to be brought in line, with a much greater emphasis on the use of African languages as a tool for cognitive growth. Most significantly, capacity should be built around the use of language teaching (particularly of first languages) for general academic and lifeskills development.

There was broad consensus amongst provinces that Q Data was performing a valuable function and assisting the provinces to run their examinations effectively. Consequently the Committee recommends that there be no new tender until Curriculum 2005 has been fully implemented. Q Data should be allowed to continue their role beyond the September 1999 deadline and should be encouraged to invest in updated operation manuals.

To improve the service further it was suggested that better use be made of the Help Desk for co-ordination and that all provinces migrate back to the Centralised Source Code. Finally it was recognised that the perception that Q Data is expensive was largely brought about because the provinces misuse Q Data consultants for lower level system administration work that should be performed by provincial staff.

Conclusion

There are a number of recommendations in this Committee's report which can be implemented in the short term. To improve the examinations in the longer term, however, the majority of the recommendations need to be implemented. Transforming our human capital involvement in all aspects of the learning and teaching continuum is the only sustainable answer to the problems facing our Grade 12 exit assessment right now and will be the only answer until Outcomes-based Education and Training (OBET) has changed the fundamental nature of the Senior Certificate.

Section 2   Section 3   Appendicies