SECTION TWO

2.1.1 Problems related to the current role of the Senior Certificate

Candidates perceive the Senior Certificate primarily as an access examination focused on university entrance. This can be demonstrated by the subject choice package of the great majority of candidates who offer a package that meets the requirements for a matriculation endorsement. The Senior Certificate has often been criticised for attempting to fulfil three disparate purposes, and failing to achieve any of them, viz.:

• It does not provide an appropriate school-leaving certificate for the majority of SA school students.
• It is not a particularly effective predictor of success at university, except in the higher aggregate ranges.
• It is not perceived by employers as a particularly good indicator of work-related competence.

In addition, there is an unacceptably high failure rate among Senior Certificate candidates. The differences between the Higher and Standard Grades do not seem to be consistent between subjects and the examinations in the various languages, particularly first languages, target very different language skills.

In this section we look at some of the ways in which the current Senior Certificate can be strengthened in the short term, to ensure that it does represent a significant and meaningful achievement, measured against nationally consistent standards. We also explore priorities in terms of a gradual introduction of the new Further Education and Training Certificates (FETC) in parallel with the current Senior Certificate, so that we can be sure that the new qualifications are established and accepted by all relevant stakeholders before the Senior Certificate is phased out. This whole process will involve piloting and should begin as soon as possible.

2.1.2 Improving the relevance of the Senior Certificate using the critical outcomes

One way of beginning to re-establish the credibility of the Senior Certificate examination would be to change the focus gradually so that more relevant and appropriate knowledge, skills and concepts are assessed. The critical outcomes, which underpin lifelong learning and form the basis for developing specific outcomes, are:
Critical cross-field outcomes that should underpin all qualifications:

• Identify and solve problems in which responses display that responsible decisions using critical and creative thinking have been made.
• Work effectively with others as a member of a team, group, organisation, or community.
• Organise and manage oneself and one's activities responsibly and effectively.
• Collect, analyse, organise and critically evaluate information.
• Communicate effectively using visual, mathematical and/or language skills in the modes of oral and/or written presentations.
• Use science and technology effectively and critically, showing responsibility towards the environment and the health of others.
• Demonstrate an understanding of the world as a set of related systems by recognising that problem solving contexts do not exist in isolation.

Additional outcomes which should inform the development of all learning programmer:

• Reflect on and explore a variety of strategies to learn more effectively.
• Participate as responsible citizens in the life of local, national and global communities.
• Be culturally and aesthetically sensitive across a range of social contexts.
• Explore education and career opportunities.
• Develop entrepreneurial skills.

There seems to be a growing consensus that it is possible to define the critical outcomes embedded in current syllabi and to use these to shift the emphasis of education and training to more relevant and useful outcomes in the interim. This means that entirely new syllabi are not necessary in order to begin this shift. Supplementary learning materials designed to show teachers and students how to use existing textbooks and which may form a basis for a more critical and creative cognitive approach, could also be provided in the interim. As the following comment from an examiner (of Biology Higher Grade) reveals, there is a serious need for assistance and direction in this regard, as well as an immediate shift in this direction:

This kind of comment points to an alarming reliance on the worst kind of rote learning. This must be addressed as soon as possible without waiting for new syllabi. Inasmuch as the Senior Certificate examination becomes increasingly focused on those outcomes which have been identified as the key to the modernisation of our curriculum (the critical cross-field outcomes), its results should be able to serve two important purposes, namely:

• To acknowledge the level of achievement reached by each student (in a criterion referenced manner); and
• To provide a basis for selection into Higher Education (HE) institutions.

The committee believes that teaching, learning and assessment throughout all grades should focus on the critical outcomes. In this context a national in-service training (INSET) for teachers campaign is needed. Such a campaign should target the teaching and assessment of the critical outcomes. As this might take time to accomplish, however, it is further recommended that in the short term:

• examiners should receive training in the identification and assessment of critical outcomes;
• these should be included in the setting of examination papers; and
• the examiners' reports should contain feedback on learners' achievements in terms of these critical outcomes.

2.1.3 Improving overall achievement levels in the Senior Certificate by improving language teaching

A specific recommendation in relation to improving all-round competence which might be acted upon immediately has arisen from HE institutions and from examiners' reports. This issue is discussed in more detail in paragraph 2.4.6.2 below. In brief, however, there is evidence that a large proportion of our schools do not give students enough practice in reading - that is to say, in developing critical, selective, analytical and interpretative reading skills - and writing - in developing critical, creative, interpretative, reflective, analytical and transactional writing skills. This lack of opportunity for practice appears to be particularly prevalent in the teaching of African Languages. As a result, questions involving these skills (in all subjects) often account for a large proportion of Senior Certificate failures.

It is suggested that language teaching in general be made an area for immediate emergency INSET intervention, particularly in the area of systematic written work in view of its importance across the whole curriculum. As this is likely to be extremely costly, both in terms of available expertise and financial resources, it is further suggested that in the short term, attention be paid to the ways in which they are examined, in an attempt to bring about speedy improvements. This point is taken further in paragraph 2.4.6.2 below.

In addition, as African First Language Senior Certificate examinations tend not to demand critical and analytical skills, learners who take an African language as their first language are disadvantaged, because they have little incentive to develop these critical tools of learning. That is to say, since the examinations tend not to assess these skills, they are likely not to be taught or developed. The assessment of language varies widely in other ways too, with different weightings for orals and other aspects of the examination.

It is recommended that the assessment of all South African languages be standardised as a matter of urgency, so that all languages at First Language level are examined in a comparable way in terms of critical thinking skills and in terms of the internal language components.

2.1.4 Improving the relevance of the Senior Certificate by introducing coherent rules of combination for all Senior Certificate/FETC qualifications

The Senior Certificate is not entirely successful in terms of providing an effective tool for selection into HE institutions. Nor, it seems, do prospective employers find it particularly useful. Most HE institutions are confident that the students who achieve A and B aggregates are likely to do welt. But they are not so confident about using subjects and symbols to choose between the students who score lower symbols. A number of HE institutions are introducing additional access tests and entrance examinations, despite the fact that this is a costly and time-consuming process and that they agree that it should not be necessary. In the short term some kind of additional selection process seems to be needed. Before the options are examined, however, the long-term goal needs to be clear.

In June 1997, SAQA outlined the criteria for qualifications registered on the NQF. The first criterion in the list specifies that:

The committee believes that the gradual introduction of FETC qualifications at Senior Certificate level will create a situation where additional selection tests are no longer felt to be necessary. An FET Certificate, which meets the above SAQA criteria, would be extending the relevance of the current Senior Certificate. Piloting such new FETC qualifications should be a matter for urgent action.

Although the endorsement requirements of the Matriculation Board of the CUP provide rules of combination in the current system, they fall short in several ways:

• They are not a particularly reliable indicator of success at university.
• They are designed to create coherent qualifications for access to universities, but not to HE in a broader sense.
• They are not designed to create coherent qualifications for access to careers.
• They do not provide any mechanism for redressing historical inequity.
• They do not provide mechanisms for lateral movement, only for vertical movement.

• They are based on the current list of school and technical college subjects/offerings, which is out-dated and restrictive.

It is the view of the Committee that the current Senior Certificate should be gradually adapted so that all certificates, not only those that have been endorsed for the purpose of university entrance, can meet the SAQA criteria set for qualifications. More specifically, the certificate should meet the rules of combination outlined for FETC Level 4 qualifications registered on the NQF. This would be one way of ensuring that the Senior Certificate has greater coherence.

The current National Training Certificates are severely criticised for consisting of 'applied' theory without the context of work experience originally provided by apprenticeships. It is frequently asserted that the level of competence required for the awarding of an N3 certificate is in reality equivalent to an NQF 2, or even an NQF 1. Steps should be taken regarding the N3 examination to ensure that each National Training Certificate qualification consists of a coherent package at the appropriate level on the NQF.

2.1.5 Reviewing the utility of Higher and Standard Grade papers in the Senior Certificate Examination

Many candidates entered for Higher Grade papers perform so poorly that their results couldn't be converted to award a Standard Grade pass. However, it is felt that many such candidates could have passed Standard Grade had they been entered for it from the start. One conclusion drawn from this could be that it is important to retain these two grades and encourage more candidates to enter Standard Grade, thus increasing the pass rate.

This view was repeatedly expressed in examiners' reports as well as in the interviews. However, the Committee feels somewhat ambivalent about it. On the one hand, it is clear that not all learners have the ability to succeed in a subject like Mathematics Higher Grade, and such learners are poorly served by taking it at this level as they are likely to do so badly that they will not gain a pass even on the Standard Grade when the score is converted. On the other hand, it is also clear that many learners who do have the ability to take Mathematics Higher Grade face the danger of being counselled to register for the Standard Grade not for reasons of lack of ability, but for some failure of the system, such as inadequate teacher preparation, lack of support materials, desire for increased pass rates at a school or in a province, or a breakdown of the culture of teaching and learning. In other words, while it is true that entering learners for subjects on the Standard Grade, not Higher Grade, will help to reduce the very high failure rates in Higher Grade subjects, the necessity to do so reflects a failure of the system and not necessarily the inability of learners to succeed on the Higher Grade level given better learning conditions.

Many educators argue that Standard Grade papers tend to demand simple rote learning without understanding or the ability to apply knowledge or transfer skills. They are of the opinion that the current Standard Grade examination perpetuates the iniquities of the Bantu Education system and that all examinations should be set at the same grade (equivalent to the current Higher Grade). However, there is an emphasis in the current Higher Grade examinations on academic theory rather than practical competence and it would be more appropriate if these components were more integrated. In certain subjects, for example Art, it might be important to combine elements of the current Art (Standard Grade) and Art (Higher Grade) to form a single new grade, giving more value to successfully applied artistic talent than is the case in the current Higher Grade examination. The notion of foundational, practical and reflective competence integrating to form applied competence, which emerged from the education training and development practices project, is gaining currency.

In certain learning areas and sub-fields it is possible to introduce a choice of subjects/learning areas. For example, if all candidates are required to demonstrate competence in Mathematics, it might be appropriate to introduce two or three kinds of mathematical studies with differing foci. The Committee does not recommend a proliferation of mathematical subjects with a very narrow and specific focus like the currently registered subject Butcher Mathematics. It is important to research the real requirements of the relevant stakeholders to determine what kinds of mathematical competence are most in demand. At this stage,

however, the Committee is not in a position to make a recommendation in this regard other than to point to the need for investigation.

The Committee believes that it is essential to combine Higher and Standard Grade subjects, defining the integrated version in the context of a genuine standards-setting process, giving sufficient time and involving all the relevant stakeholders. Note that this new, integrated version should not simply be new versions of the old Standard Grade subjects. New learning areas can be piloted and introduced gradually alongside the existing system. In this way the replacements will be tried and tested, and the outdated versions discontinued only when the replacement subjects/learning areas have been proved successful, relevant and appropriate.

2.1.6 Piloting a public examination at the General Education and Training Certificate (GETC) level

In the various discussions held in the course of the Committee's investigation, there seemed to be consensus that the introduction of a GETC and a well-managed and quality assured system of internal assessment would go a long way towards solving the problem of school-leaving certification. An effective assessment system would ensure that the achievement of each student was recognised and accredited at the level reached on leaving school. This should be based on internal assessment monitored in three ways:

• by an external moderating instrument, focusing on the critical outcomes;
• by internal quality assurance systems put in place by the provider (usually the national and provincial Departments of Education); and
• · by external moderation arranged by the Further Education and Training Quality Assurers (FETQA). The establishment of this body is discussed in paragraph 2.2.1.

Such a system of assessment would be educationally sound, would help to professionalise teachers and would ensure that national standards were met.

The GETC would give the majority a basic qualification on which to build. A general education certificate would need to be followed up by providing alternative learning pathways, so that students could select a route suited to their interests and talents. This would allow students to make more realistic choices and to avoid the current situation of having nothing to aim for other than a Senior Certificate pass - which is one of the causes of the current high failure rate nation-wide.

Their introduction of a GETC (NQF level 1/Grade 9) as an exit-level qualification for compulsory schooling would help to provide a suitable qualification for students who are not necessarily going on to traditional senior secondary schooling, or to HE, at this stage in their lives. At the same time, it would also provide a basis for lifelong learning. It would take the pressure oft the Senior Certificate, which would no longer have to serve the purpose of a school-leaving certificate for the majority of people.

The GETC could also be used to shift the emphasis from rote reaming towards the achievement of the critical outcomes in our education and training system. It should aim to prepare learners for a number of learning pathways, including an improved Senior Certificate and an effective alternative FETC system. Improved quality assurance mechanisms piloted in the context of the GETC will serve to raise the standard of education and- training generally. This will help to ensure that public confidence in both the old and the new qualifications is restored.

It is recommended that a pilot GETC, meeting the requirements outlined above, be initiated by the year 2000 at the latest.

2.1.7 A changing perception of 'recovery' end 'second-chance' programmes

The current extremely high failure rate in the Senior Certificate reflects a distorted perception of SA reamers that access to opportunity in our country is limited to those who move directly from school to HE institutions. Thousands of learners continue to attempt the Senior Certificate, in spite of the fact that they are totally unprepared for HE. Many have been shown, through placement testing for 'second-chance' programmes, to be barely at a Grade 7 level of competence.

This misconception also results in a general total unpreparedness for the labour market, as vocationally oriented programmes are seen as second best. (In this context vocationally-oriented programmes refer to learning programmes which facilitate access to immediate occupation on completion, but does not refer to programmes which are narrowly specific to one job only.)

Given the lack of alternative learning pathways, the state, not surprisingly, hesitates to implement its policy of providing free compulsory general education only up to the age of 15 years. Thus we allow people to overcrowd our schools and use our scarce resources in repeated attempts to pass the Senior Certificate. Annually thousands of candidates still study and write the Senior Certificate Examinations at the state's expense even though they are well over 20 years old. Each year large numbers of Senior Certificate candidates should never be allowed to enter for the examination in a responsible education system, as they are certain to fail. It is imperative to provide alternative routes not only into the workforce, but also, by building clear lines of articulation, to Higher Education so that the stigma currently attached to vocationally oriented education may disappear.

Even the wealthiest countries in the world select only a minority of students to follow an academic route after the age of 15, providing alternative learning pathways for the majority of students. In many countries some kind of job-related experience is provided even earlier, within the general (compulsory) school curriculum, and this is considered to be valuable for all school students. It is a part of education to which students are entitled.

It is clear from the FET White Paper and from the general acceptance of the need for research and development processes, that it is necessary to re-conceptualise and re-structure many of our existing programmes at Senior Certificate/FETC level. Youth and community colleges and 'finishing schools' have been set up. These have attempted to provide young students who have reached Grade 12 (but failed the Senior Certificate) with another opportunity to pass the same examination, or with an alternative at the equivalent level, giving access to the same kinds of HE opportunities as the traditional Senior Certificate. The ASECA and SYSTEM examinations fall broadly into this category.

Experience now indicates that many of the students directed to such courses are unsuitable. They have by no means reached the expected level of achievement of students for the current Grade 12 and should probably not have been entered for the Senior Certificate in the first place. Such programmes can provide a valuable service, but it is recommended that education departments be more selective when they direct students to such courses. If students have little hope of succeeding, the experience will prove a negative one, and valuable resources will have been misdirected.

Initiatives such as the National Access Consortium in the Western Cape, or the Regional Access Programme of the Eastern Seaboard Association of Tertiary Institutions (ESATI) offer an alternative approach to the 'second chance' philosophy. It might be worthwhile to investigate both the successes of these programme and their shortcomings, so as to arrive at an alternative model for improving access to HE institutions for students who have failed their Senior Certificate or failed to achieve endorsement.

The Committee recommends that a clear date should be clearly set during 1999 for the implementation of the age of 15 as the end of compulsory and free education. This implementation should be timed to coincide with, or follow, the introduction of a credible public examination at GETC level. This is not to suggest that learners may not continue their education through the schooling system after this age, but that free education cannot be guaranteed beyond this stage.

It is further recommended that this implementation is monitored and that alternative vocationally oriented qualifications be researched designed and piloted from the year 2000 at the latest.

2.1.8 Piloting new FETC learning programmed and qualifications to supplement the current Senior Certificate subjects

Neither business nor labour is convinced that the skills and knowledge measured and valued in the current Senior Certificate represent the most desirable selection of outcomes for use in the workplace. School and technical college curricula desperately need modernisation. It is important to consider what action may be taken in the short term, before the innovation cycle represented by the introduction of Curriculum 2005 is completed.

The Senior Certificate/FETC should move towards meeting the needs of learners, labour, business and government in the following ways:

It should serve to prepare learners for Higher Education, providing opportunities for short-term and longer-term access via a number of different learning pathways.

It should provide a link between General Education at school and work, allowing learners to choose occupationally-specific learning pathways, whilst demonstrating clear lines of articulation with HE qualifications which can be taken up at an appropriate stage in their careers.

This requires a complete overhaul of the current array of subjects offered to students in the FET band.

The Department of Education is currently engaged in the enormous task of rationalising the existing school and technical college subjects. The scope of this project goes beyond the Department of Education and it could well be performed in partnership with Higher Education and research institutions, the Department of Labour and other stakeholders who are interested in seeing the curricula modernised and rationalised.

Several Higher Education institutions and research organisations are already involved in the development of new modules in preparation for an outcomes-based Senior Certificate or FETC. In addition technological, managerial, scientific, creative, cultural and service-oriented studies at Senior Certificate level are being advocated. The Department of Education has an important role to play in the facilitation and co-ordination of this process.

The Committee believes that vocationally oriented subjects could best be revised in negotiation between the Department of Education, the appropriate standard-setting bodies and SETAs. Learnerships are an important element in vocationally oriented learning pathways. These involve extensive workplace experience, in partnership with business and industry. It is important for the Department of Education to work with the Department of Labour and SAGA to ensure that occupational qualifications are full FETC qualifications and not narrowly work-specific.

Such programmes could be conducted in the classroom, college or workplace, but they would all have a workplace component. The move towards partnerships between education and industry characterises the education systems of developed countries throughout the world. In the context of the senior phase in secondary schools, occupationally oriented learning programmes would provide an alternative to the more academic route and could co-exist with a more traditional Senior Certificate system in selected learning sites. One possible way of introducing such a choice might be to pilot a new approach in selected schools:

• to give all Grade 10 students some indication of the possible occupational application of some of the work they already do in the current interim syllabus (overview of occupational fields);
• to give all Grade 11 students an opportunity to choose (for example) up to three work-related subjects for Senior Certificate (gradually become a little more specialised); and
• to encourage some Grade 12 students to compete for learnerships as the final component of an FETC involving work experience.

In order to raise the status of such alternative programmes, it is suggested that the number of student accepted for such work-related learning pathways in the country as a whole be limited (at least initially) to a number slightly greater than the number of learnership places offered by industry. This would ensure that students compete for places, are rewarded by genuine work experiences, achieve occupational competence and are likely to end up with jobs. In addition, industry would be more likely to cooperage if it were clear that they could select the most suitable candidates who would be likely to succeed in the field.

The Committee therefore recommends that the Department of Education and SAGA should collaborate with selected SETAs to develop, by the year 2000, a number of sectorally oriented learning programmes and qualifications involving key combinations of subjects. These must be piloted, using schools, learnerships and partnerships between education and industry. Tracer studies should be undertaken so that policy makers and planners acquire sufficient data to evaluate the success of the programmed. This evaluation will guide the scale of implementation and its time frame. In addition, these pilots could serve to develop and test new partnership relationships between selected schools, technical colleges, businesses and industries. This would help to open up the current Senior Certificate so that it could be more flexible, relevant and responsive to labour market needs.

2.1.9 The impact of learning and teaching on the Senior Certificate Examinations

Numerous interviewees commented on the limited value of the work of this particular Committee, unless extensive investigation of conditions in the classroom took place. Examinations are obviously dependent on the quality of learning and teaching that takes place in the classroom. It is widely acknowledged that South Africa is experiencing a number of problems as a result of the transition from the previous racially segregated systems. These problems include resource constraints and, most importantly, a generally diminished culture of learning.

A number of factors were cited as reasons for matriculation results as low as 3% on average achieved by certain schools. These include the following:

• Textbooks sometimes reach pupils literally days before an exam is written, or not at all. In the Setswana First Language Paper in one Province, for example, the examiner commented that:

"l came across many scripts where students claimed that they met books for the first time in the examination room. As a result of this a number of candidates handed in their answer books without anything written on them. Most of the candidates wrote little notes at the end of their answers to say that they did not read the books, they did not understand, they were not taught and that they read books that were not prescribed. "

In one particular province, textbooks were stored at the provincial office, but were never distributed owing to a lack of resources. In others, the collection of textbooks from the previous year is poorly controlled, so that almost none are recovered to be used by the next class. The planning process, however, assumes that these books will be available and so new ones are neither ordered nor budgeted for.

• Another commonly cited problem is the issue of the classroom being declared a no-go area for anyone other than the teacher. This severely limits any type of quality assurance approach, especially where a relatively high percentage of teachers are under-qualified. Clearly, while it makes sense to use existing subject advisors and school development officers as part of a hands-on quality assurance approach in schools, this cannot happen unless these 'inspectors' have free access to all areas in all schools. This needs to be linked to an appraisal system, both for classroom teachers and departmental officials. Teacher organizations would ensure that correct appeal and grievance procedures were in place. Quality Assurance mechanisms cannot be implemented until all departmental officials have free access to all areas of the school.
• The attitude of educators is widely cited as a cause of poor results. A number of incidents of teachers being 'on strike' in the classroom were cited. In this regard, schools that were equally under-resourced were said to have achieved dramatically different results in the examination. The conclusion drawn was that the major variable was the attitude of the educator. Issues around appointments made for seemingly political reasons were cited as a major reason for the poor management of many underachieving schools.
• Low and erratic attendance is another cause of poor performance. Based on the information in our discussions with provinces, it appears that learner attendance is very low or erratic at the following times: after the lunch break, when few learners return to school in many schools, making it well-nigh impossible for lessons to continue; on pension payment days, when many children accompany their grandparents to collect their pensions; on pay-day, when the teachers are present for only part of the day; and in some areas on Mondays and Fridays when senior staff members are absent as a consequence of the moratorium on the use of government vehicles on those days. In addition, it seems that many schools routinely begin their school day in a haphazard fashion and seldom at the stated time, that some teachers arrive at class late and leave early, and that registers of attendance (relating to educators as well as learners) are not kept.
• Lack of appropriate standards of assessment prior to Grade 12 add to the difficulties experienced by learners. It is quite clear from the examiners' reports as well as from our discussions with the provinces that many learners sit for their first HG examination in various subjects for the first time in the Senior Certificate examinations. Until this time, they have written papers set on parts of the syllabus (e.g. educators only cover two chapters of the textbook, so they set an examination only on this and learners do not realise that they would have failed an exam which covered the syllabus) and/or on SG when the learners are meant to write on HG. In many schools, Grade 12 students registered for Mathematics HG had still not begun their Geometry syllabus by the middle of August, i.e. some 20 school days before the end of teaching. It is no wonder that learners' Senior Certificate results bear no relation to the results that they have been accustomed to achieving and come as a shock to both learners and educators. It also means, of course, that learners have no experience on which to base their preparation. In this regard, the Committee's recommendation is that the Grade 11 examinations should be moderated internally by the Subject Head or Head of Department, and that proper, full-scale 'trial Matric' examinations be held no later than August in Grade 12, preferably early enough to allow remedial action to be taken where necessary.
• Inadequate preparation on examination techniques is another problem area. In many examiners' reports, mention is made of the fact that learners appear to have no strategies for writing examinations, and thus ignore instructions or spend disproportionate amounts of time on some sections of the papers.
• Learners are generally treated in an arbitrary fashion; for example, they are registered for the wrong subjects, or promoted when they should not have been. They have no reliable way of assessing where they stand or what their chances will be in their final examination.
• Inadequately trained and uncommitted chief examiners add to the general disillusionment and lack of motivation surrounding the examination.
• There is real evidence that learners rely on the oral mode only. There appears to be little written consolidation with feedback. This is confirmed in the examiners' reports.
• In some cases, educators do not know the syllabus, e.g. that there is a practical or oral component. Often educators are not fully competent themselves in the subject; for example, many educators are not competent to teach at HG level, so they teach the whole class at SG level (and frequently in the same classroom).
• Educators do not manage their own time or work properly and in some cases may be teaching subjects they are not trained for.

Clearly, there is no overnight solution to many of these deeply seated problems. It was generally felt that a number of interventions designed to develop the skills of examiners, moderators, subject advisors and school development officers, would be a good way of addressing some of these problems in addition to the obvious interventions, i.e. a re-examination of the role and qualifications of teachers themselves. These interventions could be focused on new responsibilities for quality assurance that should be part of the new thrust towards the establishment and functions of Learning Site Effectiveness Units. There was, in addition, general agreement that a more detailed and scientific investigation into what goes on in our under-achieving classrooms needs to take place as a matter of urgency.

Another widely held belief was that assessment could be dramatically improved once educators understood and could implement outcomes-based assessment. Many interviewees felt that we should not wait until 2005, but rather concentrate on developing assessment capacity through challenging educators to develop outcomes-based assessments on existing syllabi. One grouping even suggested an incremental percentage of the syllabus to be assessed in this way annually.

2.1.10 Consolidated Recommendations

A single, cost-effective Further Education and Training Certificate should be developed to meet the requirements of a school-leaving certificate, a work-readiness certificate and access to higher education certificate. This design would include the categorisation of subjects into fundamental, core and elective, as well as subject packaging to ensure coherence. The process of rationalisation of school and technical college subjects which is currently underway in the Department of Education should be broadened to include representatives of the Department of Labour and SAGA and should extend its scope. It should take the form of a small Standing Committee which should include relevant stakeholders and have access to research and development expertise as needed over a minimum period of five years to co-ordinate the following activities:

• The rationalisation and revision of existing school and technical college subjects.
• The integration of Higher and Standard Grade subjects.
• The research and design, in collaboration with appropriate SETAs, of alternative vocationally oriented subjects/learning areas and qualifications.
• The development, in collaboration with selected SETAs and ETQAs, of a number of new sectorally oriented qualifications at NQF levels 2-4, involving key combinations of subjects and including learnerships.
• The development of learning programmes for the above qualifications.
• The development of valid and reliable assessments for the above qualifications.
• The generation of transformative standards for the above qualifications at levels 2-4 on the NQF in the context of a genuine standard-setting process (which takes time).
• The piloting of the above-mentioned new qualifications within selected schools and colleges, to begin by the year 2000 at the latest.
• The piloting of the above-mentioned new qualifications involving learnerships in the context of partnerships between education and industry to begin by the year 2000 at the latest.
• The initiation of tracer studies to acquire sufficient data to evaluate the success of the programmes and qualifications in terms of access to employment and to HE.
• The gradual phasing in of the above new qualifications, with the outdated versions only being discontinued when the replacement subjects/learning areas have been proved successful, relevant and appropriate.

• The Department of Education should set up a Task Team, including representatives from SAGA, the Department of Labour and the HE sector, to re-formulate the requirements for a Senior Certificate, and to re-examine the requirements for endorsement for university entrance: This team should concern itself with (also see Paragraph 2.3.2 below):

• reaching agreement so that by the year 2002 all Senior Certificates meet the requirements set by SAGA for FETC level 4 qualifications;
• reaching agreement on whether, within this new context, there will be any need for additional requirements for endorsement for university entrance, or whether this should be left to individual institutions and/or faculties;
• monitoring, during the redesign of Senior Certificate/FETC, existing admission tests that promote equity and inclusiveness to prevent them from becoming parallel admission systems to the current Senior Certificate examination;
• reaching agreement upon any other selection criteria or processes which might help TO identify candidates for Higher Education who have had poor educational opportunities but achieved comparatively well. In this context the increased and innovative use of existing examination data (for example to show rank order of candidates per examination centre) for use in university admissions procedures should be investigated; and
• planning an advocacy campaign aimed at the HE sector to ensure that the interim Senior Certificate qualifications agreed upon are understood and accepted as credible by the key stakeholders.

• The Committee recommends that a definite date be set during 1999 by the Minister of Education for the implementation of the age of 15 as the end of compulsory and free education. This should not take place until a credible GETC examination has been established.
• The Department of Education should ensure that a GETC be designed at a national level and piloted in selected representative schools in all provinces, beginning in the year 2000 at the latest.
• A Task Team should be set up during 1998 to make recommendations regarding the standardisation of language assessment at Senior Certificate level. This team should concern itself with:

• standardising the weightings given to different components of the assessment in first language examinations, particularly in relation to the 11 official South African languages;
• setting a time frame for a shift in the emphasis of the examinations towards the communicative and higher cognitive skills embodied in the critical outcomes; and
• setting a time frame for the training of panels of language examiners and moderators to implement these new assessment guidelines in practice.

• A national INSET campaign should be launched to begin in 1999 and continue for a period of six years. The programme should target the teaching and learning of the critical outcomes throughout all grades, with special emphasis on the context of language teaching. This will entail:

• INSET training on classroom-based management and assessment of OBET, focusing on the critical outcomes;
• INSET training targeting the use of existing materials for the teaching of language with an OBET approach, focusing particularly on the critical outcomes; and
• the Senior Certificate examinations need to begin (with immediate effect) to incorporate the assessment of the critical outcomes. This will help to focus the attention of educators and learners on their importance, and will 'kick-start' and reinforce the INSET campaigns mentioned above.

• A national INSET campaign should be launched to begin in 1999, continuing for a period of six years, targeting the management of quality assurance systems and the introduction of reliable and valid internal assessment practices. This should focus on the establishment of competent Learning Site Effectiveness Units at national and provincial levels. This will entail:

• INSET training around quality assurance management systems for assessment at school quality co-ordinator level; each school will have a designated quality co-ordinator who will be responsible for organising and running moderation and standardisation meetings, convening meetings for the setting and marking of assessments in learning area or year-group teams; the on-going development and implementation of internal whole-school assessment policy; and the internal co-ordination of INSET relating to assessment and quality assurance practices;
• INSET training around quality assurance management systems for departmental officials at middle management level; this will focus on preparing the current school development officers, subject advisors, etc., to perform the main quality management functions within the Learning Site Effectiveness Units in relation to the moderation of internal continuous assessment, including practicals, portfolios and orals; and in co-ordinating standardisation meetings at cluster and district level to ensure standards are interpreted consistently;
• INSET training around quality assurance management systems for assessment at senior management level; this will prepare participants to monitor and ensure accountability in the LSEUs and take regular action on reports from middle management quality assurance functions, as well as co-ordinating moderation at provincial level and reporting to the Department of Education LSEU and the FETQA on relevant matters.

2.2.1 The role of SAFCERT in assuring the quality of the Senior Certificate Examinations

SAFCERT is responsible for the moderation of Senior Certificate examinations across examining bodies, for the maintenance of consistent standards from year to year within each examining body and for the issuing of Senior Certificates. While this task is carried out efficiently by SAFCERT, quality assurance should involve much more than this. In carrying out these functions, SAFCERT makes regulations that must be followed by the examining bodies concerned and it charges these bodies for its services. SAFCERT is governed by a Council, which is appointed by the Minister of Education, who also appoints the Chair of the Council. This Council reports directly to Parliament on an annual basis.

The functions of SAFCERT should gradually be taken over by a new body, the Further Education and Training Quality Assurer (FETQA), established in line with the SAGA Act and regulations already gazetted by the Minister. The FETQA will be responsible for the accreditation of learner achievements in the FET band. It may delegate this function to accredited examining bodies - assessment agencies - but will remain accountable for the consistency, reliability and validity of the credits and qualifications awarded.

The FETQA will also be responsible for the accreditation of providers in the education sector. It may delegate this function to the national Department of Education and Provincial Department of Education Learning Site Effectiveness Units, or in the case of private providers to other organisations, but it will remain accountable for the monitoring of these quality assurance systems.

In the interim, SAFCERT should play a stronger role in the training function of examiners and moderators and in quality assurance in general, rather than relying mainly on the adjustment of marks to ensure acceptable standards. This will help to lay the foundation for a smooth transfer of these functions to the FETQA.

2.2.2 Moderation of standards across examining bodies and over time by moderating examination papers

The practice of using the raw scores of the previous five years to construct norms for the next year is a well established one, but it relies heavily on the assumption that the standards of the examinations (including the examination papers, the memorandums and the marking) were satisfactory during those previous five years, for all the examining bodies. The role of the moderators in ensuring that these assumptions are true cannot be overemphasised. It is therefore of the utmost importance that the moderators must be the best available, that one must be able to rely on them to perform their moderating functions well, and that nothing hinders them in performing these duties.

In relation to moderation of standards across examining bodies, the current procedures are very limited and flawed. SAFCERT has a moderation committee that appoints SAFCERT moderators for all subjects. All examining bodies are asked to send at least one examiner and one internal moderator per paper each year to a meeting called by SAFCERT, and run by the SAFCERT moderator. At this meeting the standard of different questions and overall papers is examined and an agreement is reached on how to deal with issues arising from past and current experience. Currently many of the bodies send no one, as they lack funds to cover costs. SAFCERT intends to remedy this situation from 1998 onwards by paying all costs and insisting that attendance is compulsory. Attendance of meetings of moderators and examiners must be made compulsory for the examiners and internal moderators of all examining bodies and a way must be found to cover the costs. A concerted effort must be made to ensure that each of these meetings constitutes a serious opportunity for developing quality assurance competence.

In addition, the question papers and memoranda set by each examining body are sent to the SAFCERT. moderators after the papers have been moderated internally. The SAFCERT moderator will then try to:

• improve the paper in general terms (balance, coverage, language, layout, clarity of expression etc); and
• ensure that the standard of papers in the subject in question, is the same for different bodies.

It is generally conceded that it is impossible to achieve exactly the same standard across the various bodies. However, if the process outlined above is followed conscientiously by examining bodies and performed by competent SAFCERT moderators, it would certainly provide a sound basis for the establishment of consistent national standards. In reality many of the bodies allow their examiners and internal moderators to fall well behind schedule, leaving little or no time to follow the recommendations of the SAFCERT moderators. Occasionally the papers have already been sent to the printers before they are sent for moderation.

In these circumstances, SAFCERT needs to exercise more control, even if this means insisting on seeing papers much earlier in the examination time frame. If a paper was unacceptable and SAFCERT was not convinced that it could be corrected in time, the examining body concerned could be instructed to buy in one of the papers in that subject which had already been moderated and approved. The following year they would be free to set their own paper again. This would not be a particularly harsh step to take, since examination bodies frequently buy in papers from other bodies in those subjects where few candidates write. It is imperative that candidates be protected from poor and unfair examination papers.

SAFCERT, and later the FETQA, should appoint a panel of moderators for each subject. These posts should be advertised. The date of the meeting of the panel of moderators (or some specified time before that) should be announced in advance as the deadline for the submission of papers by the examining bodies. It is imperative that these moderators be trained in both the practice and procedures of moderation.

SAFCERT seems to exercise tittle control over the flow of correspondence between the examining bodies and the SAFCERT moderators. All correspondence between the moderators and the examining bodies must be conducted through the SAFCERT office. All papers must be sent directly to SAFCERT. This situation should continue until SAFCERT has been fully integrated into the FETQA.

2.2.3 Moderation of standards across bodies by using common questions/papers

For some time SAFCERT has tried to persuade examining bodies to include a common question in the common part of the papers, which could then be used as the basis for statistical adjustment. This would therefore become the most important part of the examination for standardising candidates' marks and might result in educators focusing on the kind of question included in the common section. Teaching and learning would therefore be forced in a particular direction. This washback effect has been widely researched and is often summarised with the acronym WYTIWYG (what you test is what you get).

The common questions used in the past by SAFCERT to compare a number of examining bodies took the form of multiple-choice questions that tended to focus on content memorisation. For this reason certain bodies refused to participate, as they felt that their candidates were able to display higher order critical and creative thinking skills, conceptual understanding and application, and so on, which the common questions did not value. Concerns were also expressed about the effect of possible cultural and other biases in these questions.

It is important, therefore, to reach consensus about the design and focus of a common moderating instrument if this is to be introduced. One possibility might involve developing and pre-testing questions to eliminate bias.

A concerted effort must be made to reach consensus about common questions or a common paper on an experimental basis, even if only for a limited number of subjects and a limited number of years, for incorporation into the quality assurance approach of the FETQA.

2.2.4 Maintenance of consistent standards from year to year within each examining boa,

SAFCERT has also had the responsibility of trying to ensure consistency of standards from year to year within each examining body. This is normally achieved by using the raw scores from the previous five years to calculate a 'desired distribution' to use as a basis for comparison and, where necessary, statistical adjustment. This practice should be carried out sensitively, in consultation with all stakeholders who take into consideration all available information that should be considered. A rather crude mechanistic adjustment to force results to fit a bell-curve is undesirable. This is considered to be an important tool for moderation throughout the world.

The main difficulty with this in the South African context is that it assumes a stable group of candidates over a five-year period. It seems to be agreed that in the absence of this, SAFCERT has approached the problem in a logical way, producing the most accurate 'desired distribution' possible under the circumstances, until recently. However, the actual amalgamation of the former apartheid examination boards into provincial examination boards is widely perceived to have advantaged the formerly white schools and disadvantaged candidates from the previously black schools. In addition, reports of adjustments of up to 9% on many papers in some provincial examination departments (albeit approved by SAFCERT) have undermined public confidence in the results in some quarters.

The poor results of 1996 and 1997 (and possibly 1998) must not be allowed to dominate the norms for the next five years, should the results improve over the next few years. The effect of any improvement in the education situation in general should not be negated by these poor results.

2.2.5 The question of second language as medium of instruction

It is clear that candidates who write their non-language subjects in a language other than their first language, are at.a considerable disadvantage not only in terms of their own performance (i.e. whether they can fully understand the questions and convey their understanding to the examiners effectively) but also because they are being assessed in comparison with other candidates who are writing in their first language.

This is obviously an inequitable situation. In an ideal world, or at least in the long term, one would hope that with greatly improved language teaching, and/or as a consequence of the rapid development of all the country's official languages, these real and comparative disadvantages would disappear.

2.2.6 Reporting the results of repeaters separately

The number of candidates that repeat the Senior Certificate examination varies from province to province. In some provinces the number of repeaters can be as high as 50 000 candidates. There is no doubt that candidates who repeat the examination have a negative influence on the distribution of raw marks (i.e. they lower the pass-rates, and distort the true picture as regards education in the province). In order to determine the extent of the influence of repeaters on performance it is necessary that repeaters be identified.

The Committee recommends that Q Data be requested to devise a programme that could identify repeaters. The issue of whether the norms should be based on first -time takers of the examination and the inclusion of repeaters needs also to be investigated.

2.2.7 The levying of charges

In view of the large numbers of candidates who register for the examination but who do not actually write it and in view of the large numbers of candidates who write the examination although they are totally unprepared for it, the Committee believes that the issue of an examination fee needs urgent investigation. This is strongly supported by all who were interviewed during the investigation.

2.2.8 Lines of accountability and reporting for SAFCERT

SAFCERT reports to parliament annually. Its report is brief and consists mainly of accounts of income and expenditure and statistics relating to the Senior Certificate examination results. There is some concern about the skeletal nature of the reports. It is felt that the Department of Education and IPEC could benefit from closer interaction with SAFCERT throughout the year. For example, SAFCERT, integrated as part of the FETQA, could play an important role in helping the Department of Education and examining boards to analyse the examination results in order to identify and solve problems of delivery, quality management and assessment. In addition, the SAFCERT Council, for the interim period of its existence, could benefit from closer consultation with the Department of Education in relation to proposed changes in regulations or procedures which should be taken in the context of education and training policy.

The annual report of the FETQA (including SAFCERT) should play an educational role in addition to its present role. It should be made available to the examining bodies and could entail some analyses of the results and inferences that may be drawn from these.

SAFCERT should interact more closely with the quality assurance directorate of the Department of Education throughout the year.

2.2.9 The role of examination boards/ assessment agencies

Several concerns have been raised about the provincial education departments each running public examinations for their own provinces. One of the concerns raised in many quarters concerns transparency (it is perceived that the player acts as referee). Another concern is about capacity in terms of human resource expertise and numbers. Finally a concern was raised about the duplication of functions for small numbers of candidates, resulting in the waste of scarce national resources.

An immediate way to improve this situation would be for existing examination bodies to form consortia to share expertise and resources. Some provinces are considering making the examination departments independent of the provincial education department in response to the first concern raised above. (If this goes ahead, however, there is a danger that it will become very difficult for the examination to play a formative educational role through the close involvement of the curriculum section in the examination process, as is proposed elsewhere in this Report.) In the long run it is possible to envisage a system where there are four or five examining bodies/assessment agencies catering for GETC and FETC qualifications for schools, technical colleges and FE colleges nationally.

The Department of Education, in consultation with the existing examining bodies, should investigate the possibility of groups of provinces pooling their resources to run the Senior Certificate examinations.

2.2.10 Consolidated recommendations

• The national Department of Education (Department of Education) should draw up a plan for the establishment of an FETQA for the Further Education and Training band (FETQA) in line with the SAGA Act and regulations already gazetted by the Minister. This body has been posited in the FET Green and White Papers and the FET Act. The Department of Education should submit this plan to SAGA for approval as a matter of urgency, certainly before the end of 1998, and set up a governing body that meets the criteria for an FETQA to oversee the establishment of the new FETQA.

• The newly appointed SAFCERT Council should be instructed by the Minister to work closely with the governing body of the new FETQA to improve current quality assurance systems and establish new ones, with the understanding that SAFCERT will, during the next five years, be absorbed into the new FETQA. The new FETQA governing body should include some of the stakeholders already represented on the SAFCERT Council.

• The SAFCERT Council should take immediate action on the following recommendations:

• A high level post should be created to oversee the appointment and training of SAFCERT moderators, to monitor the process of moderation of the examination papers, to monitor and co-ordinate input from marking into the process of statistical adjustment and to ensure that quality feedback is given to educators throughout the country.
• A second person should be appointed to work with the current CEO of SAFCERT to ensure that the adjustment of raw scores continues in an educationally sound, sensitive and flexible manner after the current incumbent's retirement.
• Attendance of the meetings of moderators and examiners must be made compulsory for the examiners and internal moderators of all examining bodies and a way must be found to cover the costs. A concerted effort must be made to ensure that each of these meetings constitutes a serious opportunity for training and development.
• A panel of moderators should be appointed for each subject. These posts should be advertised. The date of the meeting of the panel of moderators (or some specified time before that) should be announced in advance as the deadline for the submission of papers by the examining bodies, in order to ensure that papers can be compared. All correspondence between the moderators and the internal examiners and moderators of the examining bodies must be conducted through the SAFCERT office. All papers must be sent directly to SAFCERT.
• A concerted effort must be made to reach consensus about common questions or a common paper on an experimental basis, even if only for a limited number of subjects or one subject. One possibility is an additional cross-curricular paper based on the critical outcomes. This should initially be introduced for a limited number of years. This could help to establish consistency of standards across different examining bodies/assessment agencies.
• The SAFCERT annual report should play an educational role in addition to its present role. It should be made available to the examining bodies and could entail some analyses of the results and inferences that may be drawn from these. In addition, SAFCERT should interact more closely with the Department of Education throughout the year.
• The new FETQA will gradually incorporate the SAFCERT functions as well as expanded quality assurance moderation functions which it should develop in consultation with the Department of Education: Quality Assurance Directorate. This will provide a basis for the accreditation of providers offering credits and qualifications in the PET band, using the resources and expertise of the proposed nationally based and provincially-based Learning Site Effectiveness Units (LSEUs). It will also provide part of the basis for the accreditation of learner achievements, including those assessed internally, or assessed in part internally. This FETQA will cover only those providers (private and state) delivering qualifications which fall outside the sphere of SETAs.
• An additional Quality Assurance body should be established as the co-ordinating and moderating agency for all the FETQA (including SETAs and professional bodies) deemed to fall within the world of work. This body will facilitate contact and co-ordination between providers and FETQA and moderate quality management across the FETQA.
• Providers who are accredited by this workplace Quality Assurance body need not register with the Registrar of private further education providers as contemplated in the FET Act. Such registration will apply to all providers falling under the jurisdiction of the FETQA.
• The question of the examination fee to be charged overall needs investigation.

2.3.1 The Role of the Matriculation Board of the CUP

The Matriculation Board of the CUP received the statutory function of determining general university admission requirements on September 1992. On behalf of the CUP it issues certificates of complete and conditional exemption from the matriculation endorsement requirement in terms of the provisions of section 7(1)(e) of the Universities Act.

The CUP, which has changed its operational name to South African Universities' Vice Chancellors' Association (SAUVCA), intends to transform itself from a statutory body to a Section 21 company to meet the demands for transformation. A number of changes in legislation have had a major impact of the way the Matriculation Board sees its role in the changing Higher Education field. These include the Higher Education Act, the Further Education and Training Bill, the South African Qualifications Authority Act and the challenges posed by Curriculum 2005.

The major criticisms of the way the Matriculation Board currently operates focus on the complexity of its rules, the level of achievement in the compulsory two languages and the fact that a variety of subjects from four different groups are required. SAUVCA itself has acknowledged the need for a simpler and more flexible university admission system that, importantly, does not lower university admission standards. To this end, SAUVCA has suggested the so-called M4 proposal as a solution to the problems surrounding university admission. In essence, this proposal does introduce more flexibility, but it is only marginally less complex than the existing rules.

No real justification for the continued existence of the Matriculation Board can be found once the FETQA, which has been discussed previously in this report, has been successfully established. This envisaged FETQA will incorporate both the functions of SAFCERT and the Matriculation Board, and it will perform an extended quality assurance function similar to that currently performed by SERTEC. Of equal importance for successful quality assurance, is the careful design of exit level qualifications in terms of the rules of combination laid out in the SAGA regulations. Properly constructed, the Further Education and Training Certificate could meet the requirements of both a school leaving certificate, a basic readiness for work certificate as well as a higher education access certificate. The nine functions spelt out by the Matriculation Board in its June '98 document, "The Role and Functioning of the Matriculation Board within the Education System", could all be performed by either existing or developing bodies.

These functions are:

• maintaining a check on evaluation for admission purposes. This will become redundant once the FETQA assumes its moderating role as spelt out further in paragraph 2.2.1, and based on the redesign of the Senior Certificate towards the FETC;
• co-ordinating curriculum expectations at FET level should become redundant once the NQF is fully implemented and the NSB's have all worked out their rules of combination. The concept of bridging is in itself a problematic one and should have no place in a properly designed and "seamless" qualification system;
• devising solutions to the under-supply of school leavers with Mathematics and Science is clearly strategic function of the Minister and his department, not of the Universities, although their assistance should be enlisted in this regard;
• contributing to the NQF and OBET debates is an important function of SAUVCA, but not specifically of the Matriculation Board;
• introducing measures to recognise prior learning is clearly and statutorily the function of the ETQA's that will be established under SAGA legislation. SAUVCA, as an important stakeholder in higher education, should channel this function through its association with the Higher Education Quality Committee (HEQC);
• satisfying needs for admission seems to be the function of individual universities, or even individual faculties within universities. If what is meant by this function is developing some common guidelines for interpretation by individual universities, then SAUVCA could delegate a sub-committee to develop these on an ad hoc basis;
• striking a balance between massification and funding is at once strategically the role of the Minister and his department, but also the function of each university. It is difficult to conceptualise the role of a Matriculation Board in this area;
• maintaining equilibrium in university admission standards could be seen as an interventionist role. If there is merit in this kind of intervention, again it should be seen as a strategic role of the Minister and his department; and
• trailing alternative assessment mechanisms should be seen as an ad hoc project of national stakeholders, rather than a function of the Matriculation Board.

It is emphasised, however, that bodies such as the Matriculation Board must be kept in place until replacement bodies are demonstrably effective.

2.3.2 Access to Higher Education

As long as places at higher education institutions remain more limited than the actual demand, access will remain an emotive and sensitive issue. Currently, the Matriculation Board endorsement, while technically giving access to universities, is no guarantee of entrance to higher education. Ironically perhaps the biggest advantage of a Matriculation Board endorsement is the coherence it gives to the qualification through its insistence on subject grouping. A growing trend within higher education is to apply additional admissions or entrance tests. It is widely felt that this is a negative practice as it undermines the credibility of the Senior Certificate examination and adds to the already untenably high cost of our education system. Furthermore, these additional requirements often detract from the transparency of the admissions process and could easily be perceived to be yet another way of raising entry barriers.

A cost-effective and acceptable alternative is already being trailed at some universities. This system involves using existing examination data and manipulating it to show the ranking of candidates within an examination centre. Borderline candidates with similar subjects and results can then be compared on the basis of how well they have performed in relation to their peers rather than in often unfair competition with more advantaged candidates at other centres.

For example, a candidate who came in the first quartile in a particularly disadvantaged school, rather than a candidate achieving the same score, but performing in the last quartile of an advantaged school, could be selected on the basis of inferred determination, ability and the will to succeed within a cohort of equally disadvantaged learners. This approach is acceptable, as it is a no-cost alternative to separately developed admission tests. Since it uses existing examination data, it can be demonstrated to be fair and transparent. Furthermore, this could help to encourage learners with consistently poor results to work harder towards admission into higher education. In doing so it would underline the importance of the Senior Certificate as it would be these results that are used.

As attempts at an open admission policy in higher education have manifestly failed, the most reliable and cost-effective solution must be data generated from a single exit exam after Grade 12. Tertiary institutions should retain the right to admit those that they feel have the best chance of success, but only in line with the Government's policy of redressing the inequalities of the past. Consequently, an admission policy and any testing or selection that accompanies these should meet or exceed the requirements generated in current policy, including the SAGA Act and the Further Education and Training Bill. Ultimately, this may mean the redesign of the Senior Certificate examination with all the implications for curriculum, learning and teaching redesign, towards the type of assessment embodied in the concept of the Further Education and training Certificate (FETC).

It is recognised, however, that the process of redesigning the Senior Certificate could be a lengthy one. In this instance and on an interim basis, admission tests should not merely be condemned and written off. Rather, tests that strive to promote equity and inclusivity should be monitored for their positive contribution and to prevent them from becoming parallel systems and as such undermining the Senior Certificate examination.

An interesting proposal in this regard was recently tabled at the SAUVCA workshop on the Senior Certificate Examination on 29 January 1998. The essence of the proposal, contained in the document "A South African Centre for Admissions Testing - SACAT" is that tests in the broad skills areas of academic literacy (in the medium of instruction) and numeracy should be developed by the HE sector. These tests, which will be non-syllabus based (i.e. they will not compete with the Grade 12 syllabi) should be written as additional papers by all Senior Certificate candidates as part of the examination. The HE sector will bear all costs in this regard, and exercise all control over the setting and marking of the papers. Results of these tests will be supplied to all HE institutions who can use them as they see fit (e.g. for decisions on candidates on the waiting list, for alternative access routes, for financial aid allocations, etc.). It is envisaged, however, that they will be used as additional, not alternative, information in terms of the Senior Certificate. Results will also be supplied to the national and provincial education departments. It is the view of the Committee that if such tests are to be introduced, their contribution to a system of Quality Assurance at Grade 12 level should be assessed.

In this regard the Committee recommends that the SACAT proposal be piloted in one province during the 1999 examination cycle. SAUVCA should be asked to raise the necessary funds for this. It is further recommended that the Matriculation Board take primary responsibility for the project, in close liaison with the Department of Education.

2.3.3 The accreditation of state, private and foreign examination boards/ assessment agencies that offer qualifications at Senior Certificate level

Coherence and reliability are important principles when one moves into a situation with multiple examination or assessment bodies. From a learner perspective multiple bodies may give increased access and choice, but from a national qualifications perspective the downside could well be the unevenness of products and services and the unreliability of results. There was little, if any support from the interviewees for a single centralised examination body for the whole country, but there was also very little support for the invasion of this country by foreign examination bodies. These bodies were thought to bring confusion to learners as well as lacking an understanding of the needs of the South African learner and society.

Many people also queried the capacity or need to have separate examination bodies for all nine provinces. Examinations, over the past two years, have clearly demonstrated that there are many differences in the capacities of the various provincial departments.

The Committee recommends that examination or assessment bodies should meet stringent and clear national requirements before being accredited to assess learners. These requirements should be derived from existing education and training policy. Examination or assessment bodies should demonstrate that they have met all these requirements and thereby further the agreed objectives of current policy. Differentiation should be made between those bodies involved in examining or assessing and other bodies who should have a responsibility to moderate. Currently the examining or assessing bodies comprise the nine provinces, the Independent Examinations Board (IEB), the technical college section of the Department of Education and two other bodies registered by SAFCERT. The Committee feels that the registration of the two examining bodies with very small numbers of candidates should be reconsidered. It is suggested that the thirteen examining bodies could be decreased to six or seven over the next few years.

A further important recommendation concerns the possibility of confusion and duplication in the FET band regarding the proposed registration of private providers contained in the FET Bill. Many workplaces that are not specialised training providers will be delivering learnership qualifications. As a result they could be made eligible to register as private providers. This situation is unacceptable to the bulk of employees who would find the registration criteria too restrictive. The inevitable result will be that their training initiatives will cease, which will have a detrimental effect on the development of skills in the country.

It is therefore proposed that registration with the workplace ETQA umbrella body should obviate any obligations to register as a private provider under the FET Act, once promulgated. However, private providers that deliver qualifications, deemed to be the equivalent of the formal education sector, should be required to meet registration criteria as outlined in the FET bill.

2.3.4 Consolidated Recommendation,

• The number of examination bodies/assessment agencies should be regulated by the imposition of stringent requirements derived from existing education and training policy. The existing 13 examination bodies currently recognised by SAFCERT should immediately be reduced to 11 based on a suggested minimum number of candidates of 500. The Department of Education, in consultation with the existing examining bodies, should investigate the possibility of pooling some of the resources to conduct the Senior Certificate examinations.
• Certain functions of the Matriculation Board of SAUVCA will be taken over by the FETQA in liaison with the Higher Education Quality Council. SAUVCA would continue to act as a spokesperson for the unified position of universities, but the function of determining criteria for endorsement for university entrance would be negotiated through SAUVCA's membership of the relevant committees of SAGA, which establish rules of combination for FET Certificates.
• The proposals contained in the SACAT document regarding the development of tests in the broad skills areas of academic literacy (in the medium of instruction) and numeracy, and their inclusion as additional papers by all Senior Certificate candidates as part of the examination, should be piloted in one province during the 1999 examination cycle. It is further recommended that SAUVCA be asked to raise the necessary funds for this and that the Matriculation Board takes primary responsibility for the project, in close liaison with the Department of Education.

The investigation revealed several serious problems regarding the use of information derived from the Senior Certificate examinations. While in the long-term the high failure rates can only be addressed through the improvement of schooling in general, several steps in terms of assessment-driven reform can be taken immediately, which will improve matters.

2.4.1 Criteria for an effective examination feedback mechanism

Eckstein and Noah (1993) suggest three criteria for a good feedback mechanism. They are as follows:

• It informs teachers and students well ahead of time of changes in the content of examination questions and of the nature of skills being tested (information before writing the exam).
• It analyses performances to determine what errors students make and why (performance analysis).
• It requires the examination body to suggest ways to improve the teaching of skills that require mastery of difficult concepts (suggestions for change).

These three facets of a good feedback mechanism were borne in mind in the analysis of the performance of the Provincial Examination Sections in terms of using the Senior Certificate examination to improve teaching and learning in the province. In addition, issues relating to Human Resources (including levels of capacity and understanding in respect of standardisation and normalisation procedures adopted by SAFCERT) were investigated.

The following steps were followed to gather data for the analysis:

• Provinces were requested to send copies of their examiners' reports to the committee. These were then studied to assess the extent to which, in their present form, they are functioning in ways that meet the above criteria.
• Likewise, provinces were requested to produce copies of the statistical information they would have produced for SAFCERT regarding the 1997 examination.
• The Examination Sections of all nine provinces were visited and interviews of up to two and a half hours were conducted, during which a wide range of relevant issues was discussed. Appendix C contains the topic areas that were agreed on and which form the framework for these discussions.

2.4.2 Human Resource Issues

2.4.2.1 Personnel vacancies

In all provinces, key personnel vacancies were identified. The examination sections are understaffed and generally some 50% of the posts have not been filled. This has seriously hampered the smooth functioning of examinations. In most provinces persons from other sections have been seconded to perform examination functions. Such staff do not consider this as part of their duties and are therefore de-motivated. In some provinces it was found that temporary appointments had been made, but the appointees had not been trained. Despite their lack of training these people were performing very sensitive and confidential examination-related functions.

The post structures and organograms vary from province to province. Provinces with large numbers of candidates are certainly under-resourced. In some provinces, the examination and curriculum sections are combined under one Director, while in others the head of examinations is either a Director or a Deputy Chief Education Specialist. In one province the examination functions are the responsibility of two Directors, one professional and one administrative.

These wide variations from one province to another require further investigation, preferably by the Public Service Commission, so that an equitable and rationalised structure can be instituted in each of the provinces.

2.4.2.2 Working relationships between the Curriculum and Examination Sections

It seems clear that any move towards promoting and maximising the formative educational contribution of the Senior Certificate will require a radical re-thinking of the relationship between the Curriculum Section (including Quality Assurance) and Examination Section in the provincial education departments. An example of the present dysfunctional relationship is as follows: in one province, the Examination Directorate sent all the examiners' reports to the Curriculum Directorate (five kilometres distant), intending these to be distributed to regions and subject advisors. Instead, the Curriculum Directorate simply returned the examiners' reports to the Examination Section, where they remain to this day. Moreover, during the discussion in which this situation was revealed, neither of the sections appeared willing to take responsibility.

A closer and mutually beneficial working relationship between these two sections or directorates could be achieved in a number of ways. As these hinge on the use of examiners' reports and the involvement of subject advisors, they are discussed in paragraph 2.4.6.1 below.

2.4.2.3 Understanding of standardisation/normalisation procedures

It is clear that in most of the provinces there is little understanding of the standardisation and normalisation procedures as determined and applied by SAFCERT. Provinces need to build capacity in this regard and appoint suitable personnel who will be able to analyse raw scores in the various subjects and make appropriate recommendations to their provinces. In order to do this, it is essential that key personnel be trained to analyse and interpret Senior Certificate statistics.

Furthermore, the manner in which the norms are determined by SAFCERT needs to be a far more open and visible process. Currently not all examining bodies are represented on SAFCERT, which nominates the committee that deals with statistics.

2.4.3 Statistical reports

A great deal of statistical information about the examination is available from Q Data. This includes subject statistics (numbers of As,Bs, etc. for each subject for the province as a whole as well as per school) and rank order of schools in terms of pass percentage. In addition, some provinces obtain the tapes from Q Data and do their own analyses in respect of performance of each of the components in a subject (i.e. the different papers, and the oral and/or practical components) with a view to providing detailed feedback to subject advisors and schools. In addition, and on request from particular schools, one province supplied information based on statistical analyses of each question within a subject in 1997. This yielded valuable information about matters such as syllabus coverage, but its sustainability in terms of personnel capacity is open to question.

Unfortunately, much of this information does not reach the institutions in the provinces. This means that valuable feedback, on which improvements to teaching and learning could be based, is wasted.

2.4.4 In-school/internal assessment

2.4.4.1 Continuous assessment

In all provinces, it is apparent that there are concerns about the introduction of continuous assessment. Moreover, there are quite deeply rooted philosophical differences: some provinces believe that there should be a fairly close (at the least, a definable) relationship between continuous assessment and the external section of the examination. Others hold that fundamentally different abilities are measured in the two modes of assessment (e.g. continuous assessment might incorporate the ability to work within a group, and group work, whereas external assessment is by its nature an individual and 'real-time' endeavour) and that the two sectors therefore need not relate to each other in any pre-determined way.

There is, however, general agreement that there is a tendency for continuous assessment to be used to inflate marks and that this is uneven. Some schools, for example, record a far higher average mark for continuous assessment than they obtain for the external portion, whereas others are more closely aligned. This results in an inequitable situation, where some candidates are advantaged and some disadvantaged, depending on the school they attend. There is, therefore, an acknowledged need for some monitoring of the relationship between internally- and externally derived marks. It is our recommendation that continuous assessment not be introduced until such a monitoring mechanism is in place.

This mechanism, it was suggested, could have two major components. Firstly, its introduction would depend on the successful introduction of the appraisal system, which is currently under discussion by the teachers' unions and the Education Department. This need is clearly demonstrated by the current difficulties reported by provinces in gaining access to schools and classrooms to moderate orals and practicals. (Many provinces admitted that this was only happening in the minority (mostly ex-House of Assembly) schools. Until educators, heads of departments in schools and principals can be held accountable for their classroom practices (for example via the appraisal system) it is difficult to imagine how effective moderation could take place. Secondly, the successful introduction of continuous assessment would rely on a statistical programme which would work within certain accepted parameters in checking the relationship between the two scores making up the final Senior Certificate marks. It could, for example, be agreed that the continuous assessment component could not differ more than 10% from the externally derived section.

Some of the dangers of an over-hasty (i.e. prior to the establishment of moderating systems) introduction of continuous assessment are: the erosion of stakeholder confidence in the meaning and reliability of the results; the lack of equity between schools and centres in that the inflation of scores at one centre will by comparison disadvantage learners at a more scrupulous contra; and the likely eventual denigration of the value of continuous assessment in the minds of learners, educators and the public.

2.4.4.2 Oral and practical examinations

As mentioned above, provinces have not found it possible to moderate all oral and practical examinations, although some provinces have established impressive systems for coping with a seriously under-resourced function. The situation is particularly worrying where there are no subject advisors to conduct the moderation and where schools simply submit marks to the provincial department out of totals which do not accord with the official total. In one province, it appears to be a frequent occurrence for educators to report that learners gained 72 marks out of a possible 50 marks. In these cases it is left to the department to decide what this 72 might mean.

A further point in connection with oral and practical examinations is the situation of part-time candidates, who do not complete this crucial component. It is the view of the Committee that if it is possible for candidates to present themselves for examination at a centre, it should be equally possible for them to present themselves for assessment in the oral or practical mode and that this should be established as soon as possible.

Finally, there should be uniformity in mark allocations, both between related subjects (e.g. all First Languages) and between provinces in the same subjects.

2.4.5 Examiners: selection, training and functions

2.4.5.1 The selection of examiners

At present it seems that very few educators apply to be examiners. It is generally agreed that wider advertising is necessary. In addition, until now only subject expertise has been seriously considered in the selection of examiners. As the function of the examiners' reports widens, however, it is necessary to include additional criteria, which may include the following:

• Knowledge of the subject
• Assessment expertise
• Teaching experience or familiarity with the teaching context

• A high degree of language proficiency in the language or languages in which the subject is being written. (This arises as a consequence of the fact that the overwhelming majority of candidates in every province write their non-language subjects through the medium of English, but, on the evidence of the 1997 examiners' reports at least, the English language proficiency of some of the examiners (and presumably also of the markers) is decidedly questionable. For example, what can one make of comments such as Because is was a open questions learners didn't know what to write and how much of each happening and the more"?)

2.4.5.2 The training of examiners

As it is unlikely that, aspirant examiners with all this expertise will be found, even with wider advertising. It will be necessary to provide thorough training. It is recommended that costs be kept to a minimum by the running of joint (i.e. non-subject specific) assessment workshops that introduce examiners to the many ways in which knowledge and skills can be elicited and the production of materials in this regard.

Past examiners' reports, in their own and cognate subjects, should be the subject of focused discussion, and it should be made clear to the examiners that their reports in turn will be read by the whole group. The use of the reports (i.e. their use by classroom educators) should be clarified and discussed. In addition, examiners should be made aware of the performance of learners in their subjects in the past, through the discussion of detailed statistical reports.

The issue of language proficiency is a difficult one. It is a sensitive area in which improvement is a lengthy process. However, given its importance to the assessment process, it must be taken seriously and monitored on appointment. Note that it is not the intention to suggest that only first language speakers of the medium of instruction be appointed, but that assistance and support must be available for those examiners who are not fully proficient in that language.

2.4.5.3 The functions of examiners

In terms of forming an important link in the examinations washback chain, examiners need to understand their function as not only looking backward (i.e. assessing past learning) but as shaping future actions on the parts of educators and learners. This is obviously not a new idea and judging from the 1997 examiners' reports, most examiners do subscribe to it. However, the usefulness of their reports in these terms is highly questionable. For example, how helpful is it for an educator to know that in the Accounting HG examination, Section C was answered excellently by some candidates and very poorly by others"?

In order to assist examiners with their feedback function, examiners' reports will need to have carefully structured and explicit Action Sheet" sections, as well as sections for examiners to give reasons for poor performance. For example, in one History HO paper, one examiner commented that very few candidates attempted a particular essay in History. The reasons for this were not given - perhaps it was the only essay requiring evaluation of evidence rather than simple description? Or was it on a particularly difficult area of the syllabus? Knowing more about this would assist educators in preparing their learners for future examinations.

2.4.6 Examiners' reports

It is evident that there is a huge variety in both the quality and the use of these reports. In some provinces, an examiner can simply write "N/A" (or score a line through) all the sections on the report she or he is supposed to complete and sign it with a flourish at the end - and no censure follows this. In such provinces, unsurprisingly, the reports play no part in the feedback from the examinations.

In other provinces, a greater emphasis is placed on the reports and a good deal of insight about the state of teaching and learning in the province can be gained from the reading of all the reports.

2.4.6.1 The Format and Function of Examiners' Reports

There is a need for clearly differentiated functions in reports. Currently these tend to emphasise technical matters (e.g. "candidates must write the number of the correct option, not the answer itself') and where more qualitative information is given (e.g. "candidates can't apply their knowledge") no assistance or advice is given in terms of actionable recommendations for classroom practice and/or policy. Sometimes this can be achieved simply by giving examples ("instead of stating xxxxx, candidates were required to show how xxxx impacted on yyyyy") At other times more fundamental assistance is required, and this will need to be identified by the curriculum section.

What can be achieved by:

• ensuring that the subject advisors, who will be tasked with using the examiners' reports after the examinations as tools to improve learning and teaching, are included in the training given to examiners and moderators;
• ensuring that subject advisors are involved in designing examiners' reports;
• ensuring that the examiners' reports contain a brief version of the questions (e.g. "In response to a question requiring candidates to discuss reasons for the Cold War, many candidates....n) as it seems that many educators read the reports some time (usually at least seven months) after the paper has been written and do not keep copies of the paper. The examiner's report is therefore often decontextualised and loses much impact.
• requiring that subject advisors evaluate the effectiveness of the examiners' reports and suggest how this may be improved on an annual basis. They should account for the use of the reports (e.g. if a report has recommended that a workshop be held on a particular topic, it is the subject advisor's duty to see that this happens or to justify why it was not possible or appropriate);
• ensuring that the Head: Curriculum Section takes responsibility for ensuring that the above takes place.This will include the production of an overall 'oversight' document, listing all the recommendations contained in the reports (this will be facilitated by the new format and orientation of the reports) - This document may form the basis of the monitoring of the use of the examiners' reports.

2.4.6.2 Other issues arising from examiners' reports: the examining of languages

It is very evident from the examiners' reports that learners are judged by different criteria. In his report on the English First Language HG Paper 1 (Writing) in one province, the examiner asserts:

"The aim of a sound educational system must be to equip learners to grapple with issues and make sense of the world in which they find themselves so that they can more easily negotiate a rapidly changing and increasingly complex future. These same aims inform Paper 1."

These sentiments are echoed in nearly all the examiners' reports, across the provinces, for English First Language. Sadly, they are not present in the reports of the examiners of African First Languages. As an examiner of Setswana First Language HG suggests:

"The problem is that candidates don't know how to interpret their language because the teacher/educator doesn't know whether he/she is doing the right thing or not, because he/she has not been advised about their language".

Indeed, it seems that the teaching and assessment of African First Languages is aimed at language maintenance and language development, rather than at cognitive development as is the case with English First Language.

We argue that these differences seriously disadvantage students who study an African language as a first language and who do not study either Afrikaans or English First Language. The basis for our argument is that it seems that the abilities sought in the examination (and, by implication, in the teaching and learning off English or Afrikaans First Language are those of argument, extended discourse, synthesis, evaluation, etc. In other words, expository writing is required - and it is expository writing (coupled of course with creative writing) which most effectively stimulates cognitive growth in learners. If the emphasis in learning a language remains at the oral level (however sophisticated and challenging), learners lose out on a powerful tool for the organisation and systematisation of their thinking. The evidence in the examiners' reports (of African First Languages) leads us to believe that the ability most prized in learners is knowledge of the rules (grammar) of the language. Its use in the sense of what Cummins terms "Cognitive Advanced Language Proficiency" (CALP) is not accorded primacy.

The implication or consequence of this (for almost 90% of the country's Senior Certificate candidates) is that CALP skills, if they are to be developed, will need to be done so in the candidates' second language. This is a considerably more difficult and unreasonable task than developing these skills in one's first language Indeed, research suggests that it is extremely ineffective to attempt to develop them through a second or additional language. It is tempting to think that because learners have to study most of their subjects through the medium of their second language (usually English), they should acquire CALP skills through the medium. Again, however, it seems that acquiring these skills through one's first language and then using them in the second (for example, by studying subjects through that medium) is actually more effective. Whatever the merits of this argument, it is undeniable that every opportunity should be grasped to develop learners' CALP skills. At present it seems that African First Languages are not being used effectively as a vehicle for this.

The Committee therefore recommends that:

• the examiners of languages (all languages) should undergo training together;
• their reports should conform to an agreed format which requires commentary on expository writing;
• subject advisors should be required to seek evidence of such writing and related activity in their moderation procedures (or, when the appraisal system is introduced, in classroom practices); and
• as soon as possible, generic syllabi should be introduced which emphasise similar skills irrespective of language.

2.4.7 Examinations and public accountability: harnessing the media

This should be seen as part of the process of involving parents and the community in education through demystifying the Senior Certificate and using it as a vehicle (by harnessing its high public interest value) to engage the public in the major debates surrounding the examination.

The need for urgent attention to be paid to the coverage of the examination process is evident. For example, releases on leak scandals routinely take over from any kind of serious reporting on the examinations. Articles in the press with headings like 'Exam Paper Syndicate are characterised by outrage and sensation, not by what the public has a right to expect, i.e. serious and constructive analysis. This is not to point a finger only at the media, though. They report on what their public wants. There is certainly no tradition in South Africa of high-quality feedback on public examinations and thus no expectation of such feedback. Indeed, legitimate questions could be asked about educationists' silence on these matters, but it can be argued that their involvement will follow once important questions have been raised.

A recent nationally based press clippings survey (Yeld et al; 1997) revealed that press coverage of the school-leaving examinations leaves much to be desired. Of the 96 articles surveyed in the period December 1995 - January 1997, only 2 were based on the views of an official examiner! Teachers' views featured in only 23% of the articles, whereas 48% drew on information provided by spokespersons of the Department of Education. 45% were classified by the researchers as 'sensational' (examination leaks, evidence of cheating, marker strikes, etc.), while only 21% could be classified as 'stimulating debate', in other words, as containing more than one point of view, or presenting different arguments or theories. Similarly, only 23% of the articles were classified as having a serious level of analysis, with 27% being extremely superficial.

Inevitably, this kind of coverage ignored the content-related issues of the examination system in favour of more saleable scandals. This is not to say that appearance of the examination papers on the market and related instances of corruption or incompetence should not be of public concern and roundly condemned - of course they should. But such outrageous incidents could be used constructively if efforts are made to determine what exactly went wrong so that the mistakes are not repeated.

The problem is that sensationalising examination problems only adds to the general confusion accompanying the transformation of the educational system and feeds the distrust surrounding the integrity of the results. It contributes nothing to raising public awareness and improving debate about school-based issues as these are reflected in the examination system. For example, what more do we know and understand of the school-leaving examination beside the fact that there were leaks and that the pass rate for the school-leaving certificate was about 47% in 1997? Do we know what skills were tested in the examination papers or how the papers were set? Do we know whether candidates' skills improved or deteriorated - are there trends that the public should know about? What do examiners have to say about the matriculation examinations? And what did teachers expect from Matric '97? Are the exams simply a giant hoax, as ex-DET students (and many of their teachers) have long suspected, or is it possible to build public confidence through a careful campaign of public education - starting with the press?

In interviews with teachers and students, one of the most striking features to emerge was that a disturbingly high proportion of interested parties view the Senior Certificate as a lottery: how you do is arbitrary and unrelated to either effort or circumstances. It has also been suggested that this view of the Senior Certificate is subtly reinforced by some teachers who are not keen to see their learners' poor performance ascribed to inadequate syllabus coverage or teaching and find it convenient to lay the blame on The examination". This clearly has major consequences for the way in which candidates prepare for the examination - if performance is unrelated to effort, why work hard?

De-mystifying the examination would contribute towards exploding some of the myths that surround the Senior Certificate and clear the ground for more serious approaches to preparation.

The kind of information that could be helpful in this regard could include the following:

• An explanation of the process - e.g. giving a time-line, appointment of examiners, setting and submission of papers, etc. How is the marking done? What checks and balances are there?
• An idea of the size of the operation (How many candidates in Province x?, How does this compare?, How many candidates are repeating? What are the 'big' subjects? Are there any new subjects, e.g. Tourism? Are there major new developments in the Senior Certificate itself?)
• The whole issue of examiners' reports. It should be made clear how these contain valuable information that will be fed into the schools. (Warn parents and pupils to watch out for these and make sure their teacher/the teacher of their children uses these.)
• User-friendly versions of core-subject examiners' reports. For example, the issue of map-work in Geography that seems to be completely omitted in most schools, and contributes greatly to high failure rates, or the lack of practice in expository writing in many language subjects - these should be sensitively discussed and suggestions should be made as to how parents and learners could assist in addressing these problems. The important point is that where there are low pass rates in a subject, it is essential that the public and the learners should know that this is the result of serious problems in the teaching and learning of the subjects concerned, and not some statistical artefact unrelated to what happens in the classroom. It is equally important that the public and learners should know that these problems are not insoluble, but that they can be addressed once they have been identified.

Our recommendation in this regard is that the Examination and Curriculum sections should work closely together in order to produce timeless, readable, educationally useful accounts of the Senior Certificate in their province. These should begin to appear in the local press (or on chat-shows etc.) before the annual examination cycle (i.e. September) and afterwards (ideally, in the week that the results are reported). In addition, the use of television and radio to inform and engage the public on issues surrounding the Senior Certificate should be investigated.

2.4.8 Consolidated recommendations

• The Examination Boards of the various examining authorities should provide detailed feedback on performance in examinations to all the role players, including subject advisors and schools. There should be a clear mechanism for monitoring the performance and effectiveness of this function. The Examination and Curriculum sections should accept joint responsibility for the distribution of information and, through careful tracking of the dissemination process, be able to account for and be held responsible for this essential function. This should take place with immediate effect.
• A close and effective working relationships between Curriculum and Examination sections need to be established as a matter of urgency. The Department of Education needs to assume a more proactive role in this regard.
• Capacity needs to be built in terms of the understanding and analysis of the standardisation and normalisation procedures. SAFCERT should actively train personnel who are involved in the statistical processes in provinces. This requires immediate attention.
• The Department of Education should institute an appraisal system which is agreed upon all by teacher organisations. The urgency of such an instrument cannot be over-emphasised, particularly in view of the commitment to introduce continuous assessment in all provinces as a major step towards re-establishing a culture of teaching and learning. In addition, without a functioning appraisal system, it is unlikely that the present system of moderation used for practicals and orals can be relied on.
• In the short term, it will be essential to introduce a statistical mechanism for ensuring the validity and reliability of continuous assessment. This matter should immediately be investigated by SAFCERT, in conjunction with IPEC.
• The Curriculum Section of the Department of Education should ensure that as from the 1999 Senior Certificate examination, there is uniformity in the mark allocations given to orals and practicals across cognate subjects (e.g. all First Languages (HG) should have the same mark allocations). In addition, consideration should be given to the recommendation that the marks of part-time candidates should be constituted in the same way as full-timers, that is including an oral or a practical component.
• The Examination Boards of the various examining authorities should ensure that the posts of examiners and internal moderators are widely advertised, and that the criteria for the selection of examiners are broadened to include a demonstrated proficiency in the languages in which the subject is to be written, and assessment expertise.
• The job descriptions of subject advisors need to be reviewed with a view to including greater responsibility in the examination process.
• Examiners' reports need to be carefully designed with special consideration of their influence on teaching and learning. In this regard, examiners should receive specific training based on past reports as well as new formats. Examiners should collaborate with subject advisors, who, in addition, should be trained and included in the marking teams. The curriculum section should manage this.
• The curriculum sections should be tasked immediately with doing an overall qualitative analysis of the examiners' reports in their provinces and making recommendations for their improvement and use. It is this section's responsibility to ensure that the reports are used maximally and that their use is reported on. I n cases where there is no subject advisor, contingency plans should be developed and implemented.
• It is urgent that the examining of all first languages be brought in line insofar as this is possible with current syllabi. This recommendation is covered in some detail under 2.1.10. From the perspective of assessment, however, all moderators and examiners of first languages should be trained as a group. The examiners' reports ought to be circulated and discussed in a group, using the media to maximise feedback. The curriculum section should undertake, as a matter of urgency, the steps needed to fulfil this function.
• Further investigation of the variations in post structure in the examination sections needs to be undertaken by the Department of Education. Clearly, the wide variations in structure militate against good administration and this requires immediate attention.
• The communication sections of the Department of Education as well as in the provinces should develop strategies for using the media to maximise feedback on examination issues and their related curricular implications.

2.4.9 Summaries of Visits to Provinces

Each provincial department was visited. In almost all instances participants in the discussions comprised units that are attached to Examinations, such as Curriculum, Subject Advisory/Learning Facilitation, In-service, Statistics and Inspectorate or Development Officers. The discussions revolved around agreed question areas (see Appendix C).

Human Resource issues

• In almost all provinces key and critical posts are not filled. This includes head offices and lower levels. This has a negative impact on the smooth running of examinations.
• With the exception of North West that has two branches of examination centres, all provinces have centralised systems that lack key personnel.
• Personnel are brought from all over the department and some departments even 'hire' temporary general assistants "off the street". Because of these HR problems and infrastructure constraints some provinces engage the IEB, even though it is costly.
• The majority of provinces have directorates that combine curriculum and examination sections, although they often do not work together closely.
• Some provinces expressed unhappiness with the organisational structure and the location of the examination section in it. For example, different units that deal with examination matters are not in the same directorate or chief directorate.
• It is apparent that most organisational structures are not informed by realities on the ground, i.e. the implications of the core business of the department.
• Security seems to be a major problem in all provinces. Almost no staff members (in examinations) have received security clearance.

Selection of examiners and examiners' reports

• The appointment of examiners is done by the professional component of the examination section and the examination board, where they exist.
• In both of the above-mentioned cases there is participation by all stakeholders, other units in the department, teachers" organisations/unions, etc.
• The length of the contract for examiners varies from province to province - it can be one, two or three years.
• Provisions of Resolution 6 are complied with in the selection of examiners.
• Impost examiners of African languages are linked to universities and it is felt that they are out of touch with the languages.
• Master copies of examiners' reports are usually, but not always, sent to regions or districts for reproduction and dissemination. Because of logistical/infrastructural differences amongst provinces, not all schools receive the reports. For example, if there is no subject advisor in a particular subject, that report is not disseminated.
• Most provinces state that compiling examiners' reports is a major administrative task that requires resources.
• Some provinces do not make use of examiners' reports. Neither the examination section nor the curriculum section sees it as its responsibility to facilitate the general use of examiners' reports by the relevant stakeholders.
• The departments expect subject advisors and members of the inspectorate to be involved in the dissemination of examiners' reports.
• There is general agreement that examiners' reports should be read together with question papers.
• The quality of examiners' reports should be improved, particularly in the area of clear suggestions for classroom action.
• Some flaws have been identified in the system, amongst other things, some examiners never submit reports and if they do some (reports) are incomplete because it had been done in a hurry. In all cases there does not seem to be any concrete strategies for censure.
• It is argued that some examiners simply condemn teachers without providing/giving constructive criticism.
• Attempts are made to cluster schools that did well in examinations with schools that performed poorly, but this is not always successful.

Higher Grade and Standard Grade

• The provinces feel that this distinction is no longer tenable and should be rethought.
• It is apparent that most candidates take subjects in the HO, thinking this is the norm.
• It is strongly felt that much support should be given to teachers regarding the handling of differentiation.

• The general consensus is that African languages tend to be assessed too rigidly, i.e. the major emphasis is on formal syntactical/linguistic elements.
• Some examiners have very poor language proficiency. They are, for example, not able to express themselves well in the language of the papers they set or mark.
• It is suggested that examiners of all first languages be trained together.

Use of statistical reports

• There are only a few provincial departments who have the capacity to use statistical data effectively.
• Once results are out the examination sections send copies to all relevant sections.
• In some instances poorly performing schools are identified and targeted for remedial action.

• With the exception of a few, provincial departments do not have personnel who are knowledgeable in this regard. Some provinces have no capacity for standardisation at all.
• The absence of persons who understand the standardisation process forced some provinces to depend on'experts' from outside the system.
• None of the provinces are happy with the norms submitted by SAFCERT and some have reacted in writing.
• Provincial teams that meet with SAFCERT during the normalisation process are generally given clear mandates from the management of the departments.
• It is strongly suggested that local conditions be taken into consideration when mark adjustments are done.

• All provinces support continuous assessment up to Grade 12, but the problem is that they do not have enough staff to supervise and/or moderate continuous assessment.
• There are philosophical differences on continuous assessment. There are provinces that believe there should be a close relationship between continuous assessment and the external section of the examinations. Others argue that fundamentally different abilities are measured in the two modes of assessment.
• The general feeling is that oral marks (and even year marks) are inflated and are at times far above the provincial average.
• It is felt that the Department of Education should have a moderation instrument in place.
• Not only will a moderation instrument serve as a tool for training, it will also help to eliminate big discrepancies between external and internal assessment.
• Not all schools will have their marks moderated, because of personnel shortages. Some schools even refuse to co-operate.
• In some provinces, e.g. the Northern Province, oral and year marks are not yet part of examination results.

.Subject advisory/Learning facilitation

• Generally there is a dearth of subject advisors (highly pronounced in key subjects). This is exacerbated by the moratorium on the filling of posts.
• Subject advisors receive copies of examiners' reports and are expected to ensure that schools receive these reports.
• There is alleged bureaucratic red tape where, for example, subject advisors must first obtain permission from a region or district before work can be done.
• The monitoring mechanism is generally chaotic, with very few exceptions. There are still 'no go' areas, differing in extent from province to province.
• It is evident that in some provinces subject advisors just do not go out to schools.
• The suggestion is that subject advisors should accord equal attention to Grades 10 and 11 and not only to Grade 12 because it is externally examined.

Production on LSM (Learning Support Materials)

• In most provinces book lists are compiled after panels on which selected expertise of stakeholders serve, have studied advertisements and screen books or manuscripts. Schools then select books from a range of approved titles.
• In the Western Cape, for example, schools submit their choices directly to publishers and the department pays, while the stakeholders are the selection process (This means the Western Cape provides per capita funding for books.)
• It is generally reported that where the textbooks do not cover the syllabus, documentary materials and workshops for teachers are provided. (This has not been confirmed.)

Reasons for poor performance

• Disruptions and boycotts in schools.
• Lack of commitment by some stakeholders (e.g. Ieamers, teachers and parents).
• Uncontrolled Voluntary Severance Packages.
• Introduction of common examination papers with no proper preparations.
• Some officers appointed beyond their competence.
• Abnormal teacher:pupil ratios.
• Standard of assessment in lower grades an inappropriate preparation for Grade 12.
• Wrong subject choices as a result of peer and community pressures.
• Some learners insisting on writing a seventh subject, even if the subject is not taught at the school,
• Teaching and learning in a second language.
• Mathematics and Science teachers that are often underqualified and a lack of laboratories.
• Lack of motivation, low morale and poor work ethics.
• No control over candidates that repeat - there is generally no record of how many candidates are repeating.
• No concerted effort is made to deal with matters of feedback on examiners' reports. If feedback mechanisms do exist, they are quite unsystematized.
• The tendency (it has become a culture in certain circles) that teachers use only the oral mode when teaching, instead of recognising that reading and writing skills are very central to the Senior Certificate examinations.
• Poor management of schools.
• Learners who do not identify with the values of schooling.
• Teachers who are out of touch with the values of the youth.
• Drunkenness, absenteeism and non-observance of full school hours by both teachers and learners.
• Heads of departments do not always understand and carry out their responsibilities.
• Learners who register for the wrong subjects or are promoted when they should not have been.
• Chief examiners who lack commitment.
• Educators who do not manage their own time and work properly.
• Teacher training in colleges is not quite up to scratch and needs serious attention.
• Educators teach subjects they are not trained for.
• Non-availability of qualified teachers for technical subjects.
• No link between practical work and theory.
• Malnutrition and poor living conditions.
• Location of schools and a general lack of resources.
• Poor career guidance.

• The Department of Education should ensure that SAFCERT has some teeth. Sometimes almost half the provincial departments do not attend the SAFCERT meetings to which they have been invited. A form of censure is necessary.
• SAFCERT must produce an annual report on each subject and give its opinion as well.
• There is strong support for the reintroduction of an examination fee.
• There should be a definite school-leaving age, but it should have a delayed introduction.
• The transformation of Senior Certificate examinations must be supported both financially and materially.

• The right questions on examinations should be asked with much sensitivity to context, e.g. increase or decrease in pass percentage may mean quite different things statistically.
• The press can be used (as in other countries) to give regular feedback on examination matters to the general public.

Other suggestions

• The focus must be on the required pass aggregate irrespective of subject choice(s), because emphasis on grouping appears to be excessive.
• Panel visits should be reintroduced but should not be structured as in the past.
• Examiners and moderators who are textbook writers should be removed from the panel because this creates a conflict of interests, which militates against good assessment.
• The Department of Education should provide a structured workshop/training programme on examination and assessment techniques.
• There is general agreement that there should be compensatory mechanisms for candidates using a second language as medium of instruction.
• There is a need for an investigation of the background of SAFCERT moderators. It is argued that most is drawn from universities and has no knowledge of what happens in metric classes.

The Information Technology sections of all provincial education departments were visited during the investigation, with a view to gaining an overview of provincial examination structures, with particular reference to the role of Q Data and the Examination Computer System. Summaries of each visit are contained in paragraph 2.5.4 below.

2.5.1 Features of the Present Examination Computer System (See Appendix D)

2.5.1. 1 Overview

The present examination computer system has been developed over a number of years. It is the property of the Department of Education and all major costs have been incurred during its initial development. It is therefore fully paid for as it operates at the present moment.

Q Data Consulting, the original developers, now has a nation-wide maintenance contract, which expires in September 1999. This includes programme and system changes, plus any additions requested by the Department, as well as the development of programmes for ad hoc queries that may come from Head Office or the provinces. These changes are charged out per request, at rates specified in the current contract.

The computer system consists of multiple Natural Adabas databases. The complete computer system and all software used are completely year 2000 compliant. The computer system is mainframe based and uses terminals or microcomputers with terminal emulation software. It is thus possible to use outdated computers to get access to the examination system.

The main computer system is used exclusively for the Senior Certificate examination, but other examinations can be added as the need arises. Thus a Grade 9 examination (such as the GETC exit-level qualification discussed in this report - see paragraph 2.1.8) can be run on the system at very little additional cost. The examination subsystems for Technical Colleges and Teacher Training Colleges are part of the current computer system and share some of the resources.

The system as it stands is suitable for the present examination format and covers all the standard functions and detailed requests as currently required. The system programme being open ended, can be modified to take into account any future changes in the examination process as they develop. However, the programme, at present, does not cater for year marks or continuous assessment. This will require additional work for the Western Cape as they currently use year marks as part of the assessment process.

According to users, the system, running as it does on Bureau Nucleus hardware, is both efficient and effective. During peak periods, such as December, the response time has sometimes (but not often) been perceived as slow. Recent and ongoing hardware upgrades at the Bureau should overcome this type of problem.

2.5.1.2 Programme Changes

All authorised requests for programme changes are reviewed immediately by Q Data. However, it must be understood that programme changes cannot normally be processed "overnights. This will particularly apply to the results programme which can take some considerable time to finalise, owing to any number of factors viz. developing parameters, available resources, IPEC/ITC approval, testing, etc.

For last year's examinations, the last two major changes to the Results Programme were only gazetted in November and December respectively. Most consultants would have refused to implement these changes at such short notice. However, Q Data, being aware of the significance of these changes, pre-empted them and initially at their own cost, started to develop these new result programme functions even before they were published in the gazette. Had they not done so it would not have been possible to meet the required deadlines. This is an example of the sort of commitment on the part of everyone that we met at Q Data to deliver quality service to the Department of Education.

Because of the time constraints, these programme changes could not be tested exhaustively and some errors were discovered during the examination run. These problems again received prompt action and solutions were put in place within a very short period of time.

2.5.1.3 The Service Offered by Q Data

The service rendered by Q Data, we believe, is of a very high standard. All the people at operations level were unanimous in their praise of Q Data and were more than satisfied with the service rendered by Q Data, the Help Desk and their consultants. The only concern is that duplication of requests might occur where the same report was requested at different times by more than one province and that each one of the provinces would have to pay for this same request. As this is possible under the present system, it is suggested that a revised administrative procedure, where all requests are centralised and then acted upon, be developed to prevent this from happening. However, whilst this may be a cause for concern, the committee did not find any actual proof of this occurring.

The cost of the service rendered by Q Data is not even market-related, but based upon the initial rates plus Consumer Price Index (CPI) increases, as agreed in the current agreement. CPI is below industry-related increases for IT professionals over the past three years. Any new contract, irrespective of supplier, will be based upon current rates, which have grown well in excess of CPI and inflation rates, primarily owing to a major skills shortage throughout South Africa. All major computer companies, as well as large end-users, also have to compete with salaries offered overseas in order to keep the necessary prime skills.

The Q Data consultants in the provinces are not cost-effective in those cases where they carry out the tasks that should be carried out by trained departmental personnel. For example, it is hard to justify a full-time Q Data consultant just for ad hoc queries. Some provinces use the consultant in the role of Systems Administrator, or even as part of the administration and planning of the examinations. Unfortunately a number of provinces have no alternative at present and an intensive training programme in order to increase (departmental) internal skills rapidly should be considered by IPEC.

The role of Q Data and the numbers of consultants vary from one province to the other:

2.5.1.4 The Help Desk

The Help Desk is an essential part of the whole system. Common programme changes are processed from there and the Help Desk supports all the provinces telephonically. The level of support required varies greatly, from the Northern Cape which relies entirely on the Help Desk, to KwaZulu-Natal which rarely uses it. The Help Desk also provides on-site system and administrative training.

2.5.1.5 The Systems Administrator

This is a key person in the successful running of the whole Senior Certificate Examination project, a process that takes approximately eighteen months from start to finish. Systems Administrators are in effect the project leaders and as such have the responsibility to ensure that in the region of fifty different activities, as specified in the System Manuals, take place timeously. Some of these activities occur daily, weekly or monthly. Others happen infrequently, but these are often fundamental to the success of the whole process.

Unfortunately, very few (if any) of the current Systems Administrators are fully conversant with, or have yet had time to get to know, their full job requirements. They therefore rely on local Q Data consultants and/or the Help Desk.

Deferent provinces also have different views on what their levels of seniority should be and this is causing discontent. These views are often based on a lack of funds or resources. Our recommendations follow, but bearing in mind the high level of responsibilities that the Systems Administrator carries we feel that all current levels need reviewing and revising upwards if the Department and provinces are to keep the staff of the required calibre in this highly critical role.

2.5.1.6 Provincial understanding of the system

The level of understanding of the system by the most hands-on users is satisfactory, given the short time that some of these people have occupied their current positions. Q Data provides training for users on specific functions of the Senior Certificate Examination system and sub-systems as needed. This is a chargeable item and has to be paid for by each province when required.

The users have access to the functions necessary to perform their duties and are trained on these functions. This means that the level of understanding of their duties is acceptable although they may know nothing about other functions. As their need increases, additional training can be carried out, normally via the Help Desk support team or the Systems Administrator.

As mentioned earlier, the level of understanding of Systems Administrators varies. Some were only recently appointed and are still on a learning curve. Apart from the Western Cape, due to constraints on budgets, none of the provinces has Systems Administrator back-up or plans for this essential contingency and therefore have to rely heavily on local Q Data consultants. Should any one of the Systems Administrators resign it could lead to major problems, particularly if this was to happen towards the end of the year or during the examinations, the marking thereof and the processing of results.

The level of understanding from management varies greatly. The programme is not a management tool where managers can extract management information easily. The managers using the system are limited to standard reports on the system. However, user documentation created by Q Data in the past is available.

The provinces, with one exception, have the capacity to conduct the Senior Certificate Examination effectively although they are all under tremendous strain as a result of personnel and/or hardware shortages. The Eastern Cape is the only exception. Whilst it has the personnel at present, activities, or the lack,thereof elsewhere within the province have adverse effects on both the morale and the physical capabilities to ensure a successful 1998 Senior Certificate Examination project in the Eastern Cape.

Taking into consideration the overall lack of budget, skilled personnel and equipment, most provinces perform far above expectations - but this cannot continue forever. The general feeling of the committee was that everyone in Examinations, at all levels and in every province, is committed to the success of the examination process. Unfortunately, far too many factors outside their control are coming into play and could eventually put the whole process in jeopardy. Apart from resource constraints and personnel shortages, these factors include failure to pay accounts timeously, a lack of commitment by senior levels to understand the gravity of their lack of action and a lack of recognition of the work done by the people in examinations and the need to reward them accordingly.

2.5.2 Shortcomings of Present System noted during Provincial Visits

2.5.2.1 Documentation

The Examination Computer System documentation comprises four extensive volumes of detail containing information on all the functions in use. Unfortunately it is difficult to find specific help and guidance as this documentation has expanded over a number of years and requires more comprehensive indexing. There is an on-line "Help" facility which is helpful whilst working on specific functions. Because of the overall complexity of the system, however, the available documentation needs to be made more user-friendly in order to assist people at operational level. Q Data is aware of this limitation, however, the cost of writing a brand new user-friendly Operations Guide for all functions at this stage would be prohibitive, and particularly as their existing system Maintenance Contract expires in September 1999.

They do, however, recognise the need, particularly as most Systems Administrators and users are comparatively new to their positions. If their Maintenance Contract is renewed, or extended, this is one area that should receive urgent attention.

Currently provinces can only obtain management information through standard reports or by requesting a new report. The first option is not flexible and the latter can be costly, as it requires the use of Q Data consultants and/or the Help Desk in most cases. One solution could be SQL- type query facilities. Failing that, flat file downloads of data, which can be imported into PC databases and manipulated locally, could be an option. There are some download functions available on the present system, but to use these effectively requires additional provincial support from the EMIS departments, as does the use of flat file downloads. IPEC has recently given Q Data the go-ahead to investigate more flexible and user-friendly alternatives.

2.5.2.3 Duplication of Requests

It is possible that the different provinces often request similar reports at different times, without reference to a central control point. This can mean that there is a duplication of costs, rather than cost sharing. The implementation of a centralised administration system would help to solve this perceived problem, or the implementation of a simplified process as mentioned in the previous paragraph.

2.5.2.4 Two sub-systems

In order to meet previous needs the main system is divided into two sub-systems. Sub-system A is for full-time students who are taking at least six subjects. These candidates also have oral and practical marks.

Sub-system B is for private candidates. These are candidates who are not attending a school on a full-time basis or who are taking less than six subjects. These candidates do not have oral or practical marks.

One problem with this approach is that some candidates cannot be classified into either of these groups. For example, where do candidates at finishing schools who have less than six subjects and who will have oral and practical marks fit in?

Another possible issue that could arise is the situation where local policy decrees that only the results of candidates from the sub-system A are used to calculate the final percentage of passes. At registration, it is possible for provinces to abuse these two sub-systems by entering possible failures under sub-system B. thereby creating a bias towards better results. An example would be that all "repeaters", with a statistically proven failure rate, could be entered under sub-system B and would thus be excluded from final calculations.

Having two sub-systems means duplication of the work for the Systems Administrator. Most of the common tasks need to be performed for both sub-systems, for example registering of centres or subjects. Not only is this time consuming, it is also error prone.

2.5.2.5 Alternatives to the existing system

Standard Systems
The committee was made aware of suppliers of two such systems, one from the United Kingdom and one from a local bureau. To determine the suitability of these programmes for South African conditions would require a detailed investigation that falls outside the brief of this Committee. Therefore this Committee can only comment on issues pertaining to such systems in general, without discussion of strengths and weaknesses of specific alternatives.

Advantages
Alternative options should have reference sites and should have been tested thoroughly in the countries of origin and possibly in other countries as well. These programmed, in general, will use the latest technologies available with graphical user interfaces, powerful query facilities, flexible reports and a high level of error reporting and data validation.

Concerns
Latest technology in software requires the latest technology in hardware. Whereas the current examinations computer system and sub-systems are able to take advantage of the older 286 or 386 computers with monochrome monitors that are currently installed throughout the provinces, more recent software offerings will, almost certainly, require Pentium computers with colour monitors.

Similarly, it is felt by the Committee that none of the provinces has the necessary infrastructure to implement any major changes or systems. It is also unlikely that they will be able to afford the number of hardware upgrades or replacements needed in the near future.

As we have stated elsewhere in this report, there is a growing consensus of opinion amongst the provinces, at operational levels in particular, that all provinces should use the same centralised computer system. Any decisions with regard to change must be centralised and approved by IPEC.

Any so-called "off the shelf" package will certainly need a degree of change or personalisation in order to satisfy "local" requirements. There will be, for example, different pass requirements and different rules for endorsement that would have to be programmed into the system. Any new system must also be able to communicate with SAFCERT and the universities in the prescribed data format. All such changes will increase the initial cost of such a programme.

2.5.3 Recommendations / Action plan

2.5.3. 1 Documentation

A set of operations manuals that outline the flow and functions of the whose system should be developed. This should be structured along the lines of a year plan and should include typical target dates, actions to perform, function codes and statistics. Built into this should be a provision for progress reporting, within the province and into the Departmental Head Office.

A number of provinces have already started to develop this type of documentation. These efforts should be co-ordinated by IPEC and the ITC to produce standardised quality documentation. Provinces, which have started to develop such documentation, should make it available at the next ITC meeting. Following an appraisal of the work done to date there should be input from all provinces to assist in producing detailed, comprehensive, user friendly documentation which everyone can use.

A project team, with a representative from each province who could stay in touch either by e-mail, fax or telephone, can then decide the level of detail that should be included in this documentation. There should be a provision for provinces to be able to personalise this documentation in order to suit their individual needs.

2.5.3.2 Management information

Provincial EMIS units should be used to facilitate the creation of queries from the flat file download where the examination unit does not have the capacity. Q Data could advise provincial EMIS personnel at one of the EMIS workshops on how to create and download the flat files. EMIS personnel could then provide training and support locally on how to manage the flat files after download and how to link to other databases and create macros on request.

Q Data should urgently proceed with their investigation of alternative options as approved by IPEC.

2.5.3.3 Duplication of Requests

All requests should initially be referred to the Help Desk. If the function or a similar programme exists, the local consultant should be informed of this. It is important that all provinces must be advised of all requests in order to avoid duplication and to facilitate buy-in for any requests that have national benefits. New and proposed functions can be discussed at the ITC meeting. It is essential that Systems Administrators and IT representatives should attend the ITC meetings.

2.5.3.4 Central Source Code

Although one cannot be prescriptive it is recommended that all provinces use the same source code. The level of uniqueness in provinces is so small that only a few specialised functions or peripheral programmes are needed. The benefits of centralised source codes are overwhelming. Provinces share the cost for programme updates, history files are common to all provinces and standardised statistics are available to the national Department, to name but a few of the benefits.

It is strongly recommended that Gauteng and Northern Province migrate back to the central source code. The amount of customisation needed is very small and could be accommodated centrally. KwaZulu- Natal is unique in that it has its own programmers to customise the central source code for their peculiar needs. There is no reason why Gauteng and Northern Province could not do the same if required.

2.5.3.5 Role of Q Data

Q Data has continued to play an important role in the success of Senior Certificate examinations in the past. In the view of the committee they have gone that "extra mile" in order to assist both the provinces and Departmental Head Office processes. There is a general consensus of opinion that without the commitment of Q Data personnel at all levels a number of provincial examinations could have been in jeopardy.

We could find no pressing reasons to either terminate Q Data's contract or not to recommend that the existing contract be extended. It is important to remember that when the new Tender for maintenance of the existing contract is written, it should highlight the need for a significant period of up to twelve months of parallel support where any new successful contractor and Q Data will need to work side-by-side.

2.5.3.6 Impact of Present Examination System on Other Systems

The impact on and close relationship of the maintenance and support of both the Help Desk and History Files sub-systems that we examined, and the sub-system for Formal Technical Colleges" and "Teachers-Examinations" are important. These are all covered by the existing agreement with Q Data. Because of the time constraints the two latter sectors were not investigated by the IT sub-committee. It is our recommendation that further investigation should be carried out in order to ensure that there is a more complete and rounded report covering all aspects of Q Data's role in supporting the total examinations system and its users. This point was also highlighted in the "Limitations" section of this report.

It is therefore recommended that approval should be given for the provision of further resources to allow an examination of these other two sections. Such an investigation would cover the whole spectrum of the services offered by Q Data, as well as their potential to operate as stand-alone sub-systems, if this is seen as being applicable in the future. This may be necessary should future contracts be awarded to other suppliers when a phased implementation is advisable. This recommendation has the support of Mr André Reyneke, Directorate: National Examination

2.5.3.7 Tender for Maintenance

The committee does not recommend a tender for a brand new examination system until Curriculum 2005 is firmly established and actual dates of implementation are finalised.

There was a feeling from some of the people interviewed that the development and maintenance of any examination system should be separated and that different companies should perform these functions. One advantage of such an approach is that this will ensure a fresh and innovative approach from the maintenance team. The disadvantage on the other hand is that an examination computer system is a very technical and complicated programme. It will take a long time before any new company can change and maintain the more technical parts such as the resulting programme. In any system the people who developed that system know the system the best and will deliver the best maintenance service. Also, when there are different companies handling different parts or functions of the system, where does the ultimate responsibility lie?

When the Department of Education goes out to tender for the maintenance of the existing system as from September 1999, there are a number of factors to consider before writing the Tender Document:

• The Tender must be open to all interested companies, but cannot exclude the company that developed the system.
• All companies must indicate how they will ensure a smooth transition. They must indicate how and when they will obtain the necessary expert knowledge of the examination computer system. This training process must be completed before the start of the maintenance contract and the cost must be included in the Tender.
• It will be impossible for another company to employ some of the current Q Data consultants owing to a clause in their contracts with Q Data.
• If a new contract is awarded to another company that specifies the instant withdrawal of Q Data consultants, this could lead to the collapse of the whole examination process. Each province's capacity to perform all the Systems Administrator duties must be carefully analysed before removing any Q Data consultants.
• Without a centralised knowledgeable Help Desk, as currently provided by Q Data, support will have to be localised. Current budget constraints mean that most provinces cannot afford this facility. Therefore any Tender must include the provision of a central Help Desk.
• If the Tender is awarded from September/October 1999 there will not be enough time to train new consultants before the start of the November 1999 examinations. The whole transition process has to be planned very carefully and we are not certain that any of the provinces has both the manpower and facilities to ensure a smooth handover.

2.5.3.8 The Help Desk

According to tender RTS 1021 the department must supply the necessary infrastructure in order for the Help Desk to function effectively. This include computers, printers, network access, software packages, etc.

The computers and printers, which the Q Data Help Desk inherited, are old and prone to breakdowns. Most of the equipment was used by the ex-departments. If one expects quality service from the Help Desk, it should be given quality equipment and support. Q Data had to buy software packages out of their own resources in order to be able to continue to offer the level of service expected.

The Help Desk should fall under the Department of Education and the Department should budget for it. No single Province should be asked to take responsibility for the Help Desk and it is impossible for all provinces to take joint responsibility.

The committee strongly recommends that the Help Desk should report to a designated person in the national Department of Education. Presently they are accountable only to IPEC and this creates problems when there are ongoing and urgent issues that need to be discussed, and decisions made. There is a precedent, as the Technical College system has a contact person at Head Office as well as an IT contact at the national Department of Education.

2.5.3.9 Fiscal Policy

All revenue collected by the examination unit will go directly to the central treasury. The examination unit acts as an agent between the applicant for a new certificate and SAFCERT. They collect the fees from the candidates and submit the request for a new certificate together with the payment to SAFCERT. Whilst the provincial department is involved in the work and incurs costs the money collected goes directly to the treasury. Mechanisms to correct this need to be examined.

2.5.3.10 Examination Fees

There is a general feeling that all candidates should pay examination fees, in line with the Constitution, which guarantees "free education up to Grade On. The strongest feeling was that all of the "privates candidates, Supplementary and re-sits" should pay at registration. This is because although the same cost is incurred in preparing for each of these candidates as for each year's new Senior Certificate examination candidates, the number of attendees that fit into this category that actually sit the examination is less than 50%, and in some provinces the figures quoted were less than 20%. As an example one of the provinces has two private candidates who enrolled for ten subjects (the maximum) each. It is inconceivable that they would write even 5 or 6 of these.

This year there is currently a total of 825,689 candidates who have registered for the-Senior Certificate examination. Of these 223 0663 are "privates. This means that costs will be incurred for between 100 000 and 150 000 candidates who have no intention of sitting their examinations. Up-front costs will minimise this problem and bring additional revenue to the system.

2.5.3.11 SAFCERT

SAFCERT operates its own computer system, maintained by two in-house programmers and one Q Data consultant. All examining bodies supply the information of candidates that qualify for a certificate to SAFCERT. SAFCERT evaluates and verifies the data and then issues one of three types of certificates:

• Senior Certificates
• Senior Certificates with matriculation endorsement
• Subject Certificates for candidates who do not qualify for either of the above

SAFCERT and the provincial results programme each other. In simple terms, SAFCERT points out errors in the provincial results programme and the provinces highlight errors in the SAFCERT programme.

Currently, SAFCERT is experiencing two major difficulties.

• Payment for SAFCERT services
  SAFCERT does not receive any funds from Treasury. Provinces pay an amount of R7 for each certificate issued by SAFCERT and this revenue finances the whole operation of SAFCERT.
• According to internal policies, payment by the provinces should be made with their submission of requests for certificates. This open does not happen and provinces delay payment until as late as possible, usually because this is the end of the financial year and most provinces do not have any funds left. If SAFCERT so decided, this could cause a legitimate delay in the processing of the requests.
• Certificates could be printed as early as February/March, but is normally only printed in May/June or even later.
• The ideal would be that provinces deposit an amount of money into a suspense account once the numbers of candidates for the current year's examinations are finalised. The amount could, for example, be based on the number of certificates issued from the previous years examinations. The provinces can earn interest on the money in the suspense account until the certificates are processed.
• If this approach is used SAFCERT will receive their payment without delay. It will also ensure that each province uses its current budget to pay for costs incurred during that year. Obviously Treasury must be consulted to find the best way of dealing with this proposal.
• Late Submission of Data
• The certification process should start as quickly as possible. Unfortunately most provinces delay the submission of their data sometimes to even as late as June. The reason for this is twofold:

• Provinces wait for the finalisation of re-marks and the supplementary examinations.
• Provinces wait for the new financial year before submitting their first data.

Provinces should be encouraged to submit a first round of data just after the closing dates for the application for re-marks and/or the supplementary examination. The current examinations computer system has the option to submit data on candidates who have no pending re-marks or supplementary examinations. Thus provinces can certify 80% of their candidates early in February and the rest after the finalisation of the supplementary examination and queries. This will mean that candidates will receive their certificates much earlier than is the case at present, which will benefit them in interviews, etc. It will also show that the Department does care about meeting the needs of candidates.

2.5.3.12 Capacity Building in Provinces

The major reason for the perception that the services of Q Data consultants are expensive is that they are often performing many of the duties that the provincial personnel should perform. These include systems administrator duties, planning, network maintenance, attending of meetings, etc. Normally this happens because the province does not have the internal resources or capacity to perform these responsibilities. It is therefore necessary to investigate strategies to develop provincial capacity in these areas as soon as possible. In doing so major savings can be implemented.

2.5.3.13 Systems Administrators

All provinces must embark on a process to identify and train their own Systems Administrators and their support staff. A fully qualified Systems Administrator should be appointed at least at the level of Assistant Director, bearing in mind that the, System Administrators plays a major role in the project planning and management of the examinations.

2.5.3.14 Trainee Consultants

Q Data, as a company, has a policy that they train affirmative action candidates at their Q Data Academy and then place them with their clients for practical experience. These "trainees" function as consultants for a period of approximately a year. The tariffs charged for these trainee consultants are such that they only cover the cost of the training that is provided on an ongoing basis. The cost is approximately a third of the cost for a normal Q Data consultant. After the one-year period the client is given the option of employing this person permanently. If the client decides otherwise, Q Data will take the trainee back and utilise his/her services elsewhere.

The Gauteng Department of Education has used three trainee consultants to manage the IT side of the regional offices. After a period of about eight months the regional offices were self-sufficient and two of the trainees went back to Q Data whilst the third one was transferred to the Provincial Head Office.

Trainee consultants can play a major role in the transition from a situation where Q Data consultants play a dominant role to where the province is virtually self-supporting. The trainee consultant can be absorbed by the province either as a Systems Administrator or an IT professional.

2.5.3.15 The History Database

Most of the computerised history is currently stored on a centralised database. Gauteng Province pays the cost for the maintenance of the history database and then bills the other provinces proportionately. The paper-based records are scattered throughout South Africa. Based on the policies of the previous examination bodies, paper based records are housed in Pretoria, Cape Town, Bloemfontein, Ulundi and Umtata.

The history files are a national asset and the need for these records will increase in importance once SAQA and the NQF become fully functional. The Department of Education must take responsibility for the maintenance and cost of the history database. The national Department of Education must have a unit that takes responsibility for all historical records.

The paper-based records vary greatly in format and specialised personnel are necessary to process them. Although it would be better to centralise all paper-based records, the personnel working with these records should receive training for each different format. Even within one examining body quite a few different formats may have been used over the years. It is therefore recommended that the paper-based records remain in the provinces where they are presently kept.

The cost of computerising the paper-based files would be very high and the frequency of usage does not justify such costs. Part of this unit should be a Help Desk where requests for copies of paper-based records are co-ordinated. All requests, irrespective of province, should be directed to this Help Desk. This Help Desk will then contact the applicable province to obtain the record.

2.5.3.16 SAQA & the IEB

Because of time constraints only a short time was allocated to the above from the perspective of their approach to IT.

The IEB believes that the existing Q Data systems fulfil its objectives, but believes that it is counterproductive to have the control of the total system in one company's hands. For this reason they argue for a Tender for a brand new system and have in mind the purchase of user-friendly examination software which is currently used in many other countries. The IEB also suggests that the tender document should separate the software purchase from consultancy and training services.

SAQA is currently installing its own internal system, funded primarily by the Canadian Government. They recognise the need to be able to access all databases containing student and employee records of achievement, and qualifications, as their policy of the National Qualifications Framework comes to fruition and "Lifelong Learning" becomes a reality.

2.5.3.17 Strategy to Retain IT Professionals

Some provinces have started to look at strategies to retain IT professionals in the province. All provinces should embark on a unified strategy to pay IT personnel market-related salaries.

This system was also developed by Q Data. No uniform decision has been taken on the future of this system. Presently three provinces still use this system. The Colleges of Education history files from the other provinces are retained in different formats and at several locations. In some cases only a spreadsheet on a single microcomputer constitutes the historical record for a province. IPEC should find a suitable solution for the processing of examination and history files of Colleges of Education.

2.5.3.19 Year 2000 (Y2K)

Although the present computer system is year 2000 compliant, it is recommended that the results be issued before 31 December 1999 so as to prevent any unforeseen problems with networks, telephone connections, electricity supply, etc.

2.5.3.20 ITC meetings

New functions and requests should be discussed at the ITC meeting. It is generally felt that the correct people do not always attend the ITC meetings and that Systems Administrators should definitely attend. The cost for attending ITC and IPEC should be borne by the Department of Education, as the cost varies greatly from province to province.

2.5.4 Provincial Feedback

Strategy
The computerised examination functions and data capture are all performed at East London, where their EMIS centre of the Eastern Cape is located. Administration is located at King Williams Town. The Eastern Cape uses the central computer system at Bureau Nucleus, as well as the centralised source code.

The province has a Systems Administrator and one Q Data consultant who was formally employed by the province as the Systems Administrator. Because of local problems regarding staffing, special clearance, as per the Maintenance Contract with Q Data, was given to allow him to remain within the province (Had this not been given, he would have left the employment of the province, as he already had a better job offer elsewhere - not with Q Data. This highlights the problems that all provinces constantly face). The new Systems Administrator has not received any formal training and relies heavily on the Q Data consultant.

Because of their geographic split and numbers of students, who feel that they need two systems administrators, one to deal with certification and one for other examination functions. This was not a common request.

Data Capture
Data is captured centrally at East London, using both Examinations and EMIS personnel and equipment. Their ideal is a core of data capturers that can be augmented during peak periods.

Infrastructure
Even though the Eastern Cape has the necessary infrastructure to conduct the examinations, the province experiences huge problems, mainly as a result of a lack of continuity of Senior Management, from MEC downwards, over the past three years. Seemingly no one is prepared to take responsibility for making decisions. Bills are not paid, submissions are ignored and essential supplies, such as paper for the printing of examination papers, are not always available timeously. As a result this year's whole examination process is behind schedule.

Morale in Examinations" is extremely low and there have been warnings of potential leakages in order to expose the problems to a wider audience. Most of the junior staff is underpaid in relation to their functions. The response by senior management throughout the province that "the alternative, to resign and try and find another position, is worse. The Department would surely not wish to endorse such an attitude.

View on Q Data
Q Data is considered to be very supportive, knowledgeable about the requirements of the system and the needs of the Province, and very helpful in assisting the Examinations department to achieve their objectives. They use the local Q Data consultant and the centralised Help Desk extensively. Even though payments of invoices have been late, this has not stopped Q Data from performing their duties, often above and beyond contractual terms.

Comment
From a purely IT perspective, the 1998 examinations can be still conducted. This will mean that more junior personnel will have to be hired on a contractual base, as often happens elsewhere. The Q Data consultant knows the requirements and functions of the systems administrator in detail and is in a position to provide in-depth backup and support.

The money for the examinations is available, but various factors contribute to the stagnation of the examination process. They have experienced a high turnover of senior management staff and the new personnel seem to be reluctant to make decisions and shy away from taking responsibility. As mentioned earlier, the administrative staff is remunerated at low salary brackets, equivalent to service personnel, yet they are required to fulfil roles normally undertaken by more senior positions. This has led to a low morale. Some of these people, it is thought, could well derail the examination process as a form of protest.

2.5.4.2 Free State

Strategy
The Free State has now opted for a centralised approach. They have one Q Data consultants on-site but no Systems Administrator. The Q Data consultant acts as Systems Administrator for the Province - which as we will see is not the most cost-effective way of using this service from Q Data

Data Capture
All data is captured on-line at the provincial Head Office.

Infrastructure
The Province has sufficient personnel and computers. They use older computers without hard disks as terminal emulators. They provide their own training for personnel and do not use the Help Desk facility for this purpose. During peak periods extra personnel are appointed on a contract base. The examination papers are printed by the provincial IT unit.

View on Q Data
They utilise the Help Desk facility for all immediate problems. They have no complaints at all about the level of service from either the Help Desk or the local consultant. All ad-hoc requests go through the local consultant.

Comment
The Free State awarded a tender to a local company to develop a new examination system. This system was initially to run parallel with the current system.

Before completion the project was terminated for a number of reasons:

• The cost of developing a second system was escalating and the cost of supporting two systems was in excess of the available budgets.
• The additional workload providing input in order to develop and test a second system put additional stress on the department.
• The Province has barely enough personnel to run one examination system let alone running two in parallel.

Any province wanting to implement their own system should take cognisance of these issues, as should the Department when going out to tender for either a new system or for maintenance of the existing system as it will be impossible to achieve a non-parallel hand-over.

2.5.4.3 Gauteng

Strategy
Gauteng has opted for a decentralised only using the Bureau Nucleus facility for the processing of data as created locally. The Head Office Examination function is made up of the Head of Examinations, a fairly new Systems Administrator, who is being mentored by an external contract person, who was previously employed by the Province, and three Q Data consultants. All other examination personnel are located at the three Regional offices.

Gauteng has its own copy of the system source code.

Data Capture
Entries are completed at the schools where they have the necessary, hardware. For schools without computers the entries are processed at the districts. All the data from schools and the eighteen district offices are accumulated at the Regional Offices. The marks are also captured at the Regions.

Infrastructure
Gauteng is fully equipped in order to support and process the whole examination project. They also have adequate staff to support the numerous processes. Whenever temporary staff is required they have a base of qualified personnel to call upon.

View on Q Data
It is generally felt that Q Data are providing a "value for money" service. The Q Data experience of the system is helping the Province to move forward and eventually "hopefully" allow them to become fully self-supporting, preferably eliminating their usage of the centralised Help Desk. They agree with a centralised system but reserve the right to "localise" it where appropriate.

Peripheral Systems
Gauteng has developed a number of useful sub-systems or standalone PC packages which can be made available to other Provinces (IPEC and the ITC should be the platform for reviewing these opportunities). They have a PC based programme for the capturing of marks so that the marks can be captured at the source. They also have a programme for the appointment and payment of markers that links directly to the Persal payment system.

Comment
Whilst the Province has a go-it alone philosophy overall they recognise the need to work to a centralised format. Being the "wealthiest" province, there is feeling that they have the biggest say in future directions but nevertheless are prepared to work with all other provinces in the best interests of the total process.

There is a feeling that all centralised activities located in Gauteng belong to Gauteng.

2.5.4.4 KwaZulu-Natal

Strategy
The examination process is administered from a central base. Their eight regions are all connected via Openet and they perform enquiries and registration for "re-marks". KwaZulu-Natal is the only province with its own full-time programmers. They do not have any Q Data consultants on site.

Their Systems Administrator is seconded from one of the regions and is only an "acting systems administrator" and could return to his original post at any time. Here they are exposed.

They are utilising the central source code although their own programmers change the system to suit their own needs. The procedure is that the programmers create the new source code and then requests the Help Desk to register it on the system. Thus Q Data always knows what changes KwaZulu-Natal has made.

Data Capture
All data is captured on-line in either Durban or Ulundi where a number of specialised low volumes tasks are carried out. They have their own PC-based programme for the selection and appointment of markers.

In the past they have used a private off-line bureau but have recognised that on-line capturing is not only quicker and more accurate but is also less expensive.

Infrastructure
KwaZulu-Natal has a very good self-contained infrastructure. Their network is highly effective and they have their own full-time network controller.

They have an excellent secure in-house printing facility where examination papers are printed. During idle periods the printing facility is made available to other state departments.

All the computers are Pentiums. Their computer bureau in Ulundi, and is not yet Year 2000 compliant.

One problem in KwaZulu-Natal is that many of the personnel are temporary, having been seconded from elsewhere, some for quite a considerable time. Hoping to have a stable examination unit without having permanent personnel in key positions is wishful thinking.

View, on Q Data
KwaZulu-Natal very rarely has the requirement to use the Help Desk. However they do have a good working relationship with Q Data and as such as more than happy with the support that they receive.

One concern that was expressed is that the internal documentation on the source code is in Afrikaans making it difficult for English speaking programmers to understand the programme documentation.

Comment
KwaZulu-Natal has shown that it is possible to function without a Q Data consultant on-site. The requirement is that the province must have complete in-house skills in such areas as networks and software languages, as well as other people whom fully under the system.

2.5.4.5 Mpumalanga

Strategy
The examination function is centralised in Middelburg and the Province uses the central source code. They have one recently appointed Systems Administrator, who has been with the Department for a number of years, and two Q Data consultants.

Data Capture
All data is captured on-line at the Middelburg office. The marking is done at the ten districts and the mark sheets are then captured in Middelburg.

Infrastructure
During the peak periods Examinations borrows computers and staff from IT-, EMIS and other units for the capture of data.

The Province has an excellent and secure facility for printing of examination papers. The whole process from typing, editing, printing and packaging is done in one building under very strict security. Papers never leave the building before the distribution to schools. Bulk printing is done at the Nucleus Bureau in Pretoria.

View of Q Data
"Q Data is good value for money". The local Q Data consultants provide excellent support and backup and respond to requests promptly. Their use of the centralised Help Desk is minimised but where used they are more than satisfied with the response.

Considering that Mpumalanga still has to sign an Agreement with Q Data, and has not paid any Q Data invoices during 1998, the fact that Q Data are prepared to continue supporting them is commendable.

Comment
The physical infrastructure employed for the whole examination process employed by Mpumalanga is extremely professional. Whilst other provinces have similar infrastructures, notably KwaZulu Natal, as a facility the Middelburg operation has much to offer as an example how things should operate. A high level of security will certainly minimise chances of leaks.

2.5.4.6 Northern Cape

Strategy
Northern Cape uses a centralised approach for their examinations. All operations are centralised on Kimberley.

The Northern Cape utilises the central source code. It has its own experienced Systems Administrator but there are no Q Data consultants on site. The Help Desk processes all requests.

Data Capture
All data capture is done on-line at the Provincial Head Office using staff from examinations as well as from other units such as EMIS.

Infrastructure
The Province has sufficient computers and terminals. Whilst they have a number of Pentium computers, older computers are used as terminal emulators in the main. These are 386 or similar, without hard disks.

Printing of examination papers is outsourced to the IEB. Bulk printing is done at the bureau in Bloemfontein.

View on Q Data

Q Data provides the necessary level of support whenever needed. The Province depends completely on the Help Desk for any programme changes, and ad hoc requests, and has not experienced problems in the past.

Comment
Northern Cape is forced by the small number of candidates to rely solely on the Help Desk for ad hoc requests. Their experience has shown that it is possible to function without a consultant on-site. The most important prerequisite is that the Systems Administrator must be very well trained and competent.

An advantage of relying on the centralised Help Desk is that the Systems Administrator can first evaluate ad hoc requests. Only "must have" requests are then passed on for processing. All "nice to have" requests will be processed only if they prove to be cost effective.

Northern Cape is currently in the process of documenting each process as part of the examination process.

2.5.4. 7 Northern Province

Strategy
The Northern Province uses a decentralised approach for the administration of their examinations. Due to skills limitations the computerised examination functions are currently centralised at the Head Office in Pietersburg.

The six regions each have an examination component, but no computers or terminals. Regional personnel have received training and utilise Persal terminals for urgent enquiries.

Northern Province, and Gauteng, are the only Provinces that have their own source code. The original plan was to migrate the complete database to the local computer bureau in LebowaKgomo but there is now recognition that a centralised approach is preferable.

Northern Province has two full-time Q Data consultants on site. The Help Desk is not normally utilised except for critical programme changes within the Resulting" sub-system.

Data Capture
Up to now all data capturing was outsourced to a local private off-line bureau - Omnidata. This year, for the first time the details of private candidates were captured on-line at Head Office by eight in-house data capturers. The full-time candidates were still captured by the private bureau. This year, for the first time Capturing errors will be corrected on-line. The private bureau also captures the examination marks.

Infrastructure
Northern Province has a serious shortage of computer equipment. The Head Office Examinations Department is adequately provided for. However each of the six regions will require at least six terminals/computers in order to be self-supporting.

There is a full-time Systems administrator who has been in this position for approximately four years. There is no network controller. Q Data consultants perform this function if necessary.

Printing of examination papers is outsourced.

The Province is embarking on a strategy to identify and train two people to act as programmers. The intention is that they should take over from the Q Data consultants as well as eventually support and backup the Systems Administrator.

View on Q Data
Q Data is acknowledged to very helpful and supportive, especially as regards in-house training on the system and its functions. The two consultants also perform both systems administrator and programming tasks when requested. Used this way by the Province, it is felt that their services can be perceived as expensive. This is so as there is no alternative as in reality the Province should have its own departmental staff carrying out these functions.

The centralised Help Desk is used for the publication of results in the media and on the Internet, as well as for the distribution of data to universities. It is also used as a backup facility. As such the Help Desk is recognised as providing effective and worthwhile support.

It is felt that ad hoc programming requests could not possibly occupy the two consultants for eight hours per day. They are employed on a full-time basis on site and many of their other duties include systems administrative and managerial tasks.

Comments
On-line versus out-sourced data capturing
The Northern Province exclusively used off-line data capturing bureaux until they started to capture private candidate's on-line. Their experience is that on-line capturing is better, more convenient, time saving and cost effective. Error correction could be done immediately and logical errors corrected on the fly.

Own source code
A strong feeling was expressed that the Northern Province should migrate back to the central source code. In retrospect they recognise that there is absolutely no reason why they should try to maintain their own source code. They don't have unique requirements in the Province and they don't have the capacity to maintain the systems themselves.

Provincial programmers
It is commendable that the Northern Province should strive to identify and train in-house programmers. They have already identified two people and training will commence as soon as the necessary approval has been granted. The problem is that it is difficult to keep IT personnel. Western Cape has lost in excess of twelve programmers during the last few years. This is just one of the reasons that they were forced to migrate back to the central source code.

Northern Province must not become a "training ground" for the industry. The Northern Province should embark on a strategy to keep their programmers, when up to speed, by paying market-related salaries.

2.5.4.8 North West

Strategy
North West uses a combination of centralised and decentralised approach for their examinations. The, Province is divided into twelve districts, and these districts are serviced by two offices, one in Potchefstroom and one in Mafikeng (previously Mmabatho). The two offices both perform activities such as entries, capturing of marks etc., but each also performs some unique functions such as printing and packaging of papers in Potchefstroom and appointment of markers in Mafikeng.

North West utilises the central source code. It has one full-time Q Data consultant on site at Mafikeng. The Help Desk is an essential part of the examination process. The Q Data consultant performs some Systems Administrator functions for the Mafikeng office. The person, who is primarily an IT person, who carries out some of the tasks of the Systems Administrator is located at Potchefstroom.

Data Capture
All data capture is performed on-line at the two regional offices.

Infrastructure
The Province has sufficient computers and terminals. About half of the computers are Pentiums, the rest are 486. Some older computers are used as terminal emulators.

Printing of examination papers is done at the Potchefstroom office. Only the papers with the bulk of candidates are printed at the government printers. There is a bureau facility in Mafikeng, but the bureau does not have a high speed printing facility.

View on Q Data
They are extremely happy with the role of Q Data, and the effectiveness of the examination computer system. They see no reason to change as things are at present and in fact believe that to do so at present would be a backward step, even if they could afford it.

Documentation
North West has developed excellent documentation. The documentation includes processes, the date and the computer function code. They also have documented function specific training for reference.

Comment
Central Source Code
North West has conducted an investigation into using their own source code versus centralised source code. The findings were that it is not cost effective to have their own source code. The cost of human resources to support such a move would be high. They feel that only essential functions that are specific to that province, if any should be decentralised to the provinces. Another advantage of using the same source code, they feel, is that provinces could more easily exchange ideas.

2.5.4.9 Western Cape

The Western Cape uses a centralised approach. All examination functions are performed from the provincial head office in Cape Town. The Western Cape was the only province who had their own unique examination programme. When they started to use their programme in 1992, they had twelve dedicated IT personnel. A number of programmers left them for the private sector until by 1997 only four were still working for the Province. When the key programmer left early in 1998, the other three programmers also left. This left Western Cape without any maintenance programmers. They then decided to adopt the central source code and appointed three Q Data Consultants to assist in migrating to the centralised system. The plan is to reduce this to two consultants.

Data Capture
All data capturing are done on-line by data capturers provided by the IT unit.

Infrastructure
The Western Cape has excellent infrastructure. All their computers are at least Pentiums and are connected in a network. The users are all trained to be computer literate. They use the bureau in Cape Town for bulk printing.

View on Q Data

Q Data is professional and the service they render is very good. Their training is very good. The fact that Data consultants know the examination computer system in detail is helping them with the changeover along with the re-training of their personnel.

Comment
Compatibility of Central Source Code
The Western Cape decided to migrate to the central source code although the programme currently does not meet all their requirements. The Western Cape is the only province that uses year marks but the system was not designed to cater for year marks. Even so the Western Cape decided to migrate to the central source code. Their view is that the system can evolve even more to suit each province's needs. The benefits of centralised source code far outweigh the disadvantages.

IT Personnel
The Western Cape's experience is that it is impossible to keep IT personnel without paying market-related salaries. They are embarking on a provincial strategy to pay there IT personnel market related salaries.

Contents    Section 3     Appendicies