Annexure B

SOUTH AFRICAN CERTIFICATION COUNCIL

MINUTES OF AN EXTRAORDINARY MEETING OF THE S A CERTIFICATION COUNCIL HELD ON 21 AND 22 JANUARY 1999, AT THE KEMPTON PARK CONFERENCE CENTRE

1. OPENING

The Chairperson welcomed all members, the meeting was constituted and he declared the meeting open. Mr Moll was requested to take the minutes of the meeting.

2. MEMBERS PRESENT

3. APOLOGIES

Apologies were received from Prof. I Bellis, Messrs J A C Botha and S J Mkhwanazi. Messrs T B Khunyeli and D Kennedy had resigned as they were no longer in the service of the Free State and Western Cape Education Departments. Since Mr Kennedy had resigned as a member of the Council, the MEC of the Western Cape nominated Mr B Schreuder as a member of the Council. The Minister had not yet confirmed his appointment, but the Chairperson was of the opinion that this would be a mere formality. It was consequently decided that Mr Schreuder be invited to attend the Strategic Planning meeting of the Council as an observer. Mr Schreuder offered to recuse himself from the extraordinary meeting but Council decided to grant him observer status at this meeting as well.

4. CONFIRMATION OF THE AGENDA

It was agreed that the extraordinary meeting would discuss just one item, viz. the report of the external moderator in Biology, Mrs A A Crowe, on the moderation of a sample of scripts from the Mpumalanga Education Department.

5. MODERATION OF A SAMPLE OF THE 1998 BIOLOGY HG SCRIPTS FROM MPUMALANGA EDUCATION DEPARTMENT

The Executive Officer presented the moderator's report (attached) and a written report on his own follow-up of the matter (also attached). He mentioned that he (on behalf of Safcert) had expressed his surprise at the results achieved in certain subjects at the standardisation meeting, but that, in the light of assurances received from the Province relating to study guides and intensive support of Grade twelve candidates, had accepted them in good faith. He also pointed out that Prof. Lotz, who had attended the Mpumalanga standardisation meeting on behalf of the central Department of Education, had reported back his concern about the Mpumalanga results to the Minister.

The Executive Officer went on to explain his analysis of the scripts incorporated in the sample, and circulated examples for the attention of Council members. There were indications of cover pages having been replaced and of marks apparently having been altered. These initial indications led him to proceed to Mpumalanga to obtain more scripts, which request had been denied. He stressed that he had been following a procedure laid down in the Council's regulations.

Discussion ensued as to why the problem with the results had not been picked up in the standardisation meeting on the basis of statistical evidence. Prof. Fatti pointed out that it was not possible to pick up the overall pass rate on evidence presented in the standardisation meeting, since overall results calculations had not been carried out at that stage. Mr Moll pointed out that Safcert administrative procedures were not tight enough during the standardisation process, in particular with regard to the recording and official sanction of decisions. Provinces kept their own records, however informally, but there was no formal Safcert documentation on decisions reached. Safcert therefore had to accept its share of the blame for the situation that had risen in Mpumalanga. Dr Mgijima pointed out that Safcert had received Mpumalanga documents relating to ogiving on 23 December, five days before the standardisation meeting, which should have given Safcert sufficient time to pick up the problem. In response the Executive Officer emphasised that although there had been nothing technically wrong with the results, he had realised that another kind of investigation would be necessary to determine the full cause of the improved raw marks.

At this stage a Mr Mello arrived at the meeting, and introduced himself as the Mpumalanga representative to the meeting. He apologised for being late indicating that he had lost his way. He mentioned that the MEC had nominated him to attend the meeting and to give an input to the Council on the matters under debate. Mr Mello's status at the meeting was discussed and the Chairperson said that Council members were appointed by the Minister and did not represent a province. He also ruled that no provision was made for a proxy. However, given that Mr Mello was present, the Chairperson invited him to put Mpumalanga's case to the meeting as a guest. It was agreed that while the written input (attached) would be recorded in the minutes, it would not be deemed to be a formal response from Mpumalanga. Mr Mello concluded by saying that he did not receive fair treatment at the meeting, that the meeting was arrogant towards him and that he felt degraded by the casual manner in which he was treated. He regarded the process as not in the interests of Mpumalanga.

The Chairperson thanked Mr Mello for his contribution, but differed strongly with Mr Mello's contention that the reception afforded to him had been indecent, casual or arrogant. He added that the Council had not known that Mr Mello would be attending the meeting, and that Mr Mello's characterisation of the meeting was not accurate. He assured Mr Mello that Council would take what he had said very seriously and would discharge its responsibility in relation to it in terms of its Act. The Chairperson also requested Mr Mello not to convey to his colleagues that the members were indecent, as that was far from the truth.

After further long and in-depth discussions on this issue the Council unanimously

1. noted a moderator's report on the 1998 Mpumalanga Biology HG examinations which indicated irregularities in some of the scripts. This was supported by an inspection by Council of scripts from a sample of forty-five taken for post facto checking, most of which had apparently been tampered with.
2. took note of the concerns raised by the Mpumalanga Education Department regarding the role and procedures applied by SAFCERT officials during the prior consideration of the 1998 results in Mpumalanga.
3. noted the view that the publication of results (by contrast with the certification) is the responsibility of the respective Education Departments or examining bodies.
4. expressed the belief that a detailed investigation was necessary to establish the extent of the possible problem (as well as the possible impact of a continuous-assessment component on the marks, and improved educational practices by the Department), and to check whether the integrity of the examination in Mpumalanga might have been compromised.
5. resolved to establish a committee to undertake the investigation, to report to the Council as soon as possible, for Council to consider its findings and decide upon their wider implications, if any. The Committee would comprise

• Dr M Nkomo - Chairman of Council
• Dr N Mgijima - National Department of Education representative
• Prof. P Fatti - Statistics Committee representative
• Prof. P Kota - Moderators Committee representative
• Dr F Calitz - Executive Officer.

The committee would have the authority to take the necessary legal opinions.

6. agreed to supply the Mpumalanga Department- with the report it requested of the irregularities encountered during the moderation and subsequent inspection of the sampled Biology HG scripts.

7. decided to request that Mpumalanga Department allow the Council's committee or its appointed officers

• to have direct access to the answer scripts and computerised or other records for Biology HG and such other subjects as they may decide, so that the committee may undertake the above investigation;
• to remove such scripts as may be necessary for the inspection of the Council.

8. further resolved to undertake a general review of the validation and other procedures of SAFCERT (including consideration of the sequence of events in this instance as an important example), towards clarifying and formalising its relationship with provincial education departments and other education authorities.

The Council also decided that the Committee should finalise the report to the Mpumalanga Education Department, and that the Executive Officer should alert Dr Ihron Rensburg of the central Department of Education of the fact that an investigation would be continuing.

MODERATION OF 1998 SENIOR CERTIFICATE SCRIPTS : MPUMALANGA

When the Mpumalanga Education Department announced very much improved results for the 1998 Senior Certificate examinations the Executive Officer decided to set the process of moderation of answer scripts in terms of the Council's regulation 2(e)(ii) into motion as soon as possible. Samples of specific candidates in Biology HG & SG, History HG, Business Economics SG, Economics HG and Geography HG were drawn, and their examination numbers were supplied to Mpumalanga with the request that their scripts be made available for moderation. It should be noted that the raw marks in the six subjects mentioned were considerably higher than in previous years, as can be seen from the details for Biology HG which are attached on pages 3 and 4.

The report from Mrs A Crowe, the moderator for Biology HG, is given on pages 5 and 6. The Executive Officer noted the concern expressed under point 2, and examined the scripts himself. There were indications that they were not the scripts of the candidates nominated for the sample, but the scripts of other candidates from which the cover pages were removed and replaced by different cover pages bearing the examination numbers and centre numbers of the candidates originally selected. In addition there were indications that marks were added to the marks which candidates obtained in the examination before these marks were entered into the computer.

The Executive Officer felt that in order to confirm or allay the suspicions aroused by these indications it was necessary to look at some more scripts, but without the intervention of any officials involved in the drawing of the original samples. He visited the Department for this purpose, but could not get access to the scripts. He was informed by the Head of the Department and the MEC for Education that they were of the opinion that they should first be given a report on the moderation of the scripts already moderated ~efore further steps flowing from that report could be taken. The MEC also indicated to him that this report should not come from him as an individual, but should be a report from the Council.

A telephonic report from Prof S L Barnard, the moderator for History, was similar to Mrs Crowe's report, and he also mentioned indication of tampering with the scripts.

Continuing with the process of selecting samples of scripts for moderation the Executive Officer requested the scripts of specified candidates in Afrikaans and English Second Language HG, History SG, Mathematics HG & SG, Biblical Studies HG & SG, Business Economics HG, Economics SG, Agricultural Science SG, Physical Science HG and Geography SG. The scripts for Business Economics HG, Agricultural Science SG, Biblical Studies HG and Economics SG have been delivered already.

Amongst the scripts for Business Economics HG there are two scripts, one in English and one in Afrikaans, both with examination number 7907175147080 and centre number 8771 and a mark of 219, but with different questions answered. Also amongst the scripts for Business Economic HG there is a second pair of scripts, one in English and one in Afrikaans, both with centre number 8759 and examination number 8006050060085. Of this second pair the English script has a mark of 248, while the Afrikaans one has a mark of 248 which was amended to 266 in pencil, possibly when the addition of marks was checked. On the computer system candidate number 8006050060085, has a mark of 248. Both these candidates wrote Afrikaans First Language HG, so that the Afrikaans scripts are probably the authentic ones.

Amongst the scripts for Economics SG there are two scripts, both in English, both with examination number 1822854 and centre number 1469 and a mark of 128, but with different questions answered.

These duplications are obvious evidence of tampering with the scripts, and are lending credibility to the allegations mentioned earlier. There is no apparent reason why it was necessary to tamper with the scripts for Business Economics HG and Economics SG. In the case of Biology HG, and perhaps History HG, it seems likely to assume that the tampering was done to conceal some irregularity, and one possibility is that it is the fact that marks were added to the marks which candidates obtained in the examination.

At a press conference on the 1998 Senior Certificate results the Minister of Education indicated that he did not intend becoming involved in the issue around the improved results of Mpumalanga, but that it was the task of the Certification Council to ensure that the results were correct. With these indications that an irregularity might have occurred, the Council should consider how the matter is to be investigated further.

A short meeting was quickly arranged wherein the Department raised professional concerns in the handling of this matter. The Department felt that it deserved to be treated with recognition, respect and trust. In that meeting the Department requested SAFCERT to table a progress report on the scripts already collected. It was finally agreed that a progress report would be tabled as requested.

In a closed meeting between the MEC for Education and the CEO of SAFCERT an agreement was reached that the progress report be tabled by the following day ( Friday, 8 January 1999 at 17h00 ) at Middelburg.

From previous interactions the MEC was concerned that subsequent communications with the CEO should be with the mandate of the Council. He thus requested that the report should be from the Council.

4. THE CONCERN OF THE DEPARTMENT

The CEO of SAFCERT received the statistics five days before the ogiving date i.e. 23/12/98, which gave him ample time to study and understand both the statistics and comparison tables. For continuous assessment and written examination.

On the ogiving date which was the 28/12/98 the CEO raised concerns around the following areas :

• Continues assessment marks for Biology. Higher Grade and Standard Grade.
• That there were no reamers who scored a zero (0) mark in History both Higher Grade and Standard Grade.

The Department agreed that the continuous assessment marks in Biology could be removed; but the CEO indicated that there is no need to do that for the l998 examinations but that an investigation needs to be made in pursuit of standardising these marks. He then visited the system's-section to make sure that the adjustments were made as he wanted. The CEO and his entourage left the Department saying that the process was satisfactory. The remark made was that there will be a slight increase in the pass rate. We were also concerned that the increase was not slight as projected by the statisticians.

On the evening of the 30 December 1998 Mr Mello, the Administrative Secretary to the MEC was phoned by Mr Mseleku - D.D.G. Human Resource and Labour Relations in the National Department of Education. Mr Mseleku indicated that SAFCERT seems to be concerned with the Mpumalanga results. A matter which was never raised wills the Department at any point. Mr Mello took the matter up with both the HOD and the MEC the following day (31 December 1998). It concerned us that he had ample time to study the statistics

and detect problems, raise them at the ogiving meeting what were the problems. In the Department via if he was not reaching consensus with the delegated officials he should have sought the intervention of their principals

The Department, through the HOD tried to make arrangements with the CEO to attend to his concerns but the CEO still maintained that the Department could proceed with the announcement as arranged for Monday, 4 January 1999. Various options were muted without any taken by the CEO. The process of moderation which would have taken place much later was brought forward to check what could be the cause of the increase. This is a normal procedure after examinations are over and has no impact usually on the results. The Department agreed to comply with SAFCERT on that process.

4.1 THE DEPARTMENT HAS PROBLEMS WITH THE FOLLOWING:

• That the CEO did not raise his concerns officially with the HOD not the MEC, but the Department heard telephonically through the National Department of Education.
• That the CEO was given the second chance to attempt to intervene and satisfy himself on the 31 December 1998 but mislead the Department by saying there is nothing wrong with the results.

The Department's assessment is that the inability to standardise and correctly project the impact of the implementation of CASS is being clouded by other processes (such as unusual moderation) which makes us to loose focus on where all the abnormalised should have been normalised and made "acceptable" that is now placing their inability to standardised the results as a departmental problem whilst it is in fact their problem.

SAFCERT has had the opportunity to standardise the results both on the 28 December 1998 and on the 31 December after they have raised their concerns through the National Department and the Department has offered an opportunity to reassess the situation on 2 January 1999 or somewhere during the first week of January but not later than 7 January 1999 as raised earlier.

5. CONCLUSION

The focus in the view of the Department should be on the failure to normalise the result or since the matter was so sensitive and important even alerting the principals on the situation and its consequences as statisticians.

Any examining authority can experience a number of problems, difficulties and flaws in the running of an examination process. At times papers might leak, the examination question papers could be too difficult or too easy, markers could be very lenient or strict, there is ever so many irregularities which might occur in this process to this end.

Through legislation the National Minister has put SAFCERT in-charge of the Standardisation of the examination results.

With the introduction of continuous assessment for Grade 12 in Mpumalanga, which was submitted to the CEO, there should have been special attention paid to how it impacts on the entire examination. As such the options given to the CEO after the results were calculated should have been implemented if the Department had the authority to do so, but this was treated casually by the CEO.

It is very unfortunate that communication from the Council does not go through line-function as such the Head of Department is kept in the dark but has to contend with handling these issues in the public as they arise. We recommend that the formal lines be utilised and other structures in the Department be acknowledged e.g. the Provincial Examination Board in which SAFCERT is represented. The Board has sub-committees such as Irregularities Committee, Moderators Committee etc.

The Department would finally like to complement the approach adopted by the National Department of Education in relation to this matter. We welcome all advises on how to approach this matter in view of how it impacts on the lives of learners, parents and the community in general.

We hope that your good office will find this information useful and of value as you deal with this matter at the National level.

Kind regards