Government spokesperson Jimmy Manyi told media in Parliament on Thursday that he would not comment on an open letter by Minister in the Presidency Trevor Manuel in which he was labelled a "racist".

"Yesterday [Wednesday] I already responded to that matter to say I have no comment on the matter. I continue to have no comment on the matter," Manyi said at a post-Cabinet meeting media briefing.

He said he was addressing the media on Thursday in the capacity of cabinet spokesperson.

"Today I am wearing one hat. Cabinet spokesperson. That's all we are going to deal with. I know some of you are interested in a whole range of things, but I will not allow the Cabinet airtime to be abused.

"Some of you may be interested in the letter issue of minister Manuel and I can tell you right now that Cabinet did not discuss the matter so I will not take questions on that."

In the open letter, which was published in the Mercury newspaper, Manuel hit out at Manyi over his comments on coloured people.

"I want to draw your attention to the fact that your statements about 'an over-concentration of coloureds' are against the letter and spirit of the South African constitution, as well as being against the values espoused by the Black Management Forum since its inception," Manuel wrote in the letter.

"I want to put it to you that these statements would make you a racist in the mould of HF Verwoerd [Hendrik Verwoerd, former South African prime minister, dubbed the 'architect of apartheid']."

Manyi, then the director-general of labour, said in a show broadcast on KykNet's Robinson Regstreeks in March 2010 that there was an "over supply" of coloureds in the Western Cape.

Asked if Manuel had been at Wednesday's Cabinet meeting, and what the atmosphere had been like between the two, Manyi confirmed that Manuel had attended the meeting.

"Yes, indeed, he was at the Cabinet meeting. We shook hands and were cordial with each other," he said.

To a further question on how confident he felt speaking for Cabinet, given that Manuel had lost confidence in him, he replied: "Very confident. Can't you read the confidence just looking at me?"

On Wednesday the African National Congress withheld any support for Manuel's scathing attack on Manyi as the Congress of SA Trade Unions (Cosatu) sided with Manuel and pressure piled on the government spokesman over his controversial race remarks.

In the ruling party's only comment on Manuel's outburst against Manyi, secretary general Gwede Mantashe hinted that the ANC heavyweight and planning minister in the presidency had acted without the party's backing.

"He does not want our view. If he wanted our view he would have written a letter to us. He went into the open. We won't join the match, we won't get into that mud with him," Mantashe said.

Cosatu general secretary Zwelinzima Vavi said he agreed that Manyi's statement about an "over-concentration" of coloured people in the Western Cape was racist and further comments along this line would call into question his fitness for high office.

"I agree with Manuel. He is right. That remark of Jimmy's was a most unfortunate statement ever to be made in a democracy," Vavi said.

"It is a bizarre statement. I don't know what he had smoked to make that statement... It is absolutely racist, in fact you can't put it in any other words.

"We'll be checking very carefully as to whether he will be making other such statements because it then calls into (question his) fitness.... whether he is fit for the position that he has just been promoted to if he harbours such views."

Manuel's former Mbeki-era Cabinet colleague Kader Asmal urged the government to make a moral choice between the views of Manuel and those of Manyi.

"Minister Manuel deserves the support and praise of all right-thinking South Africans," Asmal said in a statement.

"The choice facing us is very clear: do we stand behind the humane and generous values of Minister Manuel, or do we, by staying silent, lend our support to the mischievous and dangerous notions of Mr Manyi?"

The Democratic Alliance urged President Jacob Zuma to fire Manyi, after releasing a recording of remarks he made about Indian South Africans last year while he was still director-general of labour.