The parliamentary faction of the Christian Democratic Union (CDU) and its small sister party, the Christian Social Union (CSU), voted by 195 for to 28 against, with 16 abstentions, to expel backbench MP Martin Hohmann.
A two-thirds majority of at least 166 votes was needed to remove him.
"It's a clear result," CDU leader Angela Merkel said after the vote, the first time the alliance has expelled a member of the group with a seat in the Bundestag, the lower house of parliament.
But she admitted: "It has been a hard day for everyone".
The decision does not mean Hohmann will lose his parliamentary seat as he can remain in the Bundestag as an independent.
In a speech to constituents on October 3, the national holiday, Hohmann said Jews could, with some justification, be seen as a "race of perpetrators" for their alleged crimes against civilians in the 1917 Russian revolution.
But he then went on to say Jews were in fact not a race of perpetrators - and neither were Germans.
In a brief written statement after the vote, Hohmann apologised for his speech - without retracting it - and promised to continue to work for his electorate in Neuhof, in the central state of Hesse.
The head of the parliamentary faction of the Greens, the junior partner in Chancellor Gerhard Schroeder's governing coalition, praised the decision to expel him.
While CDU leaders had been confident of getting the majority they wanted to throw him out of the parliamentary group, the vote showed he has some support.
A separate process to exclude him from the party altogether is underway in his home state and a decision is expected later this month.
CDU leaders have been desperate to end an embarrassing debate over their stance on Germany's troubled past and Hohmann's refusal to officially retract his speech has not helped.
They were also put on the back foot when the government quickly dismissed the general at the head of Germany's special forces troops after he sent Hohmann a letter of support.
Criticism of the CDU mounted, including from Jewish leaders, who said they were astonished he had only be reprimanded and dropped from a parliamentary committee rather than removed altogether.
CDU leaders have acknowledged there is a groundswell of support for Hohmann, some of it tacit, and have sought to avoid it breaking into the open.
Privately, some CDU deputies have told newspapers they are concerned he is being discarded for the sake of political correctness.
Even the public seems undecided about the row.
A survey released on Wednesday showed 41% of Germans thought the remarks by Hohmann were enough grounds to expel him.
Another 41% thought he should stay in the party.
The figures changed when poeple were asked if he should be thrown out of the parliamentary group - 43% said yes, 38 said no.
Meanwhile, the CDU mayor of Recklinghausen, western Germany, has been told he faces possible exclusion for hanging a sign in the party's local office lamenting: "No one is allowed to tell the truth in Germany any more". – Sapa-AFP.