Andy Robinson was sent off just before half-time. The midfielder clashed with Ryan Williams and after thrusting his head forward was immediately red-carded by referee Mark Cowburn.
An angry Robinson had to be pulled away from the scene by team-mates as others accused Williams of being the cause of the trouble.
The flare-up came after Williams had committed a touchline foul, one of 15 free-kicks against Rovers in the opening half.
There was always going to be a charged atmosphere to this all-ticket match which drew a big police presence, although the gate of 7,433 was well short of capacity.
About 1,300 of them had come from Swansea and they found plenty to cheer early on as some well worked dead ball routines put the home keeper Ryan Clarke under pressure.
Williams always full of busy running on the left flank got Rovers into the game, but it was one of their ex-players Lee Thorpe who helped put them in front. Failing to control a high ball punted into the goal area by Dave Savage, he presented it to Richard Walker who neatly steered it away from stranded keeper Willy Gueret.
It was his 14th goal of the season and his eighth in the last nine games. While Swansea were still trying to recover from their blunder they allowed Jon Bass room to swing in a cross from inside the right touchline with the defence flat footed.
Junior Agogo skimmed a header into the far corner for his 22nd goal.
An angry under-current out of the tussle between Swansea's top scroer Lee Trundle and Christian Edwards, who began his career with the Welsh club saw both warned by the referee.
Early in the second half Swansea made a double substitution and collected more cards as they covered up their one-man shortage in an all out second half.
Izzy Iriekpen blasted a shot on target which Clarke somehow stopped and then substitute Paul Connor, after running clear, directed his shot wide.
With normal time running out Rovers were also down to ten when skipper James Hunt saw red for an over the top challenge which caught Iriekpen on the right knee.