Substitute Malike Sako ended Torquay's run of 10 league games without a win with a thumping left foot drive.
Barely two minutes were left before added on time when the 6'6" midfielder got the ball and drove it home from 20 yards.
Sako who had replaced Matt Hockley at half time was then booked for ripping off his shirt and waving it in celebration at the small group of travelling fans from South Devon.
It was Torquay's first away win at Rovers for nearly 60 years but came on the back of three successive draws as they fight relegation.
They could have been a goal down inside five minutes and would have been but for the sharp reflexes of Andy Marriott.
The keeper flung himself down to keep out Stuart Campbell's free shot and then beat away the angled follow up from Richard Walker.
It was all Rovers for the first 20 minutes and a busy start for Marriott who had to deal with a barrage of shots.
When Tony Bedeau finally broke out for Torquay he was pulled back for the way he barged past Chris Carruthers but at least it sparked a feeling of self belief among Torquay's players.
There was a quick follow up threat from Bedeau with a far post header after Alex Lawless squeezed a corner out of Chris Disley.
The action was now much more evenly balanced and Torquay's tactics of starving the home front runners Junior Agogo and Walker was paying off.
Even when Agogo began to start his runs from the left Craig Taylor and Adam Lockwood neutralised his crosses.
Agogo eventually limped off as the game drifted towards a likely goal-less stalemate grew. But the giant figure of Sako had the final say.
The disappointment was made worse for Rovers for after three successive wins they were looking for their best run in six years.