Rovers seemed to have no chance when Torquay went two up in the 58th minute.
Leon Constantine had fired them in front in the 20th minute, breaking a 290-minute goal drought from the start of this season.
Big Frenchman Malike Sako won nearly everything in the air up front, while the combination of Agogo and Junior Louis did not click for Rovers.
Both the Pirates' strikers missed good chances in the first half.
Sako created the opener for Constantine and he was also involved in the move which saw Matt Hewlett score Torquay's second goal in the 58th minute.
But three minutes earlier Rovers manager Ian Atkins had sent on Richard Walker for Louis, and it turned out to be a masterstroke.
Rovers were still two down with only 16 minutes left, but in the 74th minute Walker set up Agogo for a rasping low left-footer which gave Torquay 'keeper Andy Marriott no chance, and the revival was on.
Torquay missed great chances through Matt Hockley and skipper Craig Taylor, but three minutes from time Agogo produced a brilliant solo run across the goalmouth, leaving three defenders in his wake before firing past Marriott for his fourth goal of the new campaign.
A point might have been some consolation for Torquay, but even that was snatched away in stoppage-time when Agogo returned the earlier compliment, holding off two defenders to set up Walker to curl a peach of a right-foot shot into the top right-hand corner for the winner.
The result leaves relegated Torquay, who were rated among the pre-season promotion favourites, rooted at the foot of the table with only one point, while the victory almost kick-started Rovers' season.