Lambert was a thorn in the U's side all afternoon before coming up with the winner in the 53rd minute to chalk up his ninth goal of the campaign.
A minute before, Lambert had seen his goalbound header from Stuart Campbell's corner well saved by keeper Dean Gerken.
But he gave Gerken no chance with a similar header from the next corner by Campbell and hosts Colchester never really looked like finding a response.
It has been a dismal start to life in their new Community Stadium for the U's.
They are still waiting to secure their first win at their new ground following two draws and now three successive defeats.
Caretaker manager Kit Symons really needed a win to improve his chances of landing the job full-time, but his team were unable to find those three points despite a bright start which saw them almost take the lead in the 18th minute.
Clive Platt's header from a cross by John White looked a goal all the way until Rovers defender Danny Coles intervened with a goalline headed clearance.
After that, Rovers were the better side and they were denied an opening goal by a combination of the acrobatics of keeper Gerken and the woodwork in the 43rd minute.
A header by Chris Lines from a Craig Disley cross was brilliantly parried by Gerken, who pushed the ball on to the underside of the bar.
But the visitors did not lose heart and when Lambert finally broke the deadlock, there was no way back for the U's.
Their best chance of an equaliser fell to defender Matt Heath, who poked over the top following a corner by Johnnie Jackson.
Rovers have now leapfrogged over Colchester in the table and will be delighted to have chalked up their first away win.