Former Bristol City player Kayode Odejayi pushed Rovers deeper into the relegation mire as his first-half strike gave Colchester all three points.
Penney, who failed to show up for the post-match press conference, told Rovers` official website: "We didn`t start the game well, they were better than us, and we got exactly what we deserved.
"The difference in the level of performance against Oldham and Colchester is marked and we cannot continue to have that. We didn`t play well, I don`t think we made the `keeper make a save.
"They did things better than us as a team, they put balls in better areas and deserved to win. They passed it better than us, they worked harder and they had more quality - the bottom line is they were better than us in all departments. We were not good enough and we have to put it right on Saturday."
Meanwhile, on-loan midfielder Scott Davies has returned to parent club Reading after his one-month deal was not extended.
Source: PA
Source: PA