Walker helped the ball over the line at the far post after former Mansfield player Ryan Williams robbed Gus Uhlenbeek and crossed for Lewis Haldane to head goalwards just as the Stags had looked set for victory.
Stags loan striker Allan Russell lashed in his first goal for the club 16 minutes from time, but when Jonathon D'Lareya and Giles Coke both missed open goals in a tense finish, Bristol Rovers - the second most informed side in the division - took full advantage.
Teenager Nathan Arnold got his Mansfield career off to a remarkable start, smashing them ahead against the run of play in the 14th minute on an icy pitch.
Rovers' defender Steve Elliott could only flick on Jake Buxton's long throw near the corner flag and the unmarked Arnold pounced from 14 yards.
Nine minutes later Mansfield doubled their lead from a well worked free-kick as Adam Rundle run over the ball and into space before Uhlenbeek's resultant pass gave him the chance to score with a low shot through a crowded penalty area.
Within three minutes Rovers grabbed a lifeline, James Hunt stooping to delicately head in Chris Carruthers sweeping cross.
Keeper Scott Shearer's fingertip save kept out an Uhlenbeek pile driver as both sides mastered the tricky conditions.
That saved proved vital four minutes before half time when Walker crashed in a penalty, controversially awarded for holding, to complete Bristol Rovers' first half fight back.