Hereford United had a mean Christmas of four games without a goal against them but that came midway through the first half.
Coming off his line keeper Wayne Brown crashed to the ground with Rovers' Craig Disley beneath him.
The penalty decision was clear enough for Cheshire referee Scott Mathieson and Walker despatched it efficiently with a straight thump for his 13th goal this season.
Earlier Mathieson had got a decision spectacularly wrong after Hereford's Trent McClenahan had gone in with a late tackle on Aaron Lescott.
With Rovers breaking out the referee waved play on but indicated with a pointed finger he intended to go back to McClenahan.
The only trouble was when that time came he couldn't identify him again.
He held up play to consult the linesman and the fourth official. They couldn't help and McClenahan kept quiet to escape punishment.
In the hard-running first half of no-quarter tackles there were bookings, though, first for Rovers' Craig Hinton who kicked the ball away at a free-kick and Hereford's Andy Ferrell who caught home skipper Stuart Campbell with a high tackle.
McClenahan's contribution to the half was not over for, near the break, his header off the line stopped Hinton doubling the lead.
The speed of Simon Travis down the left channel caused early problems for Rovers who had to double mark him and then from 30 yards the visitors' top scorer Rob Purdie let fly with a vicious drive which Steve Phillips was relieved to see wing inches outside his right-hand post.
Campbell began to influence the tie more and more. One of his mazy runs ended with Brown hauling down a final pass aimed at Ricky Lambert in the centre of the area.
The next time he had the ball Campbell, who hasn't scored for a year, hammered a shot a foot or so to the wrong side of the post.
Tim Sills, lurking at a corner, came close to an equaliser as the half ended while Brown saving well from Walker.
Hereford set out in the second half to take control of the midfield and for a time Purdie and Ferrell gave it to them
In attack Andy Williams, a striker Rovers wanted to sign in the autumn, took over from Luke Webb, but it was Sills again who drew a save from Phillips on the run as he cut in from the left.
Scheming hard to put Rovers back in, Campbell lofted the ball over defenders on the right, catching Hereford flat-footed.
Lescott, who had seized possession, got in a low cross but under pressure Walker failed to make the connection.
Another Campbell-inspired raid ended with Brown bringing off an alert clutch to deny Walker, but the striker knew the header should have been tucked away.
In the three minutes of time added on Hereford twice came close to earning a replay, first with a shot from Sills which was parried by the keeper and then a lob from Alan Connell which lacked pace.