After going behind to a first-half penalty, they had looked well off the pace and Walker was hardly cast as the hero.
His confidence was low after scoring only once in ten games and the crowd thought coach Paul Trollope was wrong to pull off Lewis Haldane on the hour and leave Richard Walker on.
But as he hustled in his goals, the first from a touch by young loanee James Walker and the second from a cross by Aaron Lescott, his coach's decision was vindicated.
This was the first time Rovers had scored twice this season and Richard Walker came close to ending with a hat-trick only to hit the bar before directing an audacious flick towards the far post.
It was a tricky night for both teams whose form had slumped over the previous five games.
But they quickly settled down to a high-tempo match with identical attacking lines of two front runners with a support striker.
While Hereford manager Graham Turner was happy to rely on three at the back, Rovers stood by their normal back four and let themselves short in midfield.
They were quickly punished for this but the real punishment came from a bad tackle by Lescott.
In the 18th minute he brought down Trent McClenahan in the box and Rob Purdie thumped the spot-kick wide of Steve Phillips' right hand.
Phillips went on to make two saves from Stuart Fleetwood and another from Richard Rose as Hereford got well on top.
When Haldane was brought off, Rovers found extra pace upfront from teenage striker James Walker.
The newcomer made a quick impression with a bicycle kick leading to the equaliser before Richard Walker turned the entire game.