The teenager got off to a dream start with a goal on his debut after 20 minutes and Rovers were rewarded with an above-average crowd.
There was a down side though as the high winds forced Rovers to close one of their stands and Michael Collins hit an 88th-minute winner for Huddersfield who have not lost at the Pirates for 20 years.
The wind sweeping down the pitch made the quality of football a lottery in the first half but for Klein-Davies the only thing that mattered was he scored.
The build up came from a corner on the left taken by Rovers skipper Stuart Campbell. Rickie Lambert sent his header towards the far post. It hung in the air but, seizing his moment, Klein-Davies was there to nip in at the post with the final nod.
The lead lasted 19 minutes before the experienced campaigner Frank Sinclair moved up from Huddersfield's back line and squared a neat pass to James Berrett who had time and room to place his shot well out of the keeper's reach.
With the wind behind them Huddersfield, looking to follow up a midweek win over Brighton, went on to dominate the rest of the half.
They might have scored again when a misplaced ball by David Pipe brought panic to Rovers' wrong-footed defence. But they had to settle for a corner after a desperate scramble by goalkeeper Steve Phillips.
Rovers looked harshly treated when referee Paul Melin knocked off what looked to be a good goal by Pipe, a decision which had the managerial duo of Paul Trollope and Lennie Lawrence protesting on the edge of the technical area.
With the wind dropping, football began to have a chance. Young Klein-Davies supplied the 48th minute ball which saw Chris Lines give Rovers back their lead.
Again it was short-lived with Chris Brandon equalising from a ball knocked down to him by Luke Beckett and with only two minutes left Collins grabbed a dramatic winner for the visitors.