Bunn came off the bench for injured Nahki Wells in the 61st minute and cracked a 35-yard shot into the roof of the net for the best goal of his promising career.
It crowned a week in which he had signed a new contract that will keep him at Town until the summer of 2019
Yet the 23-year-old admitted: “Only three points would have made it the ideal week for me.”
The Manchester City product let fly in the 84th minute to set up a thrilling finale in which only the outstretched boot of goalkeeper Frank Fielding denied Kyle Dempsey an 88th-minute equaliser as he bundled through a crowded goalmouth.
Town had been undone by an inability to defend set pieces and the failure to convert a penalty in the first half, enabling the Robins, who recently lost 3-0 at the Millers, to sit back and hit on the break.
Penalty culprit Emyr Huws also took a knock which forced him off soon after Fielding had gone down to his left to palm away the spot-kick and central defender Aden Flint slid the ball clear before the Town midfield man could atone by converting the rebound.
It resulted in manager David Wagner giving Wells a forward partner at the break in the imposing shape of Ishmael Miller and he and Dempsey played their parts in Town’s second-half qualified recovery in the incessant rain and wind.
Bunn agreed, saying: “The manager talked about it the other week, wanting the subs to come on and make more of an impact. So, personally, it was nice to get the chance to come on and score.
“With the conditions it makes it hard for the ’keeper so I just picked my head up and saw there was a chance to shoot and, thankfully, it went in the top corner.
“It is probably at the top of the list as the best goal of my career as most of my others have been from inside the six-yard box.”
After his 12th goal in 53 appearances, he added: “Every player wants to start, but I have not started the last three games and with being suspended against Middlesbrough, but, hopefully, that goal has put me in the frame for Tuesday.
“That will be a game which will have a bit of spice added with it being a derby.
“Every game is a big game for us, but it being a Yorkshire derby adds a bit more to it and we have to get the three points.
“With Bristol being around us this was the sort of game we wanted to win but, unfortunately, we haven’t done that so we have to look forward to the big game on Tuesday and pick up the three points there.”
Bunn did not put defeat down solely to Bristol’s two goals inside the opening half hour.
“Set pieces are a big part of the game and, unfortunately, we did not do ourselves justice in defending them, but we also had several chances to score and equalise in the second half.” he said.
Huddersfield were hoping to build on the victory at Birmingham the previous week, when they lost Dean Whitehead for up to eight weeks with a knee injury that prompted a recall for Jonathan Hogg sitting in front of the defence alongside Huws.
Instead, they suffered a fourth successive home defeat for the first time since September, 2000. They went behind in the seventh minute as Bristol were handed a free-kick on the left and Jonathan Kodjia glanced home the delivery from Marlon Pack.
Town should have been back it in when Joe Lolley sent Wells down the left channel and the striker had his heels clipped by Luke Ayling for a penalty, but Huws did not get enough power behind his low spot-kick.
Town went 2-0 down in the 29th minute after conceding another disputed free-kick as Bristol pressed down the left. This time, Flint was unmarked as he headed home from Luke Freeman’s chip to the far post.
Wells twice tested Fielding, who held a shot down by his right-hand post before beating away another to his left for a corner.
However, it could have been worse for Town as Aaron Wilbraham failed to connect with a driven cross from Elliott Bennett in the 40th minute as he slid in at the far post and Liam Moore volleyed over from inside the six-yard area.
Town struggled to make an impression as the Robins sat back, but did go close when Lolley cut inside and saw his shot shave the outside of a post in the 66th minute.
Freeman, who used all his experience in midfield to disrupt Town from getting into anything like free-flowing mode, had an opportunity to make it 3-0 but chipped over from 35 yards after intercepting a poor clearance from goalkeeper Jed Steer, who was struggling to get back in his goal.
Ben Chilwell then went close after racing on to a return pass from Bunn, whose stunning shot gave Town a glimmer of hope before Fielding’s boot prevented the equaliser.
Huddersfield Town: Steer; Smith, Hudson, Cranie, Chilwell; Huws (Dempsey 18), Hogg; Scannell, Paterson (Miller 46), Lolley; Wells (Bunn 61). Unused substitutes: Allinson, Davidson, Carayol, Wallace.
Bristol City: Fielding; Ayling, Flint, Moore; Bennett, Smith, Freeman Little (85), Pack, Bryan; Kodija (Cox 78), Wilbraham (Agard 75). Unused substitutes: O’Leary, Robinson, Reid, Burns.
Referee: J Simpson (Lancs).