Sterling was the beneficiary as the ball rebounded off him into the net six minutes from time to break the resistance of the determined Terriers at the John Smith’s Stadium.
Huddersfield had claimed the lead courtesy of a Nicolas Otamendi own goal just before the break, but City responded with a Sergio Aguero penalty.
The visitors then went on to dominate the second period, but appeared to be running out of ideas when Sterling accepted a late gift for his 12th goal of the season.
Town forward Rajiv van la Parra was sent off straight after the match following an altercation with Leroy Sane, who was booked.
The game kicked off amid a fevered atmosphere and the passionate home support – much as it did for last month’s unexpected defeat of Manchester United – energised the Terriers as they blunted City’s attack.
City were composed, but it soon became clear they would have to be patient, and wary, as Huddersfield defended deep and threatened on the occasional counter-attack.
Aguero did put the ball in the net early, but his effort was ruled out for offside and Huddersfield had the first clear-cut chance when Tommy Smith cut inside and fired over.
It was some time before City pieced together a move that stretched the home defence, but Aguero scuffed wide.
Aguero had another opportunity after neat build-up play between David Silva and Kevin De Bruyne, but his touch was poor and Christopher Schindler made a vital challenge.
City turned up the heat with Otamendi firing over and Aguero forcing Jonas Lossl to save before Sterling shot wide on the rebound.
It was not one-way traffic, however, with Mathias Zanka diverting wide after an Aaron Mooy cross reached him at the back post and Tom Ince getting behind the defence to win a corner.
Another corner followed after Vincent Kompany headed dangerously close to his own goal and this time it proved critical as Ince’s set-piece was met by Schindler and deflected in off Otamendi.
The goal came in first-half stoppage time and the lead lasted little longer than the length of the break.
Soon after the restart Sterling appealed for penalties after challenges by Lossl and Scott Malone. The second was given and Aguero stroked the ball into the bottom corner.
City almost claimed the lead when Sane struck the bar with a curling free-kick, but the game then became scrappy for a spell.
Huddersfield were incensed after van la Parra went down following a challenge by Kyle Walker and referee Craig Pawson needed to speak to home manager David Wagner.
Fernandinho was booked for diving on the edge of the area as van la Parra swung his leg in his direction.
The final 20 minutes were played out almost entirely in the Huddersfield half as City pushed for a winner. Visiting manager Pep Guardiola sacrificed Kompany for the final onslaught and sent on Gabriel Jesus.
Huddersfield defended superbly, but the breakthrough finally came as Jesus’s shot was blocked by Lossl, hit Sterling and fortuitously bounced in.
Van la Parra and Sane then clashed after the full-time whistle, raising arms to each other in an unsavoury incident which saw the former red-carded and the latter cautioned.
Huddersfield Town: Lossl, Smith, Jorgensen, Schindler, Malone (Lowe 74), Ince (Quaner 82), Williams, Hogg (Mounie 86), Mooy, van La Parra, Depoitre. Unused substitutes: Kachunga, Cranie, Green, Hadergjonaj.
Manchester City: Ederson, Walker, Kompany (Gabriel Jesus 80), Otamendi, Delph, De Bruyne, Fernandinho, Silva (Mangala 89), Sterling, Aguero (Gundogan 86), Sane. Unused substitutes: Bravo, Danilo, Bernardo Silva, Alexander Zinchenko.
Referee: Craig Pawson (South Yorkshire).