Boro began the game in second place and a point behind leaders Leeds, who drew 1-1 with Brentford in the earlier kick-off.
However, Tony Pulis' side never found their rhythm against a Forest outfit who climbed into the play-off places with this victory.
Joe Lolley was central to it, scoring the first goal and setting up substitute Lewis Grabban to score the second, extending Forest's unbeaten league run to six games.
They had the upper hand from an early stage, Boro making a sloppy start and giving the ball away cheaply.
Jack Colback, the former Newcastle midfielder, tried an optimistic effort by letting fly from 25 yards and Lolley cut through the centre, taking the ball past three or four Boro players and seeing a low effort saved by Darren Randolph.
The home fans were becoming frustrated and Pulis was pontificating on the sidelines.
Middlesbrough improved, winning four corners before half-time and Martin Braithwaite hit the bar with a shot deflected by Colback.
Michael Dawson had to be replaced before the break, the Forest centre-back having come off second-best when challenging George Friend.
But losing the experienced defender did nothing to unsettle Forest, who took the lead early in the second half - and it was a rocket by Lolley.
After Daryl Murphy turned Daniel Ayala on halfway, the striker also evaded Ryan Shotton and fed the overlapping Lolley who took a touch before unleashing a powerful rising effort that Randolph had no chance of stopping.
It was the first time Boro had conceded at home this season and Pulis reacted by making a treble change, sending on Adam Clayton, Rudy Gestede and former Forest striker Britt Assombalonga, who was booed by the visiting supporters.
They were almost cheering when Lolley volleyed at goal after Aden Flint had headed a cross into his path.
Assombalonga wasted a great opportunity of an equaliser, his team's best chance of the game, midway through the second half when Jonny Howson fed him in a central position, his first touch was good, taking him into the area, but his shot was too close to Costel Pantilimon.
Grabban grabbed the goal to seal the points after 77 minutes, tapping in from close range after ghosting unmarked into the six-yard box to meet Lolley's pass from the left.
Boro were give some hope when Jack Robinson was red-carded having been shown a second yellow, but, aside from Gestede having an effort ruled out for a push on the goalkeeper, the home side did not come close to an unlikely comeback.
Middlesbrough were flat, putting in a below-par performance and conceded at home for the first time this season.
Manager Tony Pulis admitted: “I don’t think anybody saw that performance coming, but that’s the way the league is, we’re not going to be perfect every week and it brings everybody down to earth in respect of the expectations.
“It’s a tough league and everybody in every game turns up.
“We need to be much more clinical and we need to do something in that respect in the final third.”
Boro were on the back foot from the off, though Forest did not score until early in the second half, Lolley firing home after Daniel Ayala had been turned in the centre of the pitch by Daryl Murphy.
At 1-0 Pulis made a triple substitution with Britt Assombaloga among those sent on, but the former Forest striker missed a great chance.
Pulis added: “I thought we started really, really poorly, the first 20 minutes we were like a bag of potatoes, we were so sloppy and so on the back foot, we never, never got going.
“The goal was a dreadful goal and very unlike us. Then you need to score, you need to get back into the game as quickly as you can and the big, big point, the turning point was Britt’s chance.
“I think if Britt scores then on, what was it, 60-odd minutes? Then it’s a different atmosphere and a different approach from the team.”