Thompson, who joined the Minstermen earlier this month, pounced after just six minutes but Green, in his second spell with the Stags, made sure his team were in front by the half-hour mark.
Poor misses from Vadaine Oliver, Russell Penn and James Berrett meant York failed to equalise and Chris Clements could have added a third late on but his 25-yard strike hit a post.
Disappointed City chief Russ Wilcox, who saw his side booed off the pitch by some sections of the home crowd, said: “We’ve wasted a great opportunity to get into the top half of the table because a win would have taken us into 12th place.
“We scored first and had enough attempts at their goal but our quality was poor in terms of the final ball and final execution. I know there’s better quality in the dressing room, though, than we showed out there and I feel we have a better group of players than I inherited last season.”
Thompson netted from two yards when he got on the end of Luke Summerfield’s inswinging free-kick from the left flank. Green levelled when he reacted quickest in the six-yard area after Adam Chapman had helped on Reggie Lambe’s low corner.
The ex-Birmingham City striker settled matters when he collected a pass from Craig Westcarr and beat Scott Flinders with a firm 15-yard drive.
York City: Flinders; Lowe, Winfield, Ilesanmi; McCoy (Sinclair, 65), Penn, Summerfield (Godfrey, 46), Berrett, Tutonda; Oliver, Thompson (Carson, 76, 6). Unused substitutes: Ingham, Rzonca, McCombe, Nolan.
Mansfield Town: Jensen; Hunt, Pearce, Tafazolli, Benning; Lambe (Collins, 77), Chapman, Rose (J Thomas, 60), Clements, Westcarr, Green. Unused substitutes: Marsden, Beardsley, N Thomas, Yussuf, McGuire.
Referee: T Harrington (Cleveland).
Man of the match: Matt Green.