The Millers, whose relegation to League One was confirmed earlier this month, gave Warne his first win in permanent charge thanks to Tom Adeyemi's 79th-minute goal, while also avoiding an unwanted Football League record.
The three points takes them past Doncaster's record low for a 46-game season of 20 set in 1997-98. The Millers need to win their remaining two games to avoid setting the lowest ever second-tier points haul of 26.
Warne said: "It's a monkey off the back, I am in charge of this team, I am the one who took us down, so I want to finish the season on a high.
"To get to 21 points is a relief, I am happy about the win as I am the clean sheet. For us to get over the line, I am happy."
The Millers, who missed an early penalty, have improved since their drop was confirmed and Warne is confident there is more to come from his group.
"It is nice to win a football match, it is nice to keep a clean sheet," he added.
"We didn't have as much quality as the last few games so that just shows what football is about.
"I am really proud of how they kept a clean sheet and got over the finish line.
"They have got to believe in themselves more because they have more to give.
"As good as they were, and they have to be proud, but I am excited by the fact they have more to give."