The Millers were looking good value for a point until a quality finish from Republic of Ireland midfielder Jeff Hendrick floored them eight minutes from the end, although they twice went close to equalising in stoppage time.
Rotherham manager Steve Evans thought his team was worth a point on their return to the Championship.
“I think Derby supporters will go away and know their team was in a tough game today,” he said.
“They will have a lot of easier games at the iPro, that’s for sure. I think we deserved a point and I think the whole ground knows we deserved a point but sometimes you don’t get what you deserve.
“Many people around the country thought when they saw we were away at Derby it would be lambs to the slaughter but we aren’t going to be lambs to the slaughter for anyone.
“We were well organised and after they scored, we had two great chances to equalise we would normally take so we are disappointed we didn’t take them but we’re learning.”