The Millers looked to have lost a game they were in charge of when, after Richard Wood’s sending off, former Rotherham striker Ryan Taylor headed Argyle in front with eight minutes remaining.
But centre-half Ajayi had the last word in a goalmouth melee by blasting home from four yards after Joe Mattock’s initial effort had been well-saved by Kelle Roos.
Taylor thought he had snatched all three points for Plymouth on his return to his hometown club where he spent six years until 2011, stooping to head home from a left-wing corner.
A defeat would have been harsh on the Millers, who dominated proceedings without ever really creating a glut of chances.
Will Vaulks, Ryan Williams and David Ball went close in the first half, with the latter two causing Plymouth constant problems with their direct running.
Rotherham’s superiority over lowly opposition came to an end eight minutes into the second half when captain Wood was sent off for a second bookable offence.
And after growing in confidence, Taylor’s header sent the 456 Argyle fans behind the goal into raptures, and the Millers fans who he blew a kiss to in celebration, in a state of anger.
But the mood was lifted by Ajayi’s dramatic late intervention.
Despite the drama, the Millers have not won at home in League One since October 14 as they slippede further off the play-off pace.
For all their dominance of lowly opposition, they failed to create many chances. Richard Wood’s dismissal was unnecessary and left Rotherham playing catch-up in a fixture they never really should have been chasing.