The 23-year-old, who signed for the Millers in the summer of 2019 from Manchester United and went on to make a total of 72 appearances, was out of contract at the end of June and has elected to move elsewhere to seek a new challenge.
Millers manager Paul Warne said: "On behalf of everyone here, I would like to say what a pleasure it has been to work with Matty for the last couple of years.
"As we try to with every player that comes through the door, we have worked on making him a better player than when he joined us and I hope that he agrees that we have been successful in doing so.
"He leaves the club with absolutely no animosity as he looks to pursue a new challenge somewhere else and goes very much with our blessing.”
The Millers had been in discussions with Olosunde regarding a new deal, with goalkeeper Josh Vickers also spoken to regarding fresh terms.
But while Vickers has committed his future to the Millers, Olosunde has decided to depart for pastures new.
Last month, Rotherham announced their retained list, with Clark Robertson, Shaun MacDonald, Billy Jones, Jamie Proctor and Kyle Vassell all leaving the club at the end of the last campaign.