David Hoare said he had informed Education Secretary Justine Greening that he was resigning with immediate effect.
He said: “It has been a great privilege to chair the Ofsted board for the past two years.”
The former banker’s resignation comes just weeks after he was forced to apologise for his comments about the Isle of Wight.
Mr Hoare was widely criticised after being caught on tape at a teaching conference in Leeds making disparaging remarks about the Isle of Wight.
“Most people go there for sailing for two weeks a year. There’s a sailing club that is one of the best in the world, where there’s champagne,” he said in the recording, obtained by the Times Education Supplement.
“But just within inches, there are people who live in a ghetto ... They think of it as holiday land. But it is shocking. It’s a ghetto; there has been inbreeding.”
The comments were recorded at a Teach First conference in Leeds earlier this month.
He later issued an apology after his comments were reported by the TES, which prompted calls for his resignation.
Mr Hoare also made a telephone call to the leader of Isle of Wight Council, Jonathan Bacon, to make a personal apology, during which he offered to visit the island.
In a statement announcing his decision to quit, Mr Hoare said: “I have today informed the Secretary of State that I will be resigning from my position as chair with immediate effect.
“It has been a great privilege to chair the Ofsted board for the past two years. I am pleased that the organisation now has an excellent board in place with expertise across all of our remit areas, including early years and further education.”
He added: “I will miss working with an excellent team, making a real difference.”
Senior non-executive board member James Kempton has agreed to take on the role of chairman on an interim basis, Ofsted said.