The former Prime Minister had failed to demonstrate the "moral seriousness" he had promised, while "spin" and "high-handedness" intensified, he said.
Mr Miliband, who is now running to succeed Mr Brown as Labour leader, said the party had at the same time lost its optimism and ability to articulate "aspiration and hope".
The Shadow Foreign Secretary said he had agreed with Mr Brown
completely when he took over from Tony Blair.
"I supported and voted for him," he said in a speech in Wales. "I agreed that we needed greater moral seriousness and less indifference to the excesses of a celebrity-drenched culture.
"I agreed with him when he said that we needed greater coherence as a government, particularly in relation to child poverty and equality.
"I agreed with him on the importance of party reform and a meaningful internationalism that would be part of a unified government strategy.
"I agreed that we needed a civic morality to champion civility when confronting a widespread indifference to others.
"But it didn't happen."
Mr Miliband was publicly loyal to Mr Brown when he was foreign secretary to the then prime minister, although he was always suspected of harbouring leadership ambitions.
Speaking in the Keir Hardie Memorial Lecture in Wales, he said Labour's problems had got worse after Mr Brown became leader.
"It was not just more of the same," he said. "Far from correcting them, failings – tactics, spin, high-handedness – intensified, and we lost many of our strengths – optimism born of clear strategy, bold plans for change and reform, a compelling articulation of aspiration and hope.
"We did not succeed in renewing ourselves in office – and the roots of that failure were deep not recent, about procedure and openness, or lack of it, as much as policy," he said.David Miliband has criticised Gordon Brown's record in office, saying Labour's failings had worsened under his leadership.