In his first speech in his new role, Andrew Stephenson said "the entire energy of government must now be behind" the 250mph line connecting London with Leeds and Manchester.
And he said he would be leading regular cross-government meetings as happened during the 2012 London Olympics alongside Transport Secretary Grant Shapps as he warned: "We must get this project right."
The Tory MP for Pendle was given the role overseeing HS2 as well as Northern Powerhouse Rail and the £2.9bn trans-Pennine route upgrade as part of Boris Johnson's announcement that the scheme would go ahead.
The scheme was the subject of a government-commissioned inquiry which threatened its future as concerns mounted over rising costs and delays to the scheme.
Mr Stephenson told the New Statesman's Northern Powerhouse conference in central Manchester: "The critics who say that HS2 will only benefit London are simply wrong.
"They ignore the voices in towns and cities across the globe. They ignore the businesses and passengers who are crying out for investment and change. They ignore people in my own constituency of Pendle.
"I'm proud to be a northern MP, tasked with ensuring these historic projects are delivered and run for the benefit of the people here.
"But that also means we have to have a realistic, pragmatic and hard nosed management of the costs, and the determination to unleash benefits which will revitalise the North.
"I've been a long term supporter of HS2, but I've equally watched with deep concern as costs have risen and deadlines have been put back. Very simply that can't continue, we need to have a much better improved approach from HS2 this time.
"The company has a new budget for phase one [connecting Londin and Birmingham], and must now deliver it, no ifs, no buts.
"Community engagement must improve too, too many times I've heard stories of unacceptable treatment of those impacted by this line.
"There will be disruption, we can't avoid that. But we must manage this work professionally and efficiently whilst always being sympathetic to those affected.
"And finally we need tangible stringent measures to ensure accountability, so that everybody knows their responsibilities with forensic scrutiny and a dedicated Minister invested in its success.
"Now I know that HS2 is up to this challenge. It has some fine minds and some excellent people already working on this project, but it needs to regain the public's trust, the entire energy of government must now be behind this project."
The Transport Minister said the Government would be bringing forward legislation for HS2 into Manchester as soon as practical and presenting an integrated rail plan for the North and Midlands.
This would examine how HS2 could work with the £39bn Northern Powerhouse Rail project connecting the great cities of the North and wider investment in rail projects.
Concluding his speech, he said: "Now austerity has been consigned to history, now we've delivered Brexit, it's time to repay the trust of northern voters, to show communities here that they've not been left behind, and to use the railway to deliver a new Northern Powerhouse."