Brexit Secretary Steve Barclay's comments come as ministers mull over whether or not to go ahead with the high-speed line, with a decision expected in the coming weeks and a crunch meeting scheduled at Downing Street on Wednesday.
Whitehall's spending watchdog said last week that HS2 is over budget and behind schedule because its complexity and risks were under-estimated.
The National Audit Office (NAO) warned that it is impossible to "estimate with certainty what the final cost could be".
However Mr Barclay told BBC One's The Andrew Marr Show that the project was key to the Government's plans to raise the economic performance of all parts of the country.
"We have a strong commitment to levelling up all parts of the United Kingdom. High-speed two is a key part of that - not just from speed but more from a capacity point of view in the line.
"And that is a very clear commitment we have given the North."
Asked whether his gut feeling was that the project should go ahead, he replied: "Yes."
The intervention came as it was revealed scrapping the project would cost £12bn in write-offs and compensation.
The Observer reported that if the project was shelved immediately there would be an extra £3 - 4bn in costs, on top of the £9bn already spent on the scheme.
Andy Street, Mayor of the West Midlands, told the newspaper “very senior and influential business people have been in contact with the PM” to try and swing his decision.
He said it was “Narnia-land” to believe the project could be scrapped without huge costs.
He told The Observer: “Businesses have invested billions of pounds in the commitment of the British government from 2015.
“If you suddenly welch on that, where does it leave us as an international power where our word is our bond? I think that could be incredibly damaging.
“There would be a huge write-off. We are already committed to things that wouldn’t go ahead. There is also the question of how those who invested in good faith would respond to that.”
A Government-commissioned review led by former HS2 Ltd chairman Doug Oakervee leaked earlier last week stated that the project's bill could reach £106bn.
HS2 was allocated £56bn in 2015.
Phase One between London and Birmingham was due to open in 2026, but full services are now forecast to start between 2031 and 2036.
The NAO noted that the Department for Transport (DfT) set the available funding for the first phase in 2013, when there was only a "basic" design for the project.
Transport Secretary Grant Shapps has said the Government will make a decision on whether to go ahead with the project in "weeks rather than months".
Meanwhile plans for HS2 in Manchester have been described as “utterly baffling” as current designs for platforms were the wrong way around for passengers travelling to Leeds.
The Sunday Times reported how because the plans only considered HS2, and not the east-to-west Northern Powerhouse Rail scheme, those arriving at Manchester airport who want to travel through the city to get to Leeds would find the platforms face towards Liverpool instead.
A Greater Manchester transport source told The Sunday Times: “The Government is planning to spend hundreds of millions of pounds on a plan which currently requires the train drivers to leave the train, walk along the platform, get into the back of the train and reverse out of the station. That is utterly baffling.”