Mr Osborne, the architect of the Northern Powerhouse agenda, told the Great Northern Conference that Boris Johnson needed to be "more courageous" in giving local leaders the ability to make more decisions in areas like transport, skills, business support and health.
His plea was backed by Sheffield City Region mayor Mr Jarvis, who said handing over more than just "modest resources and powers" would help address "the disillusionment and division that have had such a corrosive effect on our bonds as a society and as a country."
It comes as Mr Johnson faces an ongoing backlash after imposing Tier 3 coronavirus restrictions on Greater Manchester against the will of Mr Burnham, after talks over a package of funding support broke down.
Mr Osborne, who is chairman of the Northern Powerhouse Partnership, introduced the idea of metro mayors as part of a series of devolution deals with English regions when he was chancellor.
He said that things were currently "incredibly difficult" because of the lockdown restrictions and tensions "between some of the great leaders of the northern cities and the government about what should happen next".
But he said: "If you look beyond that, what I can see is real representation in the north devolution, delivering leaders of Greater Manchester, of Liverpool, of different parts of Yorkshire, of Teesside, of Tyneside, and so on, and indeed, further south in the Midlands.
"And we now have a national political conversation, where representatives of the North of England are heard on our national news. And that is a fantastic success story.
"So let's look beyond the immediate problems and think about the future. And I would stress three areas we need to look at as we get through this disease and start to think about recovery.
"The first is devolution, what more can we do to make devolution real, we've created these new institutions, we've created these elected metro mayors, we've got to make sure that all parts of the North of England are covered by this kind of devolution.
"And then I think we need to be more courageous in giving many more powers to these local elected leaders over decisions like transport, like skills, like business support, issues like health care, which we're seeing, for example, in Manchester, where there's real control over the NHS, a level of devolution, that doesn't exist anywhere else in England.
"I would want to see us double down on devolution, because I think we're beginning to see a much more balanced healthy British society and states where not every single decision is taken in Westminster and other voices are heard. But we've got much more to do."
The former Chancellor, who is now Editor-in-Chief at the London Evening Standard newspaper, said the Northern Powerhouse was “much more than a slogan”.
Mr Osborne said: “In the six or seven years, since we’ve been working on the idea of the Northern Powerhouse, it has grown way beyond my wildest imagination.
“It’s obviously a slogan that has stuck and that’s great, but it’s so much more than a slogan.”
In his speech, Sheffield City Region mayor Dan Jarvis took aim at Mr Osborne as he pointed out that the UK has the worst regional disparity of any comparable nation.
He said: In the five years following the launch of the Northern Powerhouse in 2014, the number of children living in poverty in the North actually increased by a third, to 800,000.
"Again, policy has made or threatens to make this worse. Planned cuts to Universal Credit could leave one in three households in the North £1,000 a year worse off.
"Under austerity public spending fell £3.6 billion in the North even as it rose £4.7bn in the South East and the South West."
He said despite this, the North's strengths and potential meant it "is already a powerhouse – just one running far below its full capacity".
He said: "The potential rewards are great. It’s estimated that rebalancing national investment could add £97bn to the Northern and national economy by 2050.
"Failing to realise our potential is not just a waste, it’s an act of national self- harm. The case for levelling up is undeniable. It is both a deeply moral case, and a deeply practical one. The time to act on that case is right now."
The Barnsley MP said real investment was needed in the North, not just for infrastructure, but for education, skills, health, environment and flood prevention, or the Government would risk people losing faith in its 'levelling-up' agenda.
He said: "We’re doing what we can ourselves to help our local authorities and kickstart recovery – but with the limited resources and powers we have, we can only tinker at the edges.
"In terms of wider investment, an indication of what we need is the UK 2070 Commission’s recommendation for an additional £15bn a year for 20 years – a total of £200bn of new funding.
"That’s for all deprived areas, but it shows the scale of what we are talking about. So far though, it’s just not happened."
He said the vast majority of the £262m South Yorkshire has received from government in the last year was not specifically targeted to the county or the North.
He said: "This is not just about money. Both to be legitimate and to be effective, a plan for a Northern Powerhouse, must also empower.
"It must be done with us, not to us. That means doubling down on devolution.
"While we’ve done a lot, it’s been with modest resources and powers.
"Overall, devolution in the North and in the UK has been piecemeal, inconsistent and limited. We are still the most centralised large developed country in the world.
"We need a national process of fundamental reform – not just for the North but for the whole country.
"That will help unleash the Northern Powerhouse. But it will also help address the disillusionment and division that have had such a corrosive effect on our bonds as a society and as a country."