The former chancellor said a "northern powerhouse" rail network connecting Liverpool to Hull must be planned for as the Government presses ahead with HS2.
So far, more than 70,000 people have signed a petition demanding more investment in transport outside of London and the South East.
Earlier this year it emerged that more than half of England's annual Â£32.7 billion transport budget is spent in London.
It is estimated that investment in the North, including HS3, could create up to 850,000 jobs and generate Â£97 billion for the economy.
Mr Osborne, chairman of the Northern Powerhouse Partnership, said the body wanted a Government commitment to build links across the North, starting with a line across the Pennines.
Writing in The Financial Times, Mr Osborne said: "The Northern Powerhouse Rail fits with Mrs May's stated objective of building an economy that works for everyone.
"Far be it from me to offer advice to the Prime Minister on how to relaunch her premiership this autumn, but making this big commitment to the North at the Conservative conference in Manchester would not be a bad place to start."
He wrote: "Northern Powerhouse Rail, or HS3, must be included in the next stage of the Government's high-speed network.
"Specifically, ministers should include the planning for the future connections when they set out the design for Phase 2b of HS2 later this year, remodelling four junctions to ensure they are complimentary with the Northern Powerhouse Rail proposals, start the detailed planning work on the line itself and allocate a long-term capital budget."
Mr Osborne said plans for HS3 "will not be cheap", with some estimates for the Pennine construction reaching Â£7 billion.
But he added: "This new railway would transform the northern economy."
Mr Osborne's comments come as 50 business and civic leaders from across the north of England are to hand a letter to the Government on Tuesday demanding an increase in transport spending.
Among the signatories are Drax Power chief executive Andy Koss, Greater Manchester Chamber of Commerce chief executive Clive Memmott and York College principal Alison Birkinshaw.
A section of the letter published in the Yorkshire Post reads: "Connecting our great cities of the North with a world-class, higher-capacity rail network is not only fundamental to the success of the Northern Powerhouse, it is fundamental to the success of the entire country.
"We are calling on you to back this success and back NPR."
An independent study by think tank the Institute of Public Policy Research (IPPR) North in February found that Â£1,943 is spent per head in London on current or planned transport infrastructure projects, compared to an average of just Â£427 in the North.
The report's author Grace Blakeley said at the time: "It currently takes longer to travel by train from Liverpool to Hull than from London to Paris.
"Building better links between the North's cities will boost the nation's economy, driving up productivity."
Transport Secretary Chris Grayling angered politicians in the North by last month announcing Government support for a Â£30 billion Crossrail 2 scheme for London, days after axing or downgrading rail projects in Wales, the Midlands and the North.
Tens of thousands of commuters have signed up to the IPPR petition calling for the promised Manchester-Leeds trans-Pennine electrification to go ahead.
They are also demanding Â£59 billion investment in northern transport to match money spent in the South.
A major cross-party summit will be held in Leeds on Wednesday to discuss transport investment in the North.
Greater Manchester mayor Andy Burnham, Liverpool city region mayor Steve Rotheram, and the leaders of Leeds, Sheffield, Newcastle and Manchester councils are among the delegates.
The Prime Minister has said she remains "absolutely committed" to delivering the North-South railway HS2, but has been cautious about supporting any HS3 link.
She told the Yorkshire Post in April that "big projects are not the only way in which we are looking to invest in transport in the region".
Phase 1 of HS2 is due to open in December 2026 and will see trains travel at high speed between London and Birmingham before running on from Birmingham on the existing West Coast Main Line.
A second Y-shaped phase will open in two stages.
Phase 2a from the West Midlands to Crewe will launch in 2027 and phase 2b, from Crewe to Manchester and from the West Midlands to Leeds, South Yorkshire and the East Midlands, will open in 2033.
Mr Koss, who heads the biomass and coal-powered station near Selby, North Yorkshire, said: "At Drax we use compressed wood pellets known as biomass to generate 17% of the UK's renewable electricity - enough for four million households.
"It can take up to 10 hours for the freight trains bringing the biomass from the port in Liverpool to the power station in North Yorkshire - a journey of around 90 miles. And up to three hours from the Humber ports on the east coast - 60 miles away.
"If we invested in new or upgraded rail routes across the North, either through public or industry-led projects, we could increase capacity on the rail network and bring down journey times significantly.
"For example, we estimate that the journey from Liverpool would be four to seven hours shorter if we could open up a new route across the Pennines.
"If we could run the trains more quickly, this would also help - electric trains can run faster than diesel locomotives when travelling up gradients such as those in the Pennines, and they have the added benefit of improving air quality in towns and cities in the North."
Rail, Maritime and Transport union general secretary Mick Cash accused Mr Osborne of "hypocrisy" over his comments.
He said: "This is a man who was the key player in governments which presided over fragmented, cash-starved and privatised rail across the North and which put profiteering first while passengers were left rammed into clapped-out, lashed-up Pacer trains.
"The real legacy of George Osborne's period in government is axed electrification, modernisation and renewal programmes, and private train companies given a political instruction to axe safety-critical guards from their trains."