A new report calls for the scrapping of old diesel trains, a proper regional rail network linking major cities and a long term commitment to electrification.
The Campaign for Better Transport also recommends a radical overhaul of fares with the introduction of a London Tube-style zonal system and smartcards that can be used across the North.
Train companies are currently competing to run the northern and transpennine rail franchises - which cover most local services in Yorkshire - from 2016.
The Campaign argues that investment is needed now to ensure that there are decent local services in place to connect with the proposed HS2 and HS3 high speed rail lines due to arrive in the 2020s and 2030s.
Chief executive Stephen Joseph said: “The north’s rail network has been getting a raw deal for decades. This new research shows that major investment is not only long overdue, but is essential if we’re to tackle the north south divide.
“All eyes are on the Government. They must use the new franchises for Northern Rail and TransPennine Express to get rid of outdated rolling stock, improve stations and make sure their are enough seat to cope with demand.”
The CBT report argues the suggestion northern rail services are heavily subsidised is based on “shaky economics” which ignore the region’s geography and the claim that fares are low ignores the difference in wages between the North and South.
Suggestions rail fares might have to rise to pay for the phasing out of old Pacer diesel trains is inconsistent with the approach taken to replacing outdated trains in the South, it says.
James Lewis, chairman of the West Yorkshire Combined Authority’s transport committee, said: “Rail has been really popular in the North of England without any serious improvement in services or quality and we strongly believe there is a latent demand there if we had proper capacity, particularly to meet peak time demand.
“The central question on the franchises is about getting the rolling stock sorted out. The problem with Pacers is not that they are old but they have limited capacity, are slow and are of poor quality.
“We have had some warm words but we need to see them replaced so we can get extra capacity on the network and get more passengers on.”
The Government has said train operators will be “encouraged” to scrap Pacer trains as part of the new northern rail franchise.
Department for Transport spokesman said: “These franchises will see the phasing out of the outdated Pacer trains, additional services across the region, faster journeys on some of the busiest routes and station upgrades.
“They will put passengers at the heart of the rail network and help close the economic gap between north and south.”