A study by the RAC said tougher penalties introduced two years ago had failed to dissuade drivers from using handheld phones, with the latest figures showing the habit is "rocketing among some groups".
Nearly half (47 per cent) of drivers aged 25-34 admit making or receiving calls with a handheld phone while behind the wheel, up seven percentage points year-on-year, a survey indicated.
Some 29 per cent of motorists aged 35-44 say they use a phone to send texts, post on social media or check emails while driving, a rise of 10 percentage points on the previous year.
Since March 2017, motorists caught using a handheld phone have faced incurring six points on their licence and a £200 fine - up from the previous penalty of three points and £100.
RAC road safety spokesman Pete Williams said they feared any benefits of the new penalties had "run their course with this data showing illegal use is now rocketing among some groups of drivers."
He added: "Following the introduction of stronger penalties in 2017, we saw a promising shift with some drivers changing their behaviour for the better and becoming compliant with the law.
"Sadly, that didn't signal the start of a longer-term trend with drivers now seemingly returning to their old ways and putting themselves and millions of other road users at risk.
"There is still a huge job to do in communicating to drivers the dangers of continuing to mix driving with illegally using a mobile phone."
More than 1,800 drivers were surveyed for the annual RAC Report on Motoring.
Brake said the survey's results did not come as a surprise "with savage police cuts resulting in far fewer officers on our roads enforcing the law."
A spokesman said the Government needed to make roads policing an investment priority "so there is an active deterrent to illegal behaviour and drivers who break the law know that they will be caught and punished.”
Forty-three people were killed and 135 were seriously injured in crashes on Britain's roads in 2017 in which a driver using a mobile was a contributory factor, Department for Transport figures show.
Inspector Frazer Davey, of Avon and Somerset Police, said: "The results of the survey are concerning. We know that driver distraction is a cause of collisions.
"As a road policing inspector I see the impact of driver distraction and people are losing their lives as a result of the use of mobile devices by drivers.
"The law is clear on the use of mobile devices in vehicles and police officers across the country will continue to prosecute drivers who choose to drive while distracted on their phones."