G4S now faces a criminal investigation after the Government reported the firm - well-known for its botched handling of its Olympics security contract - to the Serious Fraud Office when it refused to take part in an additional investigation to rule out any dishonesty.
In addition, Justice Secretary Chris Grayling admitted to the House of Commons that contract managers in the Ministry of Justice discovered issues with the contract as far back as 2008 - but did nothing to tackle the problem.
A Government-wide review of all contracts held by Serco and G4S, which both saw their shares plunge on London’s leading FTSE 100 index, was also sparked by the shock revelations.
And as Mr Grayling is planning to privatise up to 70% of the Probation Service, unions called on the Justice Secretary to block the two firms from the tendering process.
“We’ve long maintained that these companies are unfit for purpose when it comes to holding important public contracts,” Ian Lawrence, general secretary of the National Association of Probation Officers (Napo) trade union, said.
“The outcome of the initial investigation into G4S and Serco suggests a good deal of malpractice has been discovered.”
An audit by big four accountancy firm PricewaterhouseCoopers, launched in May, revealed that overcharging began at least as far back as the start of the current contracts in 2005 - but could have dated as far back as the previous contracts let in 1999.
Auditors discovered that the firms had charged the Government for tagging offenders who were back in prison, had had their tags removed, had left the country or had never been tagged in the first place but had been returned to court.
Mr Grayling has told the two firms that an independent forensic audit must now be conducted to look at, among other areas, internal email trails between executives to establish what happened. While Serco has agreed to take part, G4S has refused.
The Justice Secretary said: “The billing practices in question were clearly unacceptable and the Government will take all necessary steps to secure a refund for the taxpayer.”
He added: “This is a wholly indefensible and unacceptable state of affairs. The House will share my astonishment that two of the Government’s biggest suppliers would seek to charge in this way.
“The House will also be surprised and disappointed to learn that staff in the Ministry of Justice were aware of a potential problem and yet did not take adequate steps to address it.”
Serco has agreed to withdraw from the current tender process for electronic monitoring, while the Justice Secretary is to instigate moves to exclude G4S as the company is still attempting to bid.
Serco had also been the leading bidder for upcoming prison contracts in Yorkshire, but Mr Grayling will delay the award of this contract until the fresh audit is complete.
An urgent review of contract management across the Ministry of Justice’s major contracts has also been launched and will report by autumn, he said.
The problem has arisen as Mr Grayling rolls out plans that will see large areas of the Probation Service turned over to the private sector or charities, under what he has dubbed a ‘’rehabilitation revolution’’.
The likes of G4S and Serco were set to bid for a wide range of payment-by-results contracts to supervise low to medium-level offenders across England and Wales. The providers will be judged by reoffending rates.
Public and Commercial Services union general secretary Mark Serwotka said: “It beggars belief how many times these private companies are allowed to fail and rip off taxpayers and still be awarded Government contracts.
“It ought to be a national scandal but, instead, this Tory-led Government actually wants to sell off even more of our public services to these vultures.”
G4S, which saw its shares fall nearly 4% after the announcement, is no stranger to controversy after its London 2012 Olympics contract fiasco.
It was left nursing losses of £88 million after it failed to provide all of its 10,400 contracted employees last year, prompting extra military personnel to be called in to fill the gap.
Ashley Almanza, G4S group chief executive, said: “G4S is committed to having close and open relationships with our customers and we strive to work in partnership for the mutual benefit of our organisations.
“We place the highest premium on customer service and integrity and therefore take very seriously the concerns expressed by the Ministry of Justice. We are determined to deal with these issues in a prompt and appropriate manner.”
Serco’s shares were also down by nearly 4% following the statement to the House.
The firm confirmed it will repay any amount agreed to be owed to the Government.
Serco Group chief executive Christopher Hyman said: “Serco is a business led by our values and built on the strength of our reputation for integrity.
“These values lie at the heart of the many thousands of our people who are endeavouring to deliver the highest standard of service to our customers around the world. We are deeply concerned if we fall short of the standards expected of all of us.
Cabinet Office Minister Francis Maude said Bill Crothers, the Government’s chief procurement officer, will lead the review into all Government contracts held with G4S and Serco, which include running immigration centres and the welfare-to-work scheme.
Shadow justice secretary Sadiq Khan said: “Today’s revelations are truly shocking. Given the scale of the allegations, the Government must immediately call in the police and the Serious Fraud Office to investigate both companies as fraud has potentially taken place.
“There can be no cosy relationships with either company if we are to truly get to the bottom of these very serious allegations. If it was anyone else the police would be asked to investigate potential criminality. Why isn’t this happening now?”
Keith Vaz MP, who has scrutinised the role of private contractors as chair of the Commons Home Affairs Select Committee, said: “G4S and Serco have let the country down and therefore all their contracts should be audited immediately.”