Healthcare Environmental Services has been found to be in breach of permits at four of its six sites in England which deal with clinical waste and a criminal investigation has been launched, the Environment Agency said.
At one site in Normanton, West Yorkshire, excess waste levels reached 350 tonnes in September, the HSJ reported.
This is five times more than the company's 70 tonne limit, and a small proportion of it is believed to have been human body parts.
Up to 50 trusts have contracts with the company, the HSJ said, but they are not believed to be experiencing disruption to waste collection.
The Scottish Environment Protection Agency (SEPA) also confirmed that it issued enforcement notices at sites in Dundee and Shotts last month, where its officers are conducting "ongoing monitoring".
"The Environment Agency has found Healthcare Environmental Services to be in breach of its environmental permits at four sites which deal with clinical waste," a spokesman said on Friday.
"We are taking enforcement action against the operator, which includes clearance of the excess waste, and have launched a criminal investigation.
"We are supporting the Government and the NHS to ensure there is no disruption to public services and for alternative plans to be put in place for hospitals affected to dispose of their waste safely."
He said the breaches relate to the company having more waste on its sites than its permits allow and storing waste inappropriately.
As part of its enforcement activity it has partially suspended the company's permit at one of its sites, which will prevent it from accepting any more incinerator-only waste.
The Health Service Journal (HSJ) reported that amputated limbs and pharmaceutical waste were among the matter which had not been properly disposed of and said a Cobra meeting was chaired by Health Secretary Matt Hancock last month over the issue.
The Department of Health and Social Care (DHSC) said there is "absolutely no risk" to public health.
Healthcare Environmental Services said the UK had experienced "reduced incineration capacity" over the last year, which it had repeatedly highlighted to authorities.
A SEPA spokesman said: "SEPA is aware of developments in England and is working closely with the Environment Agency.
"SEPA is clear that compliance is non-negotiable and has been robustly monitoring sites in Scotland.
"On 12th September SEPA issued Enforcement Notices to Healthcare Environmental Services regarding sites in Dundee and Shotts, with officers conducting ongoing monitoring."
A Government spokesman said: "We are monitoring the situation closely and have made sure that public services - including NHS Trusts - have contingency plans in place. There is absolutely no risk to the health of patients or the wider public.
"Our priority is to prevent disruption to the NHS and other vital public services and work is under way to ensure organisations can continue to dispose of their waste safely and efficiently."
A spokesman for Healthcare Environmental Services said: "Healthcare Environmental has highlighted the reduction in the UK's high-temperature incineration capacity for the last few years.
"This is down to the ageing infrastructure, prolonged breakdowns and the reliance on zero waste to landfill policies, taking up the limited high-temperature incineration capacity in the market.
"Over the last year, this reduced incineration capacity has been evident across all of the industry and has affected all companies."
It added it had "consistently highlighted" the issue to environmental regulators, and there has been no disruption to services to customers.
Dr Kathy McLean, chief operating officer and executive medical director of NHS Improvement, said: "The NHS has contingency plans in place for clinical waste and patients should be assured that their care will be unaffected."
Jonathan Ashworth, Labour's shadow health and social care secretary, said: "These are staggering revelations and given the number of NHS Trusts involved, along with wider environmental health implications, I'm disappointed the Health Secretary didn't inform Parliament last month.
"We need a statement in the Commons next week from ministers detailing when the Government was first informed of this stockpiling, what support is now available to Trusts and what contingency plans are in place for the future."
A spokesman for the Scottish Government said: "Clinical waste disposal services are provided to NHS Scotland by Healthcare Environment Services Ltd.
"There have been no reports of any impact on these services from NHS Boards. In the event of any disruption to service at NHS sites across Scotland, NHS Scotland already has contingency plans ready to put into place to ensure there is no impact on services to patients or staff.
"These will be in line with the standards enforced by SEPA."