Mr Justice Cobb made “many criticisms” of the investigation by West Yorkshire Police and Wakefield Council.
He outlined concerns in a ruling published on Tuesday after analysing issues in a private hearing at the Family Division of the High Court in Bradford.
The judge said the council and police had apologised and agreed to pay damages.
He said the children would get £5,000 each and their mother £10,000.
“The case has exposed serious and systemic flaws in the decision-making and information-sharing of a joint investigation involving West Yorkshire Police and Wakefield Metropolitan District Council,” said Mr Justice Cobb.
“(My ruling) contains many criticisms of the professionals involved in the investigation.”
Mr Justice Cobb said the woman and her children had claimed their human rights had been breached and had asked for compensation.
He indicated that the investigation started nearly two years ago after the youngsters - a girl then seven and a boy then two - were removed from home and placed into foster care following their parents’ arrest.
The judge said family members could not be identified.
Mr Justice Cobb said the children’s father had been jailed after admitting offences.
But he indicated that the mother had not been charged with any offences.
And he said his concerns centred on the investigation relating to the mother.
The judge said:
• There had been “no strategic leadership”.
• The inquiry had lack direction.
• A detective’s judgment had been clouded.
• Lines of communication between social services staff and police and legal departments had been “poorly defined”.
• There had been a lack of care and accuracy in record keeping and in the management of information
• The investigation had lacked structure and decision making had been incoherent.
• There had been a lack of discipline and rigour in the evaluation of evidence.
• There had been a casual regard, and in some respects total disregard, of “ordinary principles of good professional practice”.
Mr Justice Cobb said the upshot had been that the children had unfairly stayed in care - and not been reunited with their mother - for nearly 10 months.
“These failings were to a material extent aggravated by a lack of discipline of the officer in charge of the investigation,” added the judge.
“He showed poor regard for the position of the mother as an accused person, and of the integrity of the information which he received and carried.
“In his communications with the local authority lawyers and social work team, he displayed a lack of clear boundaries. “
Mr Justice Cobb said the investigation had been conducted in a way which “profoundly and obviously” breached rights to a fair trial and to respect for family life.