But the former Recorder of Bradford, his wig and gown discarded and his imagination unleashed, has forged a new career by turning out fictionalised accounts of some of his own cases.
The first was so incendiary that he had to use a pseudonym, and the next one, which will appear under his own name, is based loosely on the horror in a Sheffield suburb in 1983, when Arthur Hutchinson murdered three members of the same family, after a wedding party.
It was James Stewart QC, who had been his defence counsel.
Judge Stewart, who will turn 75 next month and is working on a fifth novel, had decided he needed a new outlet for his 46 years of legal expertise, following a stellar career that had seen him appearing for the defence at the Stephen Lawrence murder trial.
A third heart attack, six years ago, had forced his retirement from the bench, but he left with the album of press clippings about his cases that he had compiled, he says, out of vanity.
“I take the view that the best stories are based on the truth,” he said. “In my first book I stuck fairly rigidly to the facts but now I play around with them and go off at various tangents.
“And my books differ from most crime novels in that they tell the whole story, from the murder itself to the conviction and sentence, or acquittal. I always end with a trial and it’s never clear until the end which way it’s going to go. The jury does get it wrong sometimes.
“Most other stories end up with the guy being caught and confessing, which in real life tends not to happen.”
His current novel is based on the real-life case of Dr Samson Perera, a dental biologist at Leeds University, who murdered his adopted daughter in 1985.
“He told everyone that the body parts were from a cadaver he had brought from Sri Lanka and that the girl was still alive,” Mr Stewart said. “He was led from the dock, shouting, ‘She’s alive, she’s alive’. I’ve adapted that. There’s a twist at the end.”
A future story will concern the Bradford doctor who practiced medicine for 30 years without ever having qualified.
“His son-in-law eventually shopped him because he was having an affair with another woman. It went to the General Medical Council and the real doctor came over from Pakistan,” Mr Stewart said.
If the case sounds similar to the plot of last year’s BBC medical drama, Trust Me, it is no stranger than the truth as it was played out in Judge Stewart’s court.
“I was trying more murders in Bradford than anywhere else,” he said. “It had become the murder capital of the north eastern circuit at that time.”
A native of the city, who now lives near Harrogate, he turned to writing, he said, after his second wife, Deborah, produced a novel of her own.
But he already had a profile outside the legal world – his brother-in-law was Richard Whiteley, whose late sister, Helen, had been his first wife.
“I was very close to Richard,” he said. “He was a lovely man, just such fun.”
His debut novel, based on an “honour killing” within an Asian family, took him four years to complete; he now turns them out in three months flat. His next plot, he thinks, may take its cue from the Halifax man who went on TV to appeal, tearfully, for the return of his missing, 23-year-old daughter, but who knew she would never return because he had already bludgeoned her to death during an argument.
Born in the Chellow Dene area of Bradford, Mr Stewart’s father was the world famous surgeon and later Freeman of the city, Henry Hamilton Stewart.
He read law at Leeds University and began his legal career in chambers at Bradford, eventually enjoying a high-profile career that took in the Strangeways Prison riots trial , at which Mr Justice Coulson called him “the star of the show.”
He plans to host a launch event for his collection of novels in October, at his old law chambers in Leeds. His books are available on Amazon.