Neville Neville, 64, made the confession following his arrest for allegedly sexually assaulting a woman in her car near his home.
The complainant has told a jury at Bolton Crown Court that the defendant forced himself on her after she gave him a lift home following a night out in his home town of Bury, Greater Manchester, in the early hours of March 23.
Neville claims the sexual encounter was consensual and denies the offence.
Giving evidence, Neville described how he told his family about the incident following his arrest.
After telling his wife at the police station, they drove home where Phil and his sister, Tracey, a netball international, were waiting.
Neville’s barrister, Orlando Pownall QC, asked: “What did Philip say?”
Neville replied: “He erupted, basically. He said ‘You have got the most loveliest wife ever’. He said ‘She loved you to death’. I was crying at this stage.
“He (Phil) turned round to me and said ‘I’m not going to ask if you sexually assaulted her, I know you would not do that.’
“I said ‘Philip, I did not sexually assault that woman.’
“He said ‘Stop crying then, pull yourself together, get in there (the kitchen), you have got some explaining to do with Mum’.”
Manchester United coaching staff member Phil Neville and his fellow England international footballing brother, Gary, sat in the public gallery as their father gave evidence in his defence.
Neville voluntarily went to his local police station on the afternoon of March 24 and gave his version of events without asking for a solicitor.
He told the court he initially thought the police wanted to speak to him about a recent incident where he said he nearly knocked down a boy while driving.
At the end of the interview, when he was told about the nature of the complaint, he requested a phone call to his wife, Jill.
He told the jury, his voice breaking: “I’m going to drop the biggest bombshell ever on my family.
“I thought basically I have not sexually assaulted anybody, I don’t need a solicitor. I said I want to speak to my wife because I knew she would be worried.
“I was worried as well that I was going to drop the biggest bombshell of her life on top of her.”
His barrister asked him: “Did it cross your mind to say nothing?”
He said: “Never.”
Mr Pownall asked: “Did it cross your mind to deny it and avoid upset for your family?” Neville replied: “No.” His barrister continued: “How easy was it to tell the truth?” The defendant said: “It was very easy to tell the truth.”
He recalled a conversation with the interviewing detective constable after the tapes had been switched off for his interview.
“He said ‘Look, you have told the same story. Either you have lied or she has lied, it’s as simple as that’.
“I turned round and said ‘I’m ruined, I have ruined my family’s life and my life here’.”