Mr Allott, who was elected to the role earlier this year, has faced criticism over remarks he made in an interview with BBC Radio York about the false arrest of Sarah Everard.
He said Ms Everard, who was raped and murdered by serving Met police officer Wayne Couzens, should never have submitted to her arrest and that women should be “streetwise” around their legal rights.
On Thursday morning he faced local government leaders at a meeting of the Police and Crime Panel, where he defended himself, and said he was committed to regaining the public's trust.
But after a unanimous vote of no confidence in his abilities to continue in the role, he announced his resignation on Thursday afternoon.
In a statement, Mr Allott said: “I had hoped I could rebuild trust, to restore confidence. I was pleased that so many victims groups had accepted that I was genuinely sorry and were willing to work with me to help me in the mammoth task I had ahead.
“Following this morning’s meeting of the Police and Crime Panel it seems clear to me that the task will be exceptionally difficult, if it is possible at all.
“It would take a long time and a lot of resources of my office and the many groups who do excellent work supporting victims.
“This is time victims do not have. There are women and girls in York and North Yorkshire today suffering at the hands of men.
“Victims and the groups who support them need to be heard. They cannot be heard if the airwaves are filled with discussion about my future.
“That is why I am doing the honourable thing and resigning as Police, Fire and Crime Commissioner – to restore confidence in the office which I believe will be almost impossible for me to do, and to enable victims’ voices to be heard clearly without the distraction of the continued furore which surrounds me.”