Dean Farley, who was on his way to a local gym, ran up to David Cameron outside Leeds Civic Hall, where the Prime Minister had been speaking at an event.
The 28-year-old from Headingley in Leeds jogged across the street towards the Prime Minister, but police stepped in to stop him as Mr Cameron got into a waiting vehicle.
Video footage of the incident showed the Prime Minister apparently brace himself as the man approached at speed, before a bodyguard intervened to grapple with the man.
One witness said Mr Cameron “sidestepped” the man in a “lovely little move”.
Mr Farley told the BBC that he only realised it was David Cameron he had run into when he was let out of the police van after an hour.
He said: “I was asking the whole way ‘what have I done?’. An officer turned round and said ‘you know what you have done, be quiet’.
“I will be able to look back eventually and see the funny side, but at the moment it is just harrowing.”
West Yorkshire Police said there was “nothing sinister” about the incident and it was “just a man in the wrong place at the wrong time”.
Mr Farley told BBC Radio 5 Live he was on his way to the gym to meet his personal trainer as he does nearly every lunchtime.
He said: “I ran across the road and, for all I saw, a bunch of men in suits came out of the side of the Civic Hall.
“I dodged in and out and around and the next thing I know I’ve got a half dozen suited men haranguing me and manhandling me to the floor.
“The whole while I’m asking them, ‘What’s going on? Who are you?’.
“I then was told I was being detained to prevent a further breach of the peace. And then cuffs were slapped on behind me. This is the first instance I knew that it was the police that had come for me.”
Mr Farley added: “I didn’t see David Cameron. I didn’t know it was David Cameron until they let me out of the police van an hour later and told me what I’d actually done.”
Mr Farley said: “I gathered I’d run into somebody quite important but I couldn’t know it was David Cameron. This was supposed to be a secret meeting. No one knew he was in Leeds. Nobody said ‘stop police’ as I crossed the road. I couldn’t see any uniformed officers as I crossed the road. There was no cordon.
“It begs the question, how good is Cameron’s security if I managed to run between it before they stopped me?”
Mr Farley said he had no identification on him because he was going to the gym and his towel was wrapped around his hand, which the officers might have thought was a weapon.
Mr Farley denied the collision was some sort of protest, adding that he was “not a particularly political-minded person”.
“I’ve got earphones in, I’m going on a run. I’ve got my eyes to the floor and I didn’t even see who it was. And nobody told me who it was,” he said.
“I always thought if you asked a police officer why you were being detained they were supposed to tell you.”
Asked whether it was difficult to run into the Prime Minister, he said: “Not at all. No. I got on to the pavement, dodged around one man who I assume was his security and I didn’t know it was Cameron, as I’ve said, but I was directly into Cameron.”
Mr Farley’s mother said at her north Leeds home: “I’ve spoken to Dean. He’s very upset by the whole situation.
“It’s all a misunderstanding. He was just jogging to the gym. That’s all I’ve got to say.”
Chief Inspector Derek Hughes of West Yorkshire Police said: “Around midday today, a 28-year-old local man was briefly arrested after he came close to the Prime Minster’s group, who had just left the Civic Hall in Leeds.
“No threats were made, and after the man’s details were checked he was de-arrested and allowed on his way.”
But Tory MP Mark Pritchard demanded a full investigation into how the Metropolitan Police security team allowed the man to get so close to the Prime Minister.
He called for the Met’s commissioner Sir Bernard Hogan-Howe to launch an immediate internal inquiry.
He said: “The Met Police have some of the best close protection officers in the world. However, this was a clear breach of security and could have been far more sinister in outcome.
“With the terror threat level increased, this cannot be allowed to happen again. The Met Commissioner should call an immediate and swift internal inquiry to discover why this happened and implement the necessary lessons learned as quickly as possible.”
Witness Chris Wilson described the man as “just a typical hippy type”.
He said: “It looked like Cameron sidestepped him. It was a lovely little move. He should be on the stage, really.
“And then he was tackled by the person behind. And then he (Cameron) was in the car and the car was gone.
“They were on him. They were like flies round poo. One minute he was down, then he was up again and moving. It was fast.”
Mr Wilson said the man ran about 10 yards across the road towards the Prime Minister. He said he was white, with a beard and dreadlocks.
The Prime Minister looked round, he said, but did not seemed worried.