In the midst of growing controversy over his views Andrew Sabisky said he was standing down because he did not want to be a distraction to the Government.
He tweeted: "The media hysteria about my old stuff online is mad but I wanted to help HMG not be a distraction.
"Accordingly I've decided to resign as a contractor.
"I hope no.10 hires more ppl w/ good geopolitical forecasting track records & that media learn to stop selective quoting."
In another tweet he said: "I know this will disappoint a lot of ppl but I signed up to do real work, not be in the middle of a giant character assassination: if I can't do the work properly there's no point, & I have a lot of other things to do w/ my life."
It came after Boris Johnson was put under mounting pressure to sack Mr Sabisky, who was drafted in to Number 10 after the Prime Minister's chief adviser, Dominic Cummings, called for "misfits and weirdos" to apply to advise the Government.
Labour said Number 10's refusal to condemn the remarks was "disgusting", while Scottish First Minister Nicola Sturgeon said the Government must "demonstrate some basic but fundamental values".
Mr Sabisky also reportedly once suggested that the benefits of a purported cognitive enhancer, which can prove fatal, are "probably worth a dead kid once a year".
Writing on Mr Cummings' website in 2014, he said: "One way to get around the problems of unplanned pregnancies creating a permanent underclass would be to legally enforce universal uptake of long-term contraception at the onset of puberty.
"Vaccination laws give it a precedent, I would argue."
He also suggested that black Americans have a lower average IQ than white Americans.
Earlier on Monday, a Number 10 spokesman said: "I'm not going to be commenting on individual appointments."
The spokesman added: "The Prime Minister's views on a range of subjects are well publicised and documented."
Number 10 insiders insisted that Mr Johnson did not support eugenics, but the Prime Minister has courted controversy with his views on IQ in the past.
In a speech in 2013 he said any discussion about equality had to take account of the fact that 16% of "our species" had an IQ below 85 while around 2% had an IQ above 130, adding: "The harder you shake the pack, the easier it will be for some cornflakes to get to the top."
In 2000, while Mr Johnson was editor of the Spectator, the magazine carried an article from columnist Taki which said: "On average, Orientals are slower to mature, less randy, less fertile, and have larger brains and higher IQ scores. Blacks are at the other pole, and whites fall somewhere in the middle, although closer to the Orientals than the blacks."
Labour Party chairman Ian Lavery said: "It is disgusting that not only has Number 10 failed to condemn Andrew Sabisky's appalling comments, but also seems to have endorsed the idea that white people are more intelligent than black people.
"Boris Johnson should have the backbone to make a statement in his own words on why he has made this appointment, whether he stands by it, and his own views on the subject of eugenics."
Ms Sturgeon said: "These are really not acceptable headlines for any government to be generating (or allowing to be generated).
"They need to get a grip fast and demonstrate some basic but fundamental values in the terms of our public debate."
Tory MP Caroline Nokes, chairwoman of the Commons Women and Equalities Committee, said: "Cannot believe No 10 has refused to comment on Andrew Sabisky.
"I don't know him from a bar of soap, but don't think we'd get on ....... must be no place in Government for the views he's expressed."
Labour leadership hopeful Rebecca Long-Bailey said: "We've got a Prime Minister who once published an article saying black people 'are at the other pole' in terms of IQ.
"It's no accident that Johnson has hired someone who supports eugenics."
Acting Liberal Democrat leader Sir Ed Davey said: "He (the Prime Minister) must also make crystal clear why he signed off on this appointment and curb Dominic Cummings' power to appoint and sack at will."
Geneticist and broadcaster Dr Adam Rutherford also criticised the comments, writing on Twitter: "Like Cummings, he appears to be bewitched by science, without having made the effort to understand the areas he is invoking, nor its history."