Mr Milns took 401 votes, narrowly defeating Tory John Copsey on 352 votes. Turnout was just over 15 per cent.
In two other by-elections, Howdenshire and Willerby, UKIP came in second place to the Tories.
In Howdenshire Nigel Wilkinson was elected for the Conservatives on 1020 votes while Clive Waddington (UKIP) took 891 votes. In Willerby Conservative Mick Burchill scored a comfortable victory with 1522 votes against UKIP’s Robert Skinner who took 699 votes.
UKIP MEP for Yorkshire and North Lincolnshire Mike Hookem said UKIP’s success showed the disconnect between the three main political parties and voters. He said: “What we have seen in Howdenshire and Willerby is that we have pushed Labour into a poor third; the Lib Dems never had a showing.
“You have to remember Howdenshire with David Davis (MP) is the second best seat for the Tories in the country.
“In Willerby the Conservatives threw everything at it - we are making these parties who always thought these seats were safe really work at it.”