The former Cabinet minister said there was a "distinct possibility" that the UK could remain in the bloc even though the process that will lead to Brexit in March 2019 is already under way.
Sir Vince acknowledged that there could be an "amicable divorce" with Brussels which would be acceptable to the public - but there was also the potential for a "terrible mess".
He told Sky News's Sunday with Niall Paterson the "sheer complexity, the practical difficulties, the fact the Government is internally divided" could all impact on the process.
"We may get to the middle of next year and find this is just a horrible mess and there will be a growing political mood in the country and in Parliament to find a way out," he said.
"That's why we think at the end of the day the public should have a choice as to whether they want to go ahead with Brexit when we've discovered what it's about or whether they want an exit from Brexit."
He added: "We absolutely do not know what Brexit will look like - eventually we could get an amicable divorce.
"It looks increasingly unlikely but it's possible, in which case I suspect the public will settle for that.
"But if it is a terrible mess and very divisive and very costly then I think people will want to reopen the question."
Pressed on whether the Lib Dems should be performing better under his leadership, he said: "I'm confident that in time we will make an impact, initially probably in local government and then at a national level at the next election."