Mr Bullimore became world-famous after he survived for four days in the upturned hull of his boat when it capsized in the freezing waters of the Southern Ocean in 1997.
He had been taking part in the 1996 Vendee Globe single-handed around-the-world race in his boat, the Exide Challenger, and was 2,200km off the coast of Australia.
The then 57-year-old sailor, nicknamed the British bulldog, was feared to have drowned until he was dramatically rescued by the Australian navy.
Mr Bullimore was a founding member of the famous Bamboo Club in Bristol, which he opened with wife Lalel, in 1966.
The nightclub hosted stars including Bob Marley and the Wailers, as well as Ben E King, before it closed 11 years after opening because of a fire.
Bristol’s Lord Mayor Cleo Lake paid tribute to Mr Bullimore on Twitter.
She wrote: “A Bristol legend both on the waters and on the music scene. Everything you did to break down racial barriers. Sleep well Tony Bullimore and thank you”.
The BBC and ITV both reported that he died after suffering from cancer.