The port's oldest fisherman, and former chairman of Bridlington and Flamborough Fishermen's Society has seen the harbour adapt from the days when cod and haddock was King, to today when shellfishing reigns supreme.
When Mr Rollisson, 91, started out there was little in the way of navigational aids - now as son Rolo, 58, puts it "you can now see the seabed in 3D, every nook and cranny.
"You know where you are putting the gear. Before the technology you put it where you thought (you should)."
His father started on the rowing boats aged six, before his first job at 14 on a 33ft coble called Hilary.
He only packed up his last job, as ferryman for the yacht club, at 81 as he couldn't get up and down the ladders any longer.
Mr Rollisson Snr's father George died from blood poisoning after getting a plaice bone stuck in his finger and Mr Rollisson took a job on the keelboat Osprey out of Grimsby aged 16.
"I jumped at the chance," he recalled. "I was only there a week and the engine burst into flames and two of the crew had a job to put the fire out.
"We had four seasons, end of September to March for cod and haddock with long lines.
"There was crab and lobster fishing with lobster pots until July and August, then we used to go herring fishing from August 10 to the end of October, then back to line-fishing."
In 1953 he went pilchard fishing in South Africa, making enough money to buy the coble Nancy on his return, "fishing everything" - including a hammerhead shark on one memorable occasion.
He got the port's first modern trawler Enchanter built in 1967 with the help of a grant and went on to become an important advocate for local fishermen, representing their interests in Parliament.
On one protest during the Cod Wars they took a truck of "manky" cod and dumped it outside 10 Downing Street.
Eventually the cod and haddock disappeared and the port was forced to adapt, going into shellfishing, which has proved to be its saviour.
He said: "Some people say we did too much trawling, our area got warmer and the cod moved north.
"What sickens me now is that they can't catch bass. If they catch some in their nets they have to throw it back. All they are doing is feeding the seals and they have people on the cliffs watching them."
Mr Rollisson was awarded the British Empire Medal for his work, for his charity work, which raised millions over the decades.
He said: "(I got into fishing) from being young, starting young and more or less listening to what I was told by the people who knew the job and putting it into practice. I have no fear of the sea, although I have been nearly drowned on two occasions."
Proud son Rolo, 58, a semi-retired fisherman, who handcrafts the 5,000 pots for the shellfishing vessel Elsie B, said: "He has fought for the industry all his life - they wouldn't have what they have now if it wasn't for my Dad."