Mr Parsons was with his family when he died on Tuesday morning after a short illness, his agent said.
The veteran broadcaster was best known for Just A Minute, where panellists have to speak for one minute without hesitation, deviation or repetition.
He fronted the Radio 4 show since its inception in 1967 and only missed two recordings of the programme. His last episode aired on September 16 last year and the show is currently between series.
Stephen Fry was among the many stars paying tribute to the broadcaster, also known for his role as question master on TV quiz show Sale Of The Century.
"He ruled Just A Minute for Just a Lifetime," Fry wrote on Twitter. "A stunning achievement - never scripted, always immaculate. From comedian's sidekick to great institution, via Sale Of The Century and much more. Unrivalled continuity, professionalism and commitment. Farewell."
"His continued delight at being a part of show business should be an inspiration to us all," he said.
Writer and broadcaster Gyles Brandreth said it was "the end of an era."
"I thought he was immortal because he was always so alive," he said. "Such sadness today, but what a career to celebrate, from the 1940s to 2020. My hero!"
Radio 4 Today presenter Nick Robinson said "Parsons was a broadcasting giant who proved that the straight man could be the real star of comedy. Will be hugely missed by all who work at and love."
The broadcaster's agent Jean Diamond issued a statement on behalf of his family, saying: "Nicholas passed away in the early hours of the 28th of January after a short illness at the age of 96.
"He was with his beloved family who will miss him enormously and who wish to thank the wonderful staff at the Stoke Mandeville Hospital."
Julia McKenzie, creative director at BBC Studios Audio - which produces Just a Minute, said Parsons lived a "full life".
"Nicholas was such a smart and passionate man who cared deeply about his radio show and Radio 4 and its devoted listeners. He was utterly charming and faultlessly professional," she said.
"He was funny and kind in person and wonderful company. I can't really believe he is gone today, he seemed indestructible.
"His legacy will live on with over 50 years of laughter-filled shows and the lasting friendships he made through his extraordinary career in show business."
BBC Director-General Tony Hall said: "Very few people have done so much to entertain audiences over the decades, and no-one deserves to be called a broadcasting legend more than Nicholas Parsons. His charm, inventive intellect and ability to create laughs were unsurpassed."
Mohit Bakaya, controller of BBC Radio 4, said: "Nicholas Parsons was one of the greats, a first class broadcaster and an icon in the world of British comedy."
Piers Morgan said Parsons was a "wonderful man who brought so much fun, charm, wit and pleasure to so many millions of people over so many decades".
Despite his glittering CV, Parsons once said he believed he would have got more work if he had been "more rugged-looking".
The broadcaster was born in Grantham, Lincolnshire, and found TV fame appearing with comic Arthur Haynes in his ITV show in the early 1960s, and he was also a regular on The Benny Hill Show.
Parsons married his second wife Ann Reynolds in 1995, and had two children from his first marriage to Denise Bryer.