Percy Stewart had enthusiasms and he wanted his neighbours to share in them. From far-flung corners of the world he shipped back home to Pocklington a musical instrument from Tibet fashioned from two halves of a human skull, a giant eel from New Zealand, part of a very large tree he came across in Seattle and other exotic curios. He was fascinated by other cultures, although he was not always in tune with them. After listening to a performance by some up-country musicians in the Far East, he unwittingly proffered a tip so huge in local money that the players thought it must be for the purchase of all their instruments as well and handed them over. If people fascinated Percy, wild animals excited him. Together with the stream of artefacts flowing back to East Yorkshire came another of big game carcasses – the trophies from the hunts which he energetically pursued for 20 years in Africa and elsewhere.

The mounted heads of the blameless antelope Percy bagged, the stuffed body of a lionness preparing to pounce, the eel, the musical instruments and much else besides are now on show in a newly-refurbished museum in the grounds of Burnby Hall, Percy's old home. Parts of the collection are rated by UNESCO as of international importance.

Spending almost half a million pounds – the majority of it from the Heritage Lottery Fund – on showcasing Percy's career as an amateur ethnographer and big game hunter, prompted some Pocklington residents to find out more about this sometimes eccentric figure who continues to make a local impact. When he died in 1962, Percy left what he had to the community in a gesture which is still much appreciated. "We decided that if we wanted to get the stories of the people who remembered him, we had better capture them now," says Jim Ainscough, a retired history teacher who undertook the task with other members of University of the Third Age in Pocklington. "Then we wanted to put those oral recollections into context." The outcome of this project is a fascinating booklet whose joint authors are Jim and his wife Margaret, the proceeds of which will go towards the upkeep of Burnby Hall gardens.

Percy Marlborough Stewart was born with connections – he had the Duke of Marlborough as godfather and Winston Churchill as a second cousin. Percy also had brains – he got a First in Semitic Languages at Cambridge.

What this descendant of the Earls of Galloway did not have, however, was any money. He was one of eight brothers and sisters who had to make his own way which was why he ended up at Pocklington. The headmaster of the town's public school had picked up on Percy's academic abilities and recruited him as a teacher.

At Pocklington School, Percy had the opportunity to try a new pursuit, hunting, where his eye was caught by a young woman who offered advice about jumping tricky obstacles. She was Katherine Bridges, the 26-year old widow of a curate. Katherine was also an heiress. Her landed family in County Durham had made a mint out of the coal mines on their estates.

Percy's only inheritance was the adventurous streak which ran in his family – one of his brothers was tutor to the Tsar's nephews at the time of the Russian revolution, a sister was missionary in China who was in a group massacred at Huashan. Marriage to Katherine in 1901 gave Percy the opportunity to undertake his own adventures on a global stage.

Her money allowed him to buy whatever he liked. First he indulged a craze for horses which ended abruptly with the advent of motor vehicles. Somewhat like Toad of Toad Hall, Percy bought a motor bike, crashed it and moved on to motor cars. He chose a Clarkson Chelmsford, steam-driven and with with solid tyres, which cost £700 and tried it on Yorkshire's most testing stretch of road, the hairpin bends and alarming gradients of Sutton Bank. He lost control and the Clarkson was wrecked, putting his wife in hospital for three weeks. Percy escaped with a sprain.

It seems Katherine did not need much persuading to depart Pocklington. The story goes that the couple went to a dinner party where they were bored rigid. As they left, Percy recalled saying to his wife: "We're terribly dull people, let's travel around the world and then we shall have something to talk about." He added that this should be "not too hurried or performed in a niggardly manner".

So, equipped with pith helmet and rifle, and attended by native bearers carrying home comforts like a beautifully-fashioned – and extremely heavy – toilet case, Percy was at leisure to hack into any jungle of his choosing and shoot anything that moved, no expense spared.

It gave him the material for three books about his hunting, shooting and fishing in places that were out of reach for ordinary people. It was a gilded age for such stately Edwardian travellers and it ended with the First World War. Percy, despite his proven ability for killing things on foreign fields, was turned down as too old for active service abroad. Despite a modest service record, he insisted, when the war was over, that people continue to use his military title of Major Stewart.

Afterwards things were never the same. The General Strike and continuing strife in the coalfields, dried up the stream of money to Katherine that had floated their foreign expeditions. In 1927, the couple put all they had in Pocklington up for auction with a view to flitting to Florida. But it did not meet the reserve price so they stayed put. The grounds of their home seem to have become more the focus of their interest, parts of which were laid out like some of the exotic landscapes they had come across and with an artificial lake which became home to an internationally-renowned collection of water lilies. This is now one of the national collections with 80 varieties and the young garden manager Ian Murphy is about to introduce 10 more.

The lake is fed by water from a well and the lilies are planted into clay in brick beds built on the concrete bottom. "The lake was very well-engineered and is still in good condition," says Ian. "When Major Stewart wanted something doing, he always brought in the best people."

The lilies have a first flush in June then are at their best in July and August. They are perennials and need looking after rather like irises, according to Ian who comes from Bradford. In his care the grounds have become a star turn, quite scintillating. Shoals of huge carp approach visitors who stand at the margins of the lake holding food with a brashness suggesting they think they are ducks. And one of the incidental pleasures is watching the moorhens sprint across the lake via the lily pads without getting their feet wet.

One of the moorhens has built her nest – a considerable construction – on top of the fountain used to aerate the water. Each day from 11pm to 8am, the fountain is switched on and the nest is sluiced away. Every time, in a triumph of hope over experience, the moorhen returns to re-build on the same spot.

After Katherine died, Major Percy Stewart finished in quite modest circumstances, living in a flat in the hall.

She was a Quaker and probably played a big part in the decision to leave everything in trust for the people of Pocklington to enjoy.

The research by Jim Ainscough and his team does not seem to have recorded much in the way of local recollections which reveal great warmth for Percy. Someone remembers him as "civilly dismissive" which sounds a bit chilly. But Jim points out that Percy was a man of his time, with elements of both the paternalist and the autocrat, and that this was only to be expected.

One of Ian Murphy's predecessors recalled his old boss's insistence on his workers doffing their caps to him. That sort of approach does not play well today, nor does a gung-ho attitude to laying waste the world's wildlife – and pictures of Percy's youthful encounters with elephant and tiger, drawn by a famous illustrator of the time, are to be exhibited in Beverley in October.

It seems to be more for what he did – the way he looked after people – rather than what he was, that Percy is still remembered around here with affection.

Jim and Margaret Ainscough's booklet, £4.50, is available at Burnby Hall Gardens and Museum Trust or from Simply Books in Pocklington.