But a major diplomatic incident and bloodshed on a foreign field led to the building of two of their most beautiful churches.
In 1870, the stately homes were owned by two closely related families. The Vyners lived at Newby Hall, near Boroughbridge, although Lady Mary was a widow by this point. Her daughter, Henrietta, had married her cousin, Lady Mary's nephew George Robinson, Earl de Grey, who lived at Studley Royal.
The Vyners' youngest son, Frederick, was on a 'Grand Tour' of Europe when he was killed in a tragic incident in Greece.
The 23-year-old had been visiting an ancient battlefield with six other British tourists and an Italian on an organised excursion when they were abducted by Greek bandits, who demanded a ransom equivalent to Â£32,000 for the hostages' lives.
As tourist kidnappings had happened in the country before, there was initially optimism that they would be released, and both the Vyners and de Greys set aside money to pay the ransom. One of the hostages, an English aristocrat, was even sent to Athens as an envoy to collect the fee.
However, the gang also demanded an amnesty from the Greek government over their criminal activities, which was refused. Soldiers were sent to the bandits' hide-out, and the pressure caused the men to shoot the hostages one by one.
The incident became known as the Dilessia Massacre, and caused a diplomatic sensation in Britain. Queen Victoria even made an emotional speech condemning the Greek response, and visited Lord de Grey, Frederick's brother-in-law, to console him.
Lady Mary, Henrietta and the de Greys decided to instead spend the untouched ransom money on a memorial to Frederick, and chose to build two churches on their lands - St Mary's, among the ruins of Fountains Abbey at Studley Royal, and Christ the Consoler at Newby.
Frederick's brothers, Henry and Robert, later inherited Newby Hall in turn.