But there's no need to worry. They're perfectly fine and, in fact, are quite enjoying themselves.
That's because the mum, dad and their two little ones are busy exploring the UK's oldest surviving hedge maze in the grounds of Hampton Court Palace - and the place in which we're staying backs on to the historic site that King Henry VIII once called his home.
The three-star, Grade II-listed Kings Arms Hotel has just reopened following a smart but sensitive refurbishment, having been bought for £2.5 million by Nigerian oil billionaire Theophilus Danjuma. Occupying a covetable spot next to the palace’s Lion Gate and opposite Bushy Park, where magnificent deer roam wherever they please, it traces its history back to 1709 when it was named The Queens Arms - by 1772 this had changed to honour King George III.
Clearly the regal palace is a big draw for guests, but the surrounding suburb of East Molesey is a nice little corner of Surrey that is well worth spending a day or so in, populated with characterful antique shops, boutiques and cafés.
The cream-painted boutique hotel is pleasingly accessible - people journeying by car can leave their vehicles for free in the visitor area at Bushy Park, while Hampton Court railway station is 10 minutes away on foot.
Stepping inside, the Kings Arms immediately feels comfortable and cosy. The handsome tiled floor and marble tables in the reception and bar draw the eye, while elsewhere plush carpets, stained glass windows and period seating are the order of the day.
All 14 bedrooms have been renovated, and the effort has paid off. Our superior room was tightly-proportioned - unsurprisingly, given the age of the building - but it was well-appointed with a cast-iron feature fireplace and a sumptuous bathroom equipped with a fiercely revivifying drench shower. Painted in a lush green hue, taking inspiration from the maze's foliage outside, the room was completed with blinds decorated with a lovely motif of wild birds.
Downstairs, the hotel's excellent in-house venue The Six Restaurant - named after the number of wives the capricious King Henry wed during his lifetime - has a menu designed by chef Mark Kempson, whose career began in a small Hampshire hotel years before he secured a Michelin star at his own Kitchen W8. Head chef Jack Scoines steers the cookery, championing local produce - in particular, herbs and vegetables picked from the palace's kitchen garden.
Classic dishes have been realised with flair and panache, from the smoked haunch of fallow deer - sincere apologies to the animal residents of Bushy Park - and a textbook aged Dexter sirloin steak. Starters are imaginative - see the beetroot cured Loch Duart salmon with blood orange dressing - and space must be made for dessert, where treacle tart and brandy custard leads the way.
Breakfast – included in the room rate – is bountiful the next day. A table is laid out with a huge spread of pastries, cheeses, charcuterie, breads, cereal and yoghurt to graze on.
The maze is one thing but there is much more besides at Hampton Court next door, not least the chance to wander round Henry VIII's apartments and kitchens, pondering on the nature of a man who wolfed down about 5,000 calories a day.
Equally gargantuan is the palace gardens' Great Vine, the largest and oldest grape vine in the world. Planted more than 250 years ago, it still produces a healthy crop of black dessert grapes today - a true piece of living history.
Kings Arms Hotel, 2 Lion Gate, Hampton Court Road, Molesey, KT8 9DD. Rooms from £135 per night. Call 020 8016 6630 or visit www.kingsarmshamptoncourt.com to book.