But the mound they pass on the 13-mile route from Hull to the seaside is in fact an ancient scheduled monument - the site of what's thought to be the Castle of Branceholme dating back more than 800 years.
One of the few features in an otherwise flat landscape near the Bransholme estate in Hull, the motte and bailey castle built by Sir John Sayer, before 1200, is 150m long and stands 5m high.
Nothing of the castle - which was probably timber at first but later rebuilt in stone - remains, but Castle Hill's earthern banks are still impressive.
However while popular with dogwalkers, it also attracts offroad bikes and four-wheel drives, which have carved deep ruts into its ancient flanks.
There has also been problems with illegal digging - or nighthawking - both on the remains of the motte and bailey castle and on a Bronze Age mound (750BC) nearby.
But now help could be coming as the result of an Environment Agency scheme to alleviate flooding, which aims to open up access to the site, including to cyclists and horseriders.
A scheme submitted to Hull and East Riding Councils aims to take pressure off existing drainage after the area almost flooded in 2019.
The plans involve digging a meandering watercourse, to the west of the Sutton Cross drain. In the surrounding area woodland will be planted to form an “aquagreen”, that will slow the flow of surface water during heavy rainfall, but also provide valuable new habitat.
The EA will also be installing interpretation boards and paths in a bid to open up the area - while putting up barriers to try and keep out the motorbikes.
They will also create an alternative route for horseriders over the Castlehill Road bridge over the Holderness Drain.
Matt Chapman, from the Environment Agency, said they hope to get more people to use the area and interested in its fascinating history.
He said: “The more eyes and ears we have about the site the better. We want to encourage access to a really nice green place.
"There's very limited public access, partly through the lack of formalised paths and as part of this we will be establishing a network of permissive paths."
The site has been plagued by motorbikes, and last time he visited a big divot had been taken out of the mound by the sump guard of a 4X4.
He said: “It has got a lot worse in the past year. Anti-social behaviour and illegal activity is having a massive effect on the monument.”
Bransholme derives from the early medieval “Brand’s Water Meadow” and until better drainage in the 18th and early 19th centuries, the castle, on one end of a low ridge, would have been like an island at times.
“It was a silty marshland,” says archaeologist Jennifer Morrison. “The land is known to have periodically flooded from the Humber estuary. It used to flood frequently after storms. Flooding isn’t a new thing.”
In fact, she says, what the Environment Agency is doing could be seen as a continuation of what was started by the medieval monks who built land drains, some of which survive till today.
They are now seeking external funding for a community project, based on the rich local archaeology and want residents to come up with suggestions about what they would like to do
She said: “What we are hoping through the community project and by raising awareness of the monument is that people will take an interest and in a way they might become guardians of the site in future, keeping an eye on it, discouraging people from doing damage to it and having pride that they have something so important nearby.
"Having a prehistoric barrow and a medieval castle on your doorstep is quite unusual."