"But how do you tell everyone?" said one woman after the tragedy.
Emergency services were called to Seaton Garth in Staithesafter reports that a girl had suffered serious head injuries.
North Yorkshire Police said: "Sadly, despite the efforts of the emergency services, the girl died at the scene from her injuries.
"Her family are currently being supported by specialist trained officers."
Police, fire, ambulance, air ambulance and Coastguard officers all attended the incident at around 4.45pm on Wednesday.
People in the quiet village said they were shocked by what happened as the beach remained cordoned off on Thursday morning.
One man said: "We were just round there with our two yesterday, knocking on the cliffs for fossils.
"It doesn't bear thinking about. What an absolute tragedy."
Others talked about the huge emergency services response, including an air ambulance landing in the middle of the harbour beach.
A woman, who has had a house in the village for more than 30 years, said the beach was "teeming with people" on Wednesday afternoon.
"I was on there with my grandchildren and we left just before this happened," she said.
"It is absolutely terrible what's happened. Fortunately, we had walked the other way."
The woman, who did not wish to be named, said local people knew not to go too close to the cliffs as minor collapses were common.
But she added: "How do you tell everyone who comes? It's just not possible."
She said local people believe the recent weather had made the cliffs more unstable, especially with a long dry spell followed by heavy rain.
One cottage in the village has recently showed dramatic movement, with the home-owner having to leave through a window after hearing a loud crack.
The woman said the emergency services arrived very quickly, especially given the difficult access to the harbour down one narrow, cobbled street.
"They did a terrific job, all of them," she said.
The closed beach was empty on Thursday and nobody was around the area of the rock fall, which local people said was just on the eastern side of the harbour wall.
Another man, who did not want to be named, said: "I've never seen so many emergency services. And the speed they came down that cobbled hill. So sad, though."
Someone had a left a single sunflower by the cordoned-off steps down to the harbour beach - the closest accessible spot to where the accident is thought to have happened.
It is not the first rock fall tragedy to happen along the British coastline.
In June 2015, Georgina Le Fjord died after she was hit on the head by a rock falling from a cliff on a beach at Llantwit Major, between Swansea and Cardiff on the Glamorgan coast.
The 23-year-old, known to friends as Georgie Ford and originally from Salisbury, had been having a picnic with a friend and was said to be sitting about five metres away from the cliff when a rock hit her.
Three years earlier, 22-year-old Charlotte Blackman died after being buried by tonnes of rock following the collapse of a cliff on a beach in Dorset.
She was killed in July 2012 as horrified onlookers - including her father and boyfriend - tried to reach her before another huge section of cliff gave way.
Miss Blackman, of Heanor, Derbyshire, was on holiday with her family and boyfriend when tragedy struck at Hive Beach in Burton Bradstock.
Just two weeks later, five people escaped unhurt after a major rock fall along the Jurassic Coast near Charmouth, Dorset.
The British Geological Survey (BGS) has in the past warned about the dangers of getting close to cliffs, saying the coastline can be prone to occasional rock falls.
They usually occur after sustained heavy rainfall but can be unpredictable, and the BGS has urged the public to use common sense when approaching cliffs due to the fact they can be unstable.