It came at the Picnic in the Park event at the Ashes playing field in Howden on Sunday.
Pictures taken at the scene showed people running towards the large inflatable, which appeared to have blown into a tree.
Thankfully it appeared no children were on the bouncy castle, when the gust of wind hit.
Yorkshire Ambulance Service said four people had been taken to hospital including the three-year-old, who was taken to Scunthorpe General Hospital and whose injuries were not believed to be serious.
Two other patients had been taken to Hull Royal Infirmary, but the ambulance service was unable to say what their injuries were.
A fourth person had been taken to Doncaster Royal Infirmary.
Commenting on social media, a mother said she had been about to put her child on the bouncy castle but said 'No' as the rain started.
She posted: "Feeling a bit stunned but thankful for a split decision to walk away. Hope everyone ok. Pretty scary."
Toilets were blown over and branches fell from trees as the wind tore through the park.
Another commented: "Never seen wind like it before in England. Both bouncy castles were taken away like plastic bags."
And another said: “The bouncy castle headed straight for us. Thankfully stopped by the tree. The toilet block landed two feet away. We feel very lucky.”
Humberside Police said they had been called to the playing fields at 4.30pm. A man, woman and child had been taken to hospital, but their injuries were not believed to be serious.
Calls were made for a temporary ban on bouncy castles after a three-year-old girl died after an inflatable trampoline “exploded” on a Norfolk beach last year.
Ava-May Littleboy’s death followed that of Summer Grant, seven, who died after a bouncy castle was carried 300m across a park in Essex, in March 2016.
Two fairground workers were later jailed for three years each for manslaughter by gross negligence.