The two-month-old baby boy, and an 19-month-old girl, were taken to hospital after the car they were travelling in left the westbound carriageway close to Eggborough, at around 3.30pm on Sunday.
Both were pronounced dead yesterday after the crash in the Honda Jazz car.
Their mother, who is in her 20s and was a passenger, was also taken to hospital where police said she remains in a serious condition.
The father, a man in his 20s, was treated for minor injuries.
Police said the family is from the West Yorkshire area.
Traffic Sergeant Hamish Halloway said: “I’d like to extend my deepest sympathies to the family involved following this tragic incident.
“Any fatal collision is deeply upsetting for the families involved but it’s impossible to imagine the horrific ordeal this family is currently going through.
“I’d like to thank the members of the public who stopped to help, in particular the driver of a white HGV who was not involved in the collision but gave a massive amount of assistance to the family at the scene.”
Ambulance crews, including the air ambulance, firefighters, officers from Humberside and West Yorkshire Police and Highways England also attended the scene, near the junction with the A19 Whitley Bridge, Goole.
Anyone with information is asked to call police on 101, quoting incident number 12170157953.