The hosts were second best in the opening 45 minutes, but Portuguese striker Joao netted in first-half stoppage time.
But the Millers took the lead thanks to goals from Michael Smith and Richie Towell, only for Joao's header to earn a share of the spoils for Wednesday.
Barry Bannan was passed fit to play for the Owls, while the Millers were skippered by former Hillsborough defender Richard Wood.
The Miller spurned a great chance to snatch the lead in the fourth minute, but defender Semi Ajaya prodded wide from close range after a free kick caused panic in the hosts' defence.
It was a poor start from Wednesday, who struggled to get out of their own half, and goalkeeper Cameron Dawson had to be alert to deny Jon Taylor from long range.
The Owls had their best chance in the 28th minute, Adam Reach broke clear but dragged his shot well wide of Marek Rodak's goal.
Joe Mattock headed over Bannan's free-kick, but it was a derby lacking in quality or passion.
Dawson's one-handed save denied Michael Smith's header five minutes before the break, but the Owls grabbed the lead in first-half stoppage time.
Lucas Joao looked second best to Zak Vyner after a hopeful ball from Reach into Rotherham's half, but he showed strength to barge his way through and fire beyond Rodak.
Any hope the goal would settle Wednesday down crumbled 18 seconds into the second half, as Smith raced away from the hosts' defence - maybe still re-organising after Daniel Pudil came on at the break for Jordan Thorniley - and drilled the ball into Dawson's bottom corner.
Rodak saved Atdhe Nuhiu's deflected effort, while Tom Lees poke Bannan's corner wide.
But the Millers stunned Hillsborough with a second goal, and this time it was a huge error from goalkeeper Dawson.
Richie Towell's long-range strike squirmed through the goalkeeper's grasp and into the back of the net at the Leppings Lane end.
Wednesday had to come out and attack, and they grabbed an equaliser when Joao headed in Pudil's cross.
Rotherham had a great chance to retake the lead, but Will Vaulks fired wide after Smith's neat touch played him in on goal.
Nuhiu blazed over from six yards out in the 90th minute.