Second half goals from Joe Mattock and Oguchi Onyewu earned the Owls a safe passage into the next round at Spotland.
Wednesday finished the game with 10 men after Mattock was dismissed for a second bookable offence - but the 10 men held on comfortably.
Gray said: “The game was a typical FA Cup tie, great atmosphere at Spotland and at times in that first half we had to play with a gale force wind in our faces, so it was tough going.
“But I said to the players at half-time that we’d have the wind behind us in the second half and we went out and scored a couple of quick goals with that in mind, so I was really pleased we made the most of that.
“But credit to Rochdale, they are a good side and they came back into it.
“Joe was disappointed to be sent off as he felt he hadn’t made contact with their lad. But once we were down to 10 men we went with two banks of four and defended very well, and in the end Rochdale were firing balls into the area which quite suited our centre-halves.”
Mattock nudged the Owls ahead in the 51st minute when he applied the finishing touch to Chris Maguire’s corner. And the tie looked in hand when defender Onyewu doubled the lead six minutes later.
But a stunning strike from Michael Rose brought Rochdale back into the contest and when Mattock was sent off the tide appeared to have turned.
But apart from a free-kick from Rose which Damien Martinez pushed over the bar, the Owls were comfortable in keeping Dale at bay.
Rochdale boss Keith Hill was proud of his players - but was left to rue missed opportunities in the first half. Matt Done was denied by Martinez and Peter Vincenti headed wide from a promising position.
“The players and the supporters leave the competition with their heads held high,” said Hill.
“It was an enormous task for us in the first place and I thought Sheffield Wednesday paid us a lot of respect. Their performance was excellent.
“It was even stevens in the first half with chances for both sides, but after the break they scored twice from set-plays.
“We responded well and got a goal back but when they went down to 10 men it made it really difficult for us.
“They defended superbly against our extra man and they have the ability to counter-attack, having that extra little bit of pace.
“It’s unfortunate because we had chances in the first half - but you have got to convert those chances and we didn’t. Now our main focus is obviously the league.”