The younger Ayew brother, in outstanding recent form, received his marching orders after an ugly high challenge on Jonathan Hogg on 11 minutes - and could have no argument following a reckless tackle.
To their credit, the disciplined and workaholic Swans dug deep for the remainder of the game with a sea of red styming the hosts, who were not befriended by luck either.
Town hit the woodwork twice in the second half, first through Steve Mounie's powerful volley before Tom Ince's late header also knocked on wood late on.
The sight of Aaron Mooy lining up in Town colours from the off will have been a soothing one for home supporters, even if Danny Williams was somewhat unfortunate to drop down to the bench.
The dismissal of Jordan Ayew will have further added to the sense of well-being in a game of critical importance just past the ten-minute mark, but the rest of the half was an exercise in frustration.
With their game plan pretty much written after the red card, Swansea beat a retreat, with their defensive organisation and discipline being striking for the rest of a one-sided half, in which they protected Lukasz Fabianski's goal superbly.
Town, for their part, gorged upon possession, but lacked the wit to expose Swansea's numerical disadvantage.
There were a few close shaves for the Swans, who made a couple of telling last-ditch blocks, but Fabianski was rarely troubled.
Steve Mounie and Mooy were off target with efforts before Christopher Schindler headed over, with Town's best moment coming before the break when Mounie's flicked back header drifted just over the top following Mooy's centre.
The pattern of Town territorial dominance continued on the restart, but, once again, they lacked the craft and imagination to prise open Swansea.
It took an unlikely source in Hogg to almost inspire a breakthrough with his screamer fluying inches wide before Scott Malone, steaming into the box, just failed to get a telling touch to a well-flighted centre from Florent Habergjonaj.
Town's most convincing spell arrived just after the hour mark as they went desperately close to a breakthrough on two occasions.
First, a blistering volley from Mounie, after Swansea failed to clear their lines, shuddered the woodwork before a neat short corner saw Mooy tee up Habergjonaj, whose fine curling strike was inches off target.
Try as they might, Town could not find a way through against Swansea's sea of red on a frustrating 'if only' afternoon, confirmed at the death when Ince's downward header hit the woodwork.
Huddersfield Town: Lossl; Habergjonaj, Zanka, Schindler, Malone; Hogg, Mooy (Williams 69); Van La Parra (Depoitre 78), Pritchard (Quaner 65), Ince; Mounie. Substitutes unused: Coleman, Smith, Sabiri, Lowe.
Swansea City: Fabianski; Van Der Hoorn, Fernandez, Mawson; Naughton, Ki (Carroll 89), King, Clucas; A Ayew (Abraham 73); J Ayew. Substitutes unused: Nordfeldt, Bartley, Roberts, Narsingh, Dyer.
Referee: M Oliver (Northumberland).
Attendance: 23,567 (1,077 Swansea supporters).