Town made three changes as they went in search of their first League One victory of the season following three successive draws.
Jordan Rhodes was brought in to partner Lee Novak up front and Anton Robinson and Donal McDermott were handed midfield berths.
Danny Ward and Scott Arfield were relegated to the bench and Oscar Gobern did not make the squad.
Rhodes justified his recall within three minutes, netting from close range after being set up by right-back Jack Hunt, following a surging run down the flank.
Colchester hit back immediately, former Barnsley striker Kayode Odejayi holding off Peter Clarke and firing into Ian Bennett’s bottom right-hand corner.
The open encounter continued to swing from end to end but the visitors took a 27th-minute lead when Michail Antonio rose at the far post to head home from Ian Henderson’s cross.
Ward replaced Gary Roberts on the left at the start of the second half and Colchester’s Ben Coker came on for the injured Antonio.
Town were level just before the hour mark, a dipping slightly deflected strike from Rhodes outside the area giving Mark Cousins no chance.
Ian Bennett came to Town’s rescue at the other end with a one-handed save from Wordsworth.
Town then regained the lead as Novak chipped the onrushing Cousins after latching on to a through ball into the area.
Colchester forced late corners but Town stood firm to extend their all-time record unbeaten league run to 29 matches, Antony Kay brought on to operate in front of the defence.
Click on the link at the side of the page to launch our Match Centre service and for the best reports and reaction read Sports Monday, free inside Monday’s Yorkshire Post.