As a result, the Tykes suffered their first defeat of the campaign and also extended their 95-year winless sequence in Coventry to 25 matches.
Conor Chaplin should have given the home side an early breakthrough when he somehow side-footed wide from six yards.
Jacob Brown spurned two clear-cut chances for Barnsley, firstly when he intercepted Junior Brown's backpass only to see goalkeeper Lee Burge touch his low shot on to a post before firing wide when well-placed from Mamadou Thiam's low cross.
Burge tipped Thiam's 20-yard curler over the bar five minutes into the second half, but at the other end, City were denied when Chaplin's fierce finish from a Jonson Clarke-Harris knockdown was disallowed for offside in the build-up.
An end-to-end affair saw scrambles in both boxes but City snatched the winner when Michael Doyle's free-kick was headed back inside by Dominic Hyam and Willis hurled himself forward to bundle in from close range.