Player-coach Ben Morgan's team found themselves trailing 2-0 by the end of the first period after power play strikes from the hosts' Chris Sykes and Matt Campbell, the situation made worse just before the break when defenceman Tim Smith was ejected from the game.
It was a power play of their own through Adrian Palak that saw the visitors halve the deficit at 34.20, only for Billingham to restore their two-goal advantage just over three minutes through Michael Bowman.
But Liam Charnock stepped up in the 47th minute to make it a one-goal game again - the fourth power play strike of the game - before Cameron Brownley struck at 55.54 to haul the visitors level. That came shortly after the Stars' Luke Brown received a five-plus-game misconduct for spearing.
Only 35 seconds of overtime were required by the Steeldogs to gain the extra point when Milan Kolena struck the decisive blow.
"We didn't really get going until midway through the second period," said Charnock afterwards. "After penalties came in and played a part, we got our feet going, got in the right mindset and went forward from there.
"We believed that we were going to get it back going into the third period."
Hull Pirates enjoyed mixed fortunes in the NIHL Autumn Cup, beating Telford Tigers 4-3 on Saturday before going down 3-2 at Swindon Wildcats the following evening.
It was a night to remember on home ice on Saturday for Jason Hewitt's team, the Pirates coming back from 3-0 down to seal a thrilling victory over a team who have set the early pace in the NIHL regular season standings.
Richard Plant's shorthanded effort put the visitors in control in the 11th minute before two goals in as many minutes - from Joe Miller and Dan Mitchell - had the Pirates reeling.
But a quick reply in the 29th minute from the stick of Matt Davies gave the hosts hope, although it took a while for them to turn the screw on Telford, with a sensational last five minutes or so required to seal the points.
First, player-coach Hewitt made it a one-goal game at 54.21 before the hosts drew level 55 seconds later through Lee Bonner.
The winner, when it came, couldn't have been hardly any better timed - Sam Towner pouncing with just four seconds left on the clock to seal a memorable comeback victory.
Sunday's clash at Swindon opened in explosive fashion, with Hewitt's third-minute strike cancelling out Luc Johnson's opener for the hosts after just 83 seconds.
Lee Haywood then put the Pirates ahead in the 38th minute, but the game was ultimately forced into overtime after Stephen Whitfield levelled for the hosts at 44.43. Former Cardiff Devils forward proved the difference in the add-on, scoring the winner with just 80 seconds remaining.