Striker Damian Reeves struck the winning goal for the non-leaguers in the 47th minute as Lee Sinnott's side claimed the club's 17th Football League scalp in the competition.
After seven successive losses in League One this was the result that could have ended Johnson's tenure in charge.
Barnsley had made two changes from the side which lost 4-3 at Coventry in midweek.
George Maris and Marley Watkins came in for Southampton loanee winger Lloyd Isgrove, who was on the bench, and striker Michael Smith, who had not received permission from parent club Swindon to play.
Conor Hourihane also made the bench on his comeback from hamstring trouble.
Altrincham had joint top-scorer and Doncaster Rovers youth product Michael Rankine, 30, on the bench after injury along with fellow striker George Bowerman.
It left Damian Reeves up front alone for the kick-off as Altrincham went 4-1-4-1 to counter Barnsley's 4-1-2-1-2.
It was a quiet opening but Altrincham appealed unsuccessfully for a penalty when Marc Roberts appeared to handle a cross from Crowther.
Adam Davies then did well to tip over a fierce drive from Jake Moult after Lewin Nyatanga had headed out a cross.
Barnsley replied with Mawson narrowly off target with a curling shot from outside the area.
Tim Deasy was finally called into action in the 41st minute, going down to his left, to palm out a shot from Sam Winnall at the foot of the post.
The half, however, ended goalless and with no stoppage time.
Reeves, however, made it 1-0 to Altrincham in the 47th minute when he tapped home from close range after Adam Davies palmed a shot from Crowther, who had robbed Pearson in midfield, into the striker's path.
Barnsley made a triple substitution, including Hourihane for the injured Scowen.
Isgrove hit off target, Winnall shot into the side-netting and the third replacement, Conor Wilkinson, had an attempt blocked by Scott Leather as the Reds failed to equalise.
Hourihane went the closest with a 24-yard curling free-kick which was inches wide of the top upright.
Wilkinson should also have done better when he wide from 12 yards after a cross from Marley Watkins had been headed out as the rain lashed down.
The breakthrough just would not come despite the referee playing five minutes of stoppage time and goalkeeper Davies racing up for a corner.
For the latest from the match check out The Yorkshire Post’s live blog at...