And he immediately began dreaming of producing something similar against Italy.
Herbert saw full-back Winston Reid score his side's equaliser with a bullet header three minutes into stoppage time after Robert Vittek had put Slovakia ahead five minutes after half-time.
The result left the teams in Group F, which also includes Italy and Paraguay, locked on one point each after the first two games.
Herbert said: "I am very, very proud. Can it get any better? I think the nation will be in reasonably good spirits tonight."
Asked if it was their finest hour, he added: "It would have to be. We've never won a point in a World Cup before. We've thrown some really good punches today and I thought it was an extremely well-deserved result."
New Zealand, who lost all three group games on their only other World Cup appearance, in 1982, will face Italy on Sunday in Nelspruit with an outside chance of progressing to the second round.
Herbert said: "We'll keep on dreaming because we are here. We have a chance like anyone else but we still have two heavyweights to go and it will be tough. We are competitive and the brand of football we now play is more conducive to high-level results and performances."
Goalscorer Reid, who was booked for stripping off his shirt following his late goal, said: "It was awesome. First thing I did was look at the linesman to see if I was offside and after that it was great.
"Look at the smiles of the players and staff and that tells you everything."
Slovakia manager Vladimir Weiss, meanwhile, described his team's setback as "a small sporting tragedy."