Captain Harry Kane headed in at the far post in stoppage-time to clinch a dramatic victory in Volgograd after his 11th-minute opener had been cancelled out by Ferjani Sassi’s penalty.
Southgate was thrilled with his side’s performance, but knows there is plenty still to do in their remaining Group G games.
Next up for the Three Lions are Panama, who suffered a 3-0 defeat against Belgium earlier yesterday.
Southgate said: “We’ve given ourselves a great platform to build from.
“There’s so much to be pleased about – in particular for me the level of performance – but there’s a lot of work to do still.
“We knew what a tough test Tunisia would be and Panama will be a tough test in a different way because they’ll be disappointed with the way they’ve ended up (against Belgium).”
The 47-year-old believed England deserved their victory and said he would have been proud of the performance even if they had not found a late winner.
He added: “The way we kept playing, even though the clock was running down, we stayed patient, we didn’t just throw the ball in the box.
“We waited for good opportunities and I think we deserved the win. I thought even if we’d drawn the game, which I know would have made life more difficult for us, I’d have been proud of the performance.”
Southgate was keen to stress the victory was a team effort, but reserved special praise for two-goal striker Kane.
He said: “He (Kane) is a top, top striker, isn’t he?
“I’m delighted for him because if he doesn’t score [against Tunisia] we’d be asking questions about him not scoring goals in tournament football.
“It’s great that we’ve got him in the team and he took his goals really well.”
Captain Kane was proud of England’s battling performance, but did not think Tunisia should have been awarded the penalty that led to their equaliser.
He said: “I’m so proud of the lads. I thought we played really well, especially in the first half, and deserved to be ahead.
“We could have scored a few more and then – obviously I’ve not seen the penalty back – but from my point of view it didn’t look like one.
“Credit to the lads, we kept going. That’s what these games are about; it’s a World Cup, you go until the last second and I’m absolutely buzzing.”
Midfielder Jordan Henderson agreed that staying switched on until the final whistle paid off for England.
He said: “We’ve always been saying before the game, no matter what happens in the game we keep going until that whistle blows, we keep going, keep fighting and thankfully it paid off for us with a late goal and a big win in the group stage.”