Amber Rudd said it was "completely unacceptable" for the likes of WhatsApp to offer end-to-end encryption which means security services cannot eavesdrop on messages sent through it.
The call comes amid reports that Westminster terrorist Khalid Mahmood used the messaging service seconds before launching Wednesday's attack, but agencies are unable to see what was communicated.
Ms Rudd also insisted the likes of Google, which runs the social video sharing platform YouTube, and other smaller sites such as WordPress must realise that they are now publishing - rather than technology - companies and take more responsibility for taking down extreme material.
The Home Secretary left the door open to changing the law if necessary.
But she said she would rather see an industry-wide board doing it independently, as the best people to take action are those who understand the technology and the "necessary hashtags".
On encrypted messaging services, she told BBC One's Andrew Marr Show: "It is completely unacceptable, there should be no place for terrorists to hide.
"We need to make sure that organisations like WhatsApp, and there are plenty of others like that, don't provide a secret place for terrorists to communicate with each other.
"It used to be that people would steam-open envelopes or just listen in on phones when they wanted to find out what people were doing, legally, through warrantry.
"But on this situation we need to make sure that our intelligence services have the ability to get into situations like encrypted WhatsApp."
Ms Rudd said she was calling in a "fairly long list" of relevant organisations for a meeting on the issue this week, including social media platforms.
"What these companies have to realise is that they are now publishing companies, they are not technology companies, they are platforms and we need to make sure that that (hosting extremist material) stops," she said.
"You are right, we will not resile from taking action if we need to do so."
But she went on: "I would rather get a situation where we get all these people around the table agreeing to do it.
"I know it sounds a bit like we're stepping away from legislation but we're not.
"What I'm saying is the best people who understand the technology, who understand the necessary hashtags to stop this stuff even being put up, not just taking it down, but stopping it being put up in the first place, are going to be them."
Commons Home Affairs Committee chairwoman Yvette Cooper said the Government should consider Germany-style laws to fine companies that fail to remove extreme material, although Ms Rudd distanced herself from the idea.
Ms Cooper told Sky News's Sophy Ridge on Sunday: "Frankly, I think it's not going to be enough for the Government simply to have more meetings with them.
"I think we're going to have to have much more pressure on them to act because I know that Amber Rudd wants to have another summit with them and another meeting with them. I'm sure that's very good.
"But David Cameron had lots and lots of meetings with them that kind of went around these issues again and again.
"I think they have to act."