The coach driver was killed in Friday's attack in Angola, while two Togo players were also shot and injured.
Hull's Gabon striker Daniel Cousin and Nigerian midfielder Seyi Olofinjana are among 31 Premier League players in Africa for the tournament.
And Brown told The Sun: "I have two players on duty and I want them home."
Defender Serge Akakpo and reserve goalkeeper Kodjovi Obilale were hit in the sustained attack while several other players required hospital treatment and were later seen with bandages on legs, hands and faces.
Togo captain and Manchester City striker Emmanuel Adebayor, who was on the coach but escapeed unharmed, says the team is considering whether to withdraw from the tournament, although organisers insist the competition, which begins on Sunday when host nation Angola play Mali, will go on.
But Tottenham boss Harry Redknapp, who does not have any players at the tournament, said: "It's frightening. It's worth considering calling the whole thing off. We can't just sit around and wait for the next shooting."
Aston Villa manager Martin O'Neill expressed his shock at the incident but was relieved his midfielder Moustapha Salifou was not among the injured.
"The club have been in contact with him and he has reassured us that he is OK but he is extremely shocked and upset, which he would be in these circumstances."
Portsmouth have four players at the tournament and club spokesman Gary Double said they had sought reassurance from the English Football Association and world governing body Fifa about their players' security.
"Our players' safety is paramount and if that can't be guaranteed the players should be sent home," he said.
Professional Footballers' Association chief executive Gordon Taylor said Premier League clubs should not be able to dictate whether their players were sent home.
He told BBC Radio 5 live: "Bearing in mind what has happened, for English clubs to expect players to come home from any other team apart from Togo would be too far.
"It's a decision to be taken by the clubs but that could be in conflict with the African Confederation and Fifa and that is the dilemma of the day, not just in sport, but in many other issues regarding terrorism.
"Do we try to continue life as normally as possible in order to win on the day?
"We need to know the whole circumstances and have a proper inquiry, but you can understand clubs asking for players to come home after such an incident.
"Remember, the driver was killed and the players were under gunfire for half an hour. It doesn't get any more serious than that.
"Clubs release their players under Fifa regulations but they have every right to expect that the security issues and safety is going to be paramount and if that is not the case, they have a serious reason to withdraw them."