The Trade Union Share Owners group (TUSO) will make the call at the firm’s annual meeting next week.
The group, a coalition of trade union funds, said former police chief Keith Hellawell had failed to tackle issues such as extensive use of zero-hours contracts.
There was also criticism of Mr Hellawell following the collapse of Sports Direct’s subsidiary company USC earlier this year, said TUSO.
TUC general secretary Frances O’Grady said: “Shareholders and workers both h
ave an interest in reform at Sports Direct. We all want to see a successful business, but this success needs to be built on strong governance and good employment practices, not zero-hours contracts.
“Mr Hellawell is ultimately responsible for the performance of his board and must be held accountable for its actions. Without root-and-branch changes Sports Direct’s reputation will continue to be dragged through the mud.”
Unite general secretary Len McCluskey said: “There are serious questions about the corporate governance and employment practices of Sports Direct.
“An estimated 3,000 agency workers are on zero-hours contracts at its Shirebrook depot, earning just above the minimum wage and being subjected to working conditions that are more akin to a workhouse than a FTSE 100 company.
“Separately, by Mr Hellawell’s own admission, a further 75% of staff across its UK stores are also on zero-hours contracts, with Sports Direct accounting for a fifth of all such contracts in the retail sector.
“These employment practices combined with weak corporate governance mean a change of chair is needed to lead reform of Sports Direct and avoid lasting reputational damage.”
The annual meeting is on September 9.
Huddersfield-born Mr Hellawell is a former Chief Constable of West Yorkshire Police, who became the New Labour government’s “drugs czar” in 1998.