N Brown said Mr Higginson will remain as chairman during the search for his replacement and through what it called “an orderly handover period”.
Chief executive Angela Spindler said: “I would like to thank Andy for his support and advice since my appointment as CEO almost four years ago.
“Andy has led the group during a period of significant transformation and his dedication and insight have made a significant contribution to this process.”
Profits at N Brown fell 20% to £57.6 million last year as the firm was stung by exceptional costs relating to the mis-selling of PPI.
N Brown took a £25.2 million hit to cover compensation for customers after the deadline for settling claims was set for August 2019.
The retailer offers consumer credit via a business which helps shoppers pay for products.
N Brown has also recently signed a partnership with Tesco to sell its Power, Simply Be and Jacamo brands on the supermarket’s website.