Gary Don was visiting the home of a man in his 80s to value a handful of modest heirlooms when he spotted the painting on his way out.
Its owner was aware of its high value but does not want to sell it, says Mr Don, who is guarding the man’s name and address.
“When I saw the portrait my heart almost stopped. The owner told me it had been passed down over the years.
“He knew it was worth £1m but I think it’s worth far more than that, maybe £2m or £3m. He doesn’t want to sell it, he just wants to enjoy it.
“He doesn’t need the money. He’s a lovely fellow, with a lovely attitude to life. He said the pleasure it gives him far outweighs any money that he would get from it.
“I think he does have a family but am not sure they are aware of the value of the picture. There might be a squabble over it after he goes.”
Toulouse-Lautrec died aged 36 in 1901. He is best known for his portrayal of life in 19th century Paris.