The man, who was in his 20s, attended the gathering at St Botolph’s on Saturday at 12.15pm.
He grabbed the basket, which was filled with cash and cheques, and made off.
A police spokesman said: “This was clearly an act of cowardice and opportunism as the person looking after the basket was vulnerable through age.
“The wicker basket along with some loose change has since been recovered, although several cheques and a large quantity of notes are still outstanding.
“We are asking for your help to identify the suspect as the church does a lot for the community holding events and the money stolen not only affects the church but also the services it provides to the community.”
The suspect was described as being white, tall, wearing dark top and bottoms and his appearance was said to be clean and tidy.
Anyone with any information about the theft should call police on 101 quoting reference number 13180291995.