The artwork, done by Elwell when he was in Paris in 1896, depicts an “Artist’s Appreciation Evening”, men and women enjoying a free-spirited evening, with wine, cigarettes and a naked woman on a pedestal - perhaps a sculpture or even a live model.
It is not known who tore the painting up, when, where or why.
Elwell is a favourite artist with visitors to the town’s Art Gallery, which has the world’s biggest collection of his paintings - more than 100.
Curator Helena Cox said: “The new painting is of a rather small size but is important for our Elwell collection. It is a gouache and there is a certain mystery to it, too!
"It was found at one of Fred’s studios in Beverley, opposite the railway station. Allegedly, it has been kept in two pieces for a long time.
“So, someone must have been so upset with the piece that they tried to destroy it, and then someone else must have loved it so much they’ve done all they could to save it.”
Elwell studied at the Academie Julian in Paris, an independent school which attracted many artists from all over the world, including women who were not admitted to the official École des Beaux-Arts, until 1897.
It was an exciting international crossroads in the centre of Paris - the most artistically progressive city at the time.
The artist, who was born in 1870, often focused on Beverley's less obvious aspects: hotel kitchens, the Police Court, a garage, a munitions factory, a wheelwright’s shop, life "below stairs".
As The Times put it he was "pre-eminently a painter of domesticity".
He died at his home, The Bar House in Beverley, in 1958.
Ms Cox said they were indebted to the Friends of Beverley Art Gallery, who bought the "beautifully painted" black and white gouache at auction, adding: “There is no other Elwell quite like it not only in our collection but also in any other UK collection, public or private, as far as we know.”