The brothers, aged 74 and 82, were attacked by a cyclist while walking along the Aire and Calder Navigation, close to Birkwood Road, Normanton.
Police are appealing for information on the incident, which took place at around 9.30am on Wednesday morning.
And a relative of the victims has now described the moment that the attack took place.
She said the attackers struck after the brothers passed two cyclists on the canal path.
The relative, who asked not to be named, said: "The cyclist told them to move and apparently they moved the wrong way.
"Then he jumped off his bike and struck them both with it. He punched one of them to the ground and then started on the other one.
"The man punched him in the face with such force that when he got to the hospital they thought his jaw was broken.
"He pushed his brother to the floor and then kicked him. He actually kicked him while was down."
The brothers remain in "good spirits" after the attack, despite minor injuries, but family members are now calling for anyone with information about the attack to step forward.
One relative said: "I think they don't want to let it get them down.
"I'm only interested in finding out who this person is. For somebody to pick their bike up and hit old folks, I'm speechless. I'm mind blown that it's happened.
"Walking is the only thing they've got at the moment. They walk every day, everybody knows them because they're always walking together.
"I want to bring justice. We're a very close family and it's just soul-destroying."
A spokesperson for West Yorkshire Police said: "At about 9.30am on Wednesday 7 April 2021, two men were walking together along the canal path near Birkwood Road, Normanton, when two cyclists, a man and a woman, rode towards them.
"It is reported that a short argument happened between the walkers and cyclists during which the male cyclist assaulted the two walkers by punching them and striking them with his bike. The cyclists rode away."
The men sustained minor injuries. Anyone who witnessed the incident or has information that could identify the two cyclists should contact West Yorkshire Police on 101, quoting log number 356 of 07/04/21.