A witness saw the pair firing an air rifle at the animals near Calverley Bridge on Thursday afternoon.
A stray pellet also hit a narrowboat that was moored nearby while its owner was on board.
It is not clear if the birds were shot in the water or in privately-owned fields nearby.
Police searched the beauty spot but could not find the men.
A statement said:
"At 4.10pm yesterday, we received reports of two men with what was thought to be an air rifle shooting geese in a field next to the canal at Calverley Bridge.
"One caller raised concerns that his boat had been hit by a pellet but did not believe the men were deliberately firing at it.
"Officers attended and searched the area but could not find the men and were unable to confirm if they were shooting on private land with permission.
"Neighbourhood officers covering the area have been informed of the incident and will be continue to monitor the situation.
"Anyone with further information about the incident can contact the Outer West Neighbourhood Policing Team at Pudsey Police Station via 101 quoting log number 1126 of September 7."