Transport Secretary Grant Shapps suggested Britons should only buy the essentials in their once-weekly trip to the supermarket.
The official coronavirus guidance issued by the Government does not set out how often people are allowed to leave their home to go shopping, but recommends that it is "as infrequent as possible".
Some police forces have been accused of heavy-handed tactics in enforcing the lockdown restrictions imposed by the Government.
Mr Shapps acknowledged there had been "teething problems" but said people should follow the rules.
"People know the rules that have been set, try and shop just once a week," he told the BBC.
"Just do the essentials, not everything else."
Mr Shapps said there had been "one or two instances" of police being overzealous with enforcement measures but forces were generally being "sensible".
He said: "I think the police are doing a difficult job.
"There will be one or two instances where they have perhaps not approached it in the right way but in general, actually, across the country not only are people complying very well but, generally speaking, the police are taking a very sensible approach to it."
Mr Shapps also said people should not be getting into cars to drive to the countryside to take their daily exercise.
"The simple thing is, if at all possible, please take exercise close to your home," he told the BBC.
"I've got dogs and, rather than put them in the car and drive somewhere with them, it's about stepping out of the house and walking them around the block, or whatever it requires."