US researchers analysed more than 300,000 ratings of 1,300 products on Amazon and found “a substantial disconnect” between the objective quality of the item and the number of stars awarded.
The findings suggest consumers see a star rating - for example 4.6 stars out of five - and consider this more seriously than is merited.
The study, published in the Journal of Consumer Research, also found that consumers are more likely to give a positive review to more expensive products and brands they know, irrespective of their quality.
Almost half the time, Amazon reviewers and experts at Consumer Reports - the US equivalent to Which? in the UK - disagreed about which item in a random pair was better.
The report said: “For two randomly chosen products, there is only a 57% chance that the product with the higher average user rating is rated higher by Consumer Reports.”
It added: “The combined influence of price and brand image on the average rating is much larger than the effect of objective quality, as measured by Consumer Reports, explaining more than four times as much variance.”
The study found shoppers rely heavily on star ratings when deciding which product to buy, even when an item has a statistically insignificant number of reviews.
It said: “Consumers fail to moderate their reliance on the average user rating when the sample size is insufficient. Averages based on small samples are treated the same as averages based on large samples.”
Bart de Langhe, assistant professor of marketing at the University of Colorado, told the New York Times that consumers take star ratings far more seriously than they merited.
He said: “You should rely much less on reviews than you currently do.”