Scientists found that taking even small amounts of the drug by the age of 14 could disrupt the development of parts of the brain which control emotion and memory.
Those who took the drug once or twice a day had greater “grey matter” nerve cells in parts of the brain.
The research found that cannabis could interfere with a process of “pruning” in which the brain removes unwanted nerve connections and strengthens those that are most needed.
Lead scientist Professor Hugh Garavan, from the University of Vermont, USA, said: “Consuming just one or two joints seems to change grey matter volumes in these young adolescents.
“One possibility is they’ve actually disrupted that pruning process.
“You’re changing your brain with just one or two joints.”
The study is part of a long-term European project called Imagen that has collected brain-imaging data from 2,000 children and young adults.
Among participants in the research were 46 children from England, France and Germany who had reported smoking cannabis once or twice.