It suggests that budget pressures, and the introduction of new performance measures are among the reasons for the fall.
The study, published by the Education Policy Institute think tank, examined all exams entries, including GCSEs, for art and design, drama, media, film and TV studies, music, dance and performing arts by 14 to 16-year-olds between 2007 and 2016.
It found that 53.5% of pupils entered for an arts subject in 2016, down from 55.9% in 2015 and a high of 57.1% in 2014.
It is the lowest proportion in the last 10 years, the study concluded, with the figure standing at 55.6% in 2007.
The study calculates that if the same proportion of pupils had taken at least one arts subject in 2016 as in 2014, then around 19,000 more youngsters would have studied an arts-based course last year.
It also concludes that the average number of exam entries has fallen since 2013, when it stood at 0.8 entries per student, to 0.7 entries per student in 2016.
Although there was an increase in the entry rate between 2010 and 2013, the report says, the 2016 rate is still the lowest in the last 10 years.
It also found a clear divide between the North and South of England, with higher entry rates in the South.
The North East in particular saw a sharp drop in arts entries last year, with 47.8% of pupils entering for at least one of these subjects, compared to 57.3% in the highest region, the South West.
Girls were more likely to take an arts course, with 64.7% taking at least one, compared to 42.5% of their male classmates.
The study notes that the move to introduce the English Baccalaureate (EBacc) - which recognises pupils who study English, maths, science, history or geography and a foreign language at GCSE - has led to concerns that other subjects, such as arts-based courses, are being put at risk.
“For much of the past decade, entries to arts subjects have demonstrated remarkable resilience in the face of a number of different policy changes,” the study says.
“In 2016, however, arts entries at Key Stage 4 fell to a level slightly below that for any year in the period under review; this is the case both for the proportion of arts entries and the percentage of pupils with at least one entry to an arts qualification.”
It says that this is likely to be due to a combination of factors, adding: “The EBacc does not bar access to the arts for those who take it, but it does limit the number of option subject slots that can be filled by non-EBacc subjects.”
A new key government school performance measure - Progress 8 - which covers the progress pupils make in eight subjects between starting and finishing secondary school may have had an impact last year.
Progress 8, used for the first time in 2016, requires pupils to enter for at least five EBacc subjects, which limits the number of other GCSE options that pupils can choose, such as arts subjects.
But it adds that this measure does also offer some protection to these subjects, as there are three “slots” that can be filled by a variety of courses, including the arts, which means that schools which get high grades in arts subjects have an incentive to continue to offer them.
Becky Johnes, EPI senior researcher, said: “Our analysis shows that entries to arts subjects are currently declining, following several years of slight increases. The 2016 entry rate was the lowest of the decade.
“This recent drop in arts entries is driven by several factors, including changes to the way school performance is measured such as EBacc or Progress 8, financial pressures on schools, and of course local decisions taken by school leaders.”
Responding to the report, Geoff Barton, general secretary of the Association of School and College Leaders, said: “The government must stop burying its head in the sand.
“The evidence is clear that arts subjects are being driven to the fringes of the curriculum by accountability measures which heavily prioritise a narrow range of academic subjects and an education funding crisis which means schools are having to cut courses.”
A Department for Education spokeswoman said: “Our current research contradicts the EPI and has found there has been broad stability between pupils taking GCSEs in EBacc subjects and the arts.
“There is a correlation between schools increasing their EBacc entry and an increase in the uptake of arts subjects.
“The EPI report presents different findings because it takes different subject groupings and qualifications into account in its calculations.”