Oliver grew up in a horse riding background as his father was an event rider and his mother showed side-saddle riding.
He won his first event when he was 11 at the Horse of the Year Show, where he won the junior newcomers show jumping with his pony, Cool Mule.
He rode the same pony in the European Eventing Championships when he was just 13 and left school at 16 to pursue his career in riding and selling horses professionally.
Oliver won his first gold medal in the team event at the 2007 European Championships and again two years later.
However, in 2010, during the Rolex Kentucky Three Day his horse fell on top of him and he broke his collarbone, shoulder bones, sternum and four ribs. He was grateful for having had the airbag vest on, which prevented any further permanent damage. He then took a break from competing for the next four years.
From 2014 onwards, he competed in various other international equestrian games like the World Equestrian Games and the World Young Horse Championships. Between the years 2017 and 2019 he won two medals; gold and silver at the team European Championships events.