Murray had been heavy favourite to clinch a first Melbourne title following the early exit of Novak Djokovic but Zverev produced a volleying masterclass on Rod Laver Arena to seal a 7-5 5-7 6-2 6-4 victory.
Zverev won a staggering 65 points at the net and served and volleyed 119 times. He also broke Murray on eight separate occasions.
The Scot, however, usually so adept in the art of the pass and lob, was woefully out of sorts, succumbing to his earliest defeat in this tournament since 2009.
Asked if he felt flat, Murray said: “I don’t think I was flat. I feel like that’s something I get asked all the time when I lose matches. Were you flat? I don’t think I was today.
“I was getting myself pumped up. Sometimes at the end of the sets I was trying to get a little more energy, show more sort of positive body language.
“And I did that at the end of the match, at the end of the first and second sets. It just wasn’t to be today.”
Murray’s superb form at the end of last year meant he endured a hectic end to 2016 as he made five tournament finals in a row.
He then reached the last hurdle in Doha earlier this month, before losing to Djokovic, but the British number one does not believe he has suffered from fatigue.
“Obviously the off-season for me was fairly short just because I had to take a decent break after the Tour Finals,” Murray said.
“But it was a great way to finish last year. I was full of confidence coming into the beginning of this year. I prepared as best as I could.
“Maybe we’ll have to have a look back and assess some things and see maybe if there’s some stuff I could have done differently, or did my opponent just play a great match? Sometimes that can happen as well.
“I don’t know if the end of last season had any bearing on today. I wouldn’t have thought so.”
Zverev is the first player outside the top 40 to beat Murray at a grand slam since the US Open 10 years ago, and his lowest-ranked conqueror since losing here to Juan Ignacio Chela in 2006.
“I’m obviously down about it. It’s just tennis,” Murray said.
“I had great success for a number of months. Obviously in the biggest events you want to do your best. Obviously that’s not been the case here. It happens.
“I’ve had tough losses in my career in the past. I’ve come back from them. This is a tough one.
“I’m sure I’ll come back okay from it but right now I’m obviously very down because I wanted to go further in this event, and it wasn’t to be.”
Murray had been unlikely to play in Britain’s Davis Cup tie against Canada in two weeks’ time but appeared to suggest his involvement was now a possibility.
“Davis Cup would be next on the schedule but whether I stay here or go home in the meantime, I’m not sure,” Murray said.
Zverev, meanwhile, goes through to meet either Kei Nishikori or Roger Federer, who were playing their fourth round match later on Sunday.
The German has never gone this far before at a grand slam and had lost his last meeting with Murray, in 2015, 6-2 6-2.
“It was definitely the best match of my life, not only because it was a best-of-five set match, it was at a slam,” Zverev said.
“I don’t know. It was just incredible.”
Dan Evans’ fairytale run at the Australian Open came to an end after the British number three was beaten by Jo-Wilfried Tsonga.
Evans knocked out world number seven Marin Cilic and then Bernard Tomic to reach his first grand slam fourth round but Tsonga proved a step too far, the Frenchman sealing a 6-7 (4/7) 6-2 6-4 6-4 victory.
Despite defeat, Evans is set to climb from 51st to 45th in the world rankings and leaves Melbourne knowing he can compete with the elite.
Tsonga, the 12th seed, goes through to face US Open champion Stan Wawrinka for a place in the semi-finals.