The former Doncaster Rovers, Sheffield United and Southampton striker, who only joined the Elland Road club on Wednesday, pounced three minutes from time with a trademark poacher’s goal.
Middlesbrough goalkeeper Tomas Meijas spilled Michael Tonge’s 20-yard strike and Sharp was on hand to slide in and steer home the rebound.
The scoreline may have a been a little harsh on Middlesbrough - who had a first-half goal wrongly ruled out for a high foot - but it was a game that was there for the taking and typical of Sharp that he would be the man to decide it.
The 28-year-old’s dramatic intervention gave Dave Hockaday the first win of his reign as head coach and also lifted some of the negativity that had hung over Elland Road since his inauspicious appointment in the summer.
Leeds had begun with a 2-0 reverse on the opening day at Millwall that did nothing to dispel those fears.
But a United side boasting fresh recruits in 28-year-old Sharp and defender Liam Cooper at least looked more resolute and adventurous against a Boro side who had made a bright start to the season.
Sharp was lively in the first half, his movement setting up an early chance for Nicky Ajose that was cleared off the line by Daniel Ayala, while Cooper - a midweek recruit from Chesterfield - looked accomplished bringing the ball out of defence and solid in the air.
Boro had the better of the possession and also legitimate claims for a first-half goal.
The Leeds defence allowed a George Friend throw to bounce in the box before Albert Adomah swivelled and swung a flying boot at the ball that sent it into the bottom corner, only for referee Stuart Attwell to blow for a high foot against the Middlesbrough player.
Lee Tomlin was central to everything good about Boro’s play when allowed time to dictate, but generally they did not supply striker Kike with enough chances to add to the two goals he has scored since joining in the summer.
Sharp could have pinched victory for Leeds with 13 minutes remaining when he headed Luke Murphy’s free-kick wide from eight yards.
But no matter, for he was there when it counted on 87 minutes, reacting quickest to Meijas’s parry to guide the ball home and give Elland Road a much-needed lift.
“It was a tough game, there were quite a few chances and it took a goal poacher to sneak the win,” Leeds boss Dave Hockaday said.
“Boro are a good team, one of the form teams, and to come away with a clean sheet and three points, I can’t ask for more.”
Hockaday refused to take any personal glory from the victory, though.
Asked if he had proved anyone wrong, he said: “I am here to prove the president (Cellino) right. I have been, I am the head coach at Leeds United - how tough is that? I am honoured and can’t thank him enough.
“I am fortunate to be given this chance and the players worked their socks off today. Nobody can say we didn’t sweat blood for that white shirt.
“I’ve been in football for 40 years and pressure is my middle name.”
Hockaday may have been in the game for four decades, but match-winner Sharp has been a Leeds player for only four days.
“It’s what he does,” Hockaday said of the 28-year-old. “There’s more to his game, he linked the play and it was the complete performance from him.”
The outcome could have been different had Middlesbrough not had a first-half goal disallowed.
There appeared to be little wrong with Albert Adomah’s overhead kick on the back of a mistake by Liam Cooper, but referee Stuart Atwell saw something in the moment when the winger went for the ball close to Stephen Warnock’s head.
Adomah ran away to celebrate and the goal appeared to have stood until eventually it was ruled out.
“I can’t understand why the goal is not on the scoreboard,” manager Aitor Karanka said. “I think it was a good height (Admoah’s foot) if you see the replay.
“The referee was the person who saw the foot, but the replay does not show this decision in the first second he was waiting four, five, six seconds as we were running to the middle of the pitch.
“I didn’t see the incident, but if you have a foul you have it on one second not six.”
Karanka, whose side earned plaudits in the first week of the season with convincing wins over Birmingham and Oldham, felt his players paid the price for getting dragged into a war by Leeds.
“We played in their style,” he added. “We didn’t keep the ball. We tried to do it by putting Adam Clayton on the pitch but we didn’t get to play in our style.”
Leeds: Silvestri, Byram, Pearce, Cooper, Warnock; Austin (Cook 86), Bianchi, Ajose (Poleon 75), Murphy; Doukara (Tonge 53), Sharp. Unused substitutes: Taylor, Smith, Wootton, Taylor, Cook.
Middlesbrough: Mejias, Abella, Friend, Ayala, Omeruo; Whitehead (Clayton 60), Leadbitter; Adomah, Tomlin (Williams 85), Reach (Nsue 75); Kike. Unused substitutes: Konstantopoulos, Husband, Fewster, Woodgate.
Referee: S Attwell.