Neil Warnock`s men, who were thrashed 4-0 at home to Bolton last weekend, came away from Goodison Park with maximum points thanks to Tommy Smith`s 31st-minute goal.
The forward took in Akos Buzsaky`s pass to leave Tim Howard with no chance, while at the other end, his side survived Leighton Baines hitting the bar and penalty appeals from Jack Rodwell and Tim Cahill.
Liverpool recorded their first away win over Arsenal since 2000 to heap more misery on the Gunners - who saw Emmanuel Frimpong sent off and lost injured centre-back Laurent Koscielny in a 2-0 defeat at Emirates Stadium.
In one of the day`s two early kick-offs, Samir Nasri impressed for the home side, despite being expected to join Manchester City, but Arsenal’s luck was out after Aaron Ramsey’s own goal and a last-minute strike from substitute Luis Suarez handed the visitors victory.
Elsewhere early on, Ryan Taylor wrote his name into Geordie folklore by firing Newcastle to a 1-0 derby victory at Sunderland.
The full-back curled home a superb 62nd-minute free-kick to win a hard-fought contest which the home side ended with 10 men after Phil Bardsley`s late red card.
In the other 3pm starts, the first Premier League game to be played in Wales ended goalless as Swansea took on Wigan.
Michael Vorm, who was impressive on his debut in Swansea`s goal at Manchester City on Monday, was the hero for the home side, saving Ben Watson`s 73rd-minute penalty after Ashley Williams had felled Jordi Gomez.
Victor Moses and Ben Watson also hit the woodwork for the visitors, who have drawn their first two games, while the point is Swansea`s first in the top flight.
At Villa Park, Gabriel Agbonlahor scored his 50th Premier League goal for Aston Villa as they beat Blackburn 3-1.
With Alex McLeish taking charge of a home match for the first time, Agbonlahor gave him the perfect start when he drifted in off the left with 12 minutes gone and gave Paul Robinson no chance from 18 yards.
Emile Heskey made it two from the edge of the box with 25 minute gone and, although Morten Gamst Pedersen`s header pulled one back (52), Darren Bent wrapped up the win with 23 minutes left, slotting home after Stiliyan Petrov`s shot was blocked by Ryan Nelsen and fell kindly too him.