Forster had broken former Aberdeen goalkeeper Bobby Clark's 43-year-old Scottish league record on Saturday and it took something special to finally beat him after 1,256 minutes as Hayes struck a spectacular long-range effort.
Having conceded for the first time in 14 Scottish Premiership games, the England international shipped another goal four minutes later when Adam Rooney sent a diving header past him from Niall McGinn's cross seconds before the interval.
Substitute James Forrest's excellent finish in the 61st minute got Celtic back in the game following a spell of pressure but they had been up against it from the 11th minute, when Virgil van Dijk was sent off for bringing down Peter Pawlett.
And the Dons held out to inflict Celtic's first league defeat for 10 months since they lost at Motherwell.
Celtic had won their previous 15 Premiership games and not lost in 26 this season but if any team were to break Celtic's impressive runs, Aberdeen would have been the obvious choice.
Forster previously conceded a Scottish Premiership goal against McGinn three months earlier and the Dons knocked them out of the William Hill Scottish Cup earlier this month.
Celtic started the game well and Jamie Langfield made good stops from Charlie Mulgrew and Scott Brown inside the first 10 minutes after the visitors had carved out shooting chances inside the box.
But the game turned on a loose pass from Efe Ambrose. The Celtic defender gifted possession to Rooney and the Irishman released Pawlett, whose burst of speed took the ball past the sliding challenge of Van Dijk and forced the Dutchman to bring him down two yards outside the box.
Craig Thomson showed little hesitation in brandishing the red card and Neil Lennon filled the hole in defence with Mulgrew, who had started in midfield after Emilio Izaguirre replaced the injured Stefan Johansen.
The Dons enjoyed spells of possession and pressure but Langfield had to be out of his box quickly to deny Nir Biton after a searching long ball from Anthony Stokes.
Rooney was just shy of connecting with a wonderful cross from Hayes but Langfield made another excellent stop after Stokes had set up Biton.
Celtic and Forster's resistance was broken in the 41st minute. Aberdeen were knocking the ball about patiently until Shaleum Logan fed Hayes 30 yards out.
The Irishman took a touch and unleashed a powerful left-footed strike that flew into the top corner.
Logan was involved in the build-up to the second. Willo Flood's crossfield ball found him in space on the right wing and he knocked the ball back to McGinn, whose cross was perfectly weighted for Rooney to run on to and glance a header just inside the far post.
Forrest replaced Kris Commons at the break and helped Celtic start brightly as they put pressure on Aberdeen, who survived a penalty appeal when the visitors claimed Mark Reynolds had handled after a long throw.
Celtic soon got themselves back in the game after another ball into the box when Brown laid it off to Forrest to sidefoot into the roof of the net from 12 yards.
Brown shot wide after a one-two with Stokes before Forster saved from Barry Robson and Hayes as Aberdeen tried to relieve the pressure.
Georgios Samaras replaced Griffiths and Celtic were rejuvenated, but Langfield held a long-range effort from Stokes and was then protected by his defence in the final stages.