Calamity goalkeeper Robert Douglas helped Rangers deservedly end their five-year Celtic Park hoodoo with a terrible second-half blunder as Rangers moved three points clear of their arch rivals at the top of the Bank of Scotland Premier League.
Alex McLeish's men had not won here since March 2000 - 11 matches ago - but they gained the upper hand in the championship race in an error-strewn battle in the east of Glasgow.
Scotland number one Douglas let Gregory Vignal's shot slip through his grasp and into the net before Nacho Novo wrapped up the points by chipping Douglas late on.
Celtic manager Martin O'Neill took a gamble in handing new bhoy Craig Bellamy his debut after joining on loan from Newcastle with Chris Sutton moving into midfield.
The temperamental Wales international looked short of match fitness.
But Bellamy showed his frightening pace as early as the fourth minute when he cut inside Marvin Andrews before driving in a low right-foot shot from a wide angle which another debutant Ronald Waterreus looked unconvincing as he gathered at the second attempt.
The livewire, however, could have stamped his name in Celtic folklore in the 15th minute when he was sent clean through on goal by John Hartson.
The Welshman emerged with the ball from a tackle with Kyrgiakos and fed his countryman but Waterreus stood up well to save his left-foot curler with his foot.
There was no early sign of the indiscipline which marred last year's ugly Ibrox league game and Thompson's shuddering 22nd-minute challenge on Fernando Ricksen was a fair one.
Bellamy returned the favour for Hartson after his earlier chance in the 25th minute but the former Manchester City goalkeeper was again equal to it.
Sutton found the diminutive striker and he played a great ball through for Petrov to put the ball into Hartson's path.
The frontman got a good contact on the ball but Waterreus again performed heroics as he thrust his hand up to keep the ball out.
Rangers looked menacing on the break and Kyrgiakos should have done better when he got on the end of Michael Ball's corner but headed over the top from eight yards out.
Referee Mike McCurry, who was wearing an almost identical colour top to Douglas, had spoken about being victimised by Scottish football fans in midweek and he was upsetting both sides with some strange decisions during the first 45 minutes.
Vignal provoked the home crowd by pushing a ballboy as he attempted to retrieve the ball quickly for a throw-in but the official let the incident pass.
Douglas had to race off his line to clear the danger when Novo threatened to pounce but the scores remained level at half-time.
However, Rangers started the second period strongly and Prso's low drive was deflected just wide before Bobo Balde was booked for a crunching challenge on Fernando Ricksen.
Novo then turned Stanislav Varga inside out in the area but Laursen got back well to clear and McCurry carded Andrews for climbing all over Hartson moments earlier.
But Douglas kept the scores level in 55th minute when Rangers ripped the dozing Celtic defence wide open.
Thomas Buffel fed Prso and Balde went sleep walking towards the ball to give the Croatian a route to goal but the goalkeeper kept his stinging right-foot effort out.
Alan Hutton was the next man booked for tripping Hartson before Celtic almost made the most of the resulting free-kick.
Hartson looked set to latch on to Thompson's cross with a diving header but Andrews got a vital deflection to divert the ball behind for a corner.
That woke Celtic up and the Welshman would again have put them in front but for the heroics of Waterreus.
The Rangers defence left him in acres of space from Thompson's cross but the goalkeeper blocked his powerful effort.
The game became pulsating again with Prso firing into the side-netting and Hutton blazing wide at the other end.
But, while Waterreus had stepped into the sizeable shoes of Stefan Klos with great credit, Douglas was to make a terrible mistake to gift Rangers the lead in the 71st minute.
Vignal let fly with a 25-yard shot which the experienced Scotland number one should have stopped but he let it slip through his grasp and into the back of the net to the surprise of everyone including the Frenchman, who ran away to celebrate.
The Rangers supporters cruelly chanted the name of the embarrassed Celtic goalkeeper as their heroes continued to threaten with the keeper saving Prso's shot.
But Novo was also celebrating in the 82nd minute when he brushed Laursen off the ball from Ricksen's long ball and then lobbed the red-faced Douglas.
He must have wanted the ground to swallow him up and he made a meal of turning Prso's overhead kick over.
And there was late controversy when Ricksen, who had applauded the Rangers fans in front of the Celtic fans with five minutes remaining, appeared to be hit by a missile thrown from the home supporters.
Novo could have made it three late on but he pulled his right-foot shot wide before the referee put Celtic out of their misery while sparking wild Rangers celebrations.
Teams
Celtic Douglas, McNamara (McGeady 77), Balde, Varga, Laursen,
Petrov, Lennon, Sutton, Thompson, Bellamy, Hartson.
Subs Not Used: Marshall, Henchoz, Juninho Paulista, Lambert,
Maloney, Wallace.
Booked: Balde.
Rangers Waterreus, Hutton, Kyrgiakos, Andrews,
Ball (Alex Rae 73), Buffel, Ferguson, Ricksen, Vignal, Prso,
Novo.
Subs Not Used: McGregor, Namouchi, Burke, Malcolm, Thompson,
Lovenkrands.
Booked: Andrews, Hutton.
Goals: Vignal 71, Novo 82.
Att: 59,041
Ref: M McCurry (Scotland).