Steven Davis scored with the last kick of the ball to give Rangers a Champions League lifeline against Malmo.
A two-minute lapse in Sweden could yet prove costly but a deflected strike from Davis in the fifth minute of stoppage time reduced the deficit to 2-1 midway through the third qualifying round tie.
Rangers goalkeeper Allan McGregor only needed to make one first-half save but he was beaten twice inside the opening four minutes of the second period.
Goals from Soren Rieks and Veljko Birmancevic put Jon Dahl Tomasson’s side in the driving seat for a place in the play-offs, where Olympiacos or Ludogorets will be waiting.
But Davis – who played for Rangers in a Champions League qualifying defeat by Malmo 10 years ago – struck a powerful left-footed effort from 20 yards which hit a defender and wrongfooted the home goalkeeper.
Steven Gerrard made seven changes for Rangers’ first European game of the season with Cedric Itten starting amid the continued absence of Alfredo Morelos. As well as the suspended Leon Balogun, Glen Kamara and Kemar Roofe, Gerrard was also missing Joe Aribo.
Malmo started with two up front and Rangers took time to settle but McGregor’s only first-half concern was a low effort from Jo Inge Berget, which he dived to save.
Malmo were leaving space at the back and Rangers had first-half chances. Scott Arfield had an effort saved and Ryan Kent and Scott Wright both caused problems without testing the goalkeeper.
The game turned just after the interval. The Rangers defence got dragged to the left and Birmancevic crossed for Rieks to volley home in space.
Tavernier soon misplaced a pass near the halfway line and the backtracking Connor Goldson could not cut out the pass, leaving Birmancevic to net.
Rangers took time to respond but they pushed Malmo back for the final 20 minutes and Davis struck in the nick of time.
