Albion went top of the Scottish Third Division as two goals in the last eight minutes gave them a 4-2 victory over Queen's Park.
A brace from Jered Stirling - both from the spot - had given Rovers a 2-0 lead at the interval but Stewart Jack and Stevie Moffat netted to level things up after 81 minutes.
But Rovers secured the points as Iain Dack and Jim Coulter netted late on to send them above East Fife, who lost 2-1 at Morton.
Alex Williams put the hosts in front on nine minutes and Jani Uotinen made it 2-0 a minute before the break.
Ross Graham gave the visitors hope two minutes after the interval, but they could not force a second.
Peterhead moved level on points with Morton thanks to a 3-0 win at Montrose.
Goals from Andy Roddie (14), Kevin Bain (60) and Lamile Camara (77) proved the difference between the sides.
Elgin secured their spot in mid-table with a 2-1 victory at East Stirling.
The hosts went in front as early as the eighth minute when Brian Fairbian netted.
But Kevin Steele's 29th-minute penalty and a 54th-minute strike from David Ross ensured the visitors took all three points.
Stirling also won on their travels, beating Gretna 2-0.
Goals from Colin McKinnon (14) and Robert Dunn were enough.