Darlington led for 86 minutes but trailed to two late Walsall goals as both sides finished the game with 10 men.
Jamaica international midfielder Omar Holness put Darlington ahead after 17 minutes and things looked good for the visitors when Walsall's Dan Scarr was dismissed with eight minutes to go.
But four minutes later Caolan Lavery drew Walsall level before Alfie Bates hit a late second.
Darlington's Ben Hedley was then shown a second yellow card before Rivers pounced seven minutes into stoppage time.
There was heartache for Maidenhead who led against League One Rotherham but have still to reach round two following a 3-1 loss.
Jake Cassidy gave the National League side the lead but, following a red card for Alan Massey after 54 minutes, the visitors took control through Michael Ihiekwe, Freddie Ladapo and Matt Crooks.
Southern League Premier Division side Stourbridge drew 2-2 with National League Eastleigh with goals from Aaron Lloyd and Scott Rendell, the latter a late equaliser.
National League South side Billericay saw their FA Cup adventure ended by Forest Green who won 4-0 with goals from Matt Stevens, Joseph Mills, Jack Aitchison and Liam Shephard.
National League South outfit Oxford City lost 5-1 at home to Solihull as James Ball hit a hat-trick.
Carshalton were beaten 4-1 at home by Boston and Ben Turner scored a late winner as Notts County won 3-2 away to fellow National League side Ebbsfleet.
There were 1-0 wins for Maidstone against Torquay, Mansfield over Chorley and AFC Fyle at Nantwich.
Blackpool triumphed 4-1 over Morecambe, Plymouth were 1-0 victors away to Bolton, Coventry won 2-0 at Colchester, Crawley beat Scunthorpe 4-1 and Port Vale won 1-0 at MK Dons.
Tranmere were held 2-2 by Wycombe and there were 1-1 draws in the games between AFC Wimbledon and Doncaster, Cambridge and Exeter, Cheltenham against Swindon and Grimsby and Newport.
Replays are also needed following 1-1 draws between Ipswich and Lincoln, Salford and Burton, Shrewsbury against Bradford, Stevenage and Peterborough and Sunderland and Gillingham.
Crewe were winning 1-0 at Accrington in a game delayed after injuries suffered by two match officials.
Follow Sporting Life on social media - find us on Facebook here or tweet @SportingLifeFC
Related football links
