Kevin Ellison scored his first goal for Rotherham as they dumped last season's Carling Cup semi-finalists Derby out of the competition.
After some early pressure from the home side which saw Paul Warne shoot just wide, the Championship side went ahead after 11 minutes from their first attack.
Paul Connolly's pass set Lee Croft free on the right and with everyone expecting a shot he pulled the ball back across the edge of the area for Gary Teale to drill home.
But Rotherham refused to lie down and they kept Derby pinned in their own half. Pablo Mills went close after 22 minutes and their pressure finally paid off a minute before the break.
Jamie Green's free kick into the penalty area was headed down rather than cleared and Paul Warne pounced for his second goal of the season from 10 yards out.
Nicky Law almost set up Warne for number two just after the break, but with 65 minutes gone the League Two side went in front. Law crossed from the left and Warne laid the ball off for Kevin Ellison to fire home.
County almost rescued it, but Dean Moxey's injury time shot was just inches the wrong side of the post.
Teams
Rotherham Warrington, Tonge, Sharps, Fenton, Green,Ellison (Liddell 70), Mills, Jason Taylor, Law, Warne, Pope.
Subs Not Used: Annerson, Harrison, Cummins, Joseph, Lynch,
Ryan Taylor.
Goals: Warne 44, Ellison 65.
Derby Bywater, Connolly, Addison, Buxton, Moxey, Croft,Pringle (McEveley 80), Savage, Pearson, Teale,Davies (Varney 15).
Subs Not Used: Deeney, Leacock, Barker, Mendy.
Booked: Connolly.
Goals: Teale 11.
Att: 4,345
Ref: David Webb (Durham).