Rowett's men started on the front foot and after 15 minutes went ahead when Shaun Williams nodded home from Jed Wallace's pinpoint cross.
Grabban was then introduced just after the hour mark and took less than a minute to equalise before clinically dispatching Sammy Ameobi's centre two minutes from time to send the away fans wild before O'Brien's late strike gave the game one last twist.
The south-east Londoners have picked up nine points from their last six outings and are unbeaten in six at The Den.
They began the match in understandably confident fashion, with Ben Thompson flashing a close-range header over from Mahlon Romeo's centre after five minutes.
It was not long before the hosts had got their noses in front. Wallace's delivery was pinpoint and despite a scrum of bodies in the penalty area, Williams arrived unmarked to nod in from around six yards.
Before Rowett's arrival, the Dublin-born midfielder last scored at the beginning of March 2019 against Norwich, but now the 33-year-old has scored in successive games following his eye-catching strike against Birmingham at St Andrew's last Saturday.
Romeo then tested Brice Samba with a rasping drive as Wallace ran the show.
Sabri Lamouchi's men knew they had to show more quality in the second half and Ameobi almost provided the perfect response, with Bartosz Bialkowski getting down smartly to keep out the former Bolton winger's volley.
Grabban's arrival was unsurprisingly met with a chorus of boos, but he silenced the hosts by stabbing home from on the goal line following Joe Worrall's header back across Bialkowski.
The game was being played at 100 miles an hour and Thompson almost came up with an inspired assist, hooking the ball over the Forest backline only for Jake Cooper's header to be shovelled away by Samba.
Cooper then had another header cleared off the line, before Alfa Semedo screwed a shot into the side-netting after Lamouchi's men had looked to have carved open the home side with a flowing team move.
Grabban then looked set to perfectly fulfil the villain's role, racing into the box to get on the end of Ameobi's perfect delivery as Millwall fans howled in despair.
However, the visitors were dealt a late blow when Williams saw his hopeful snapshot from 25 yards spilled by Samba, with O'Brien on hand to bury the rebound.
