Huddersfield manager Simon Grayson promised to launch an appeal after turning on referee Lee Collins for the first-half dismissal of Keith Southern.
- Related Content
The Terriers were rescued by Adam Clayton's late penalty to cancel out Rob Hulse's opening goal for Charlton.
But Southern's controversial red card, following a challenge with Michael Morrison, changed the outcome of a game that Huddersfield were controlling.
It meant play-off chasing Town had to settle for a draw - but Grayson knows it could have been so much better.
"I've seen it on the DVD and it's two committed players going in 50-50 on a wet pitch," he said.
"He clearly got the ball first.
"But the referee made a quick decision. It's changed the outcome of the game.
"It was a shame as it would have been a great game 11-11. The conditions were there to pass the ball around and it had the makings of a great game.
"But then it was spoilt.
"Even Morrison said to Keith that you got the ball first. He has not jumped off the ground.
"He has won so many tackles like that this season and over his career. And he has hardly been booked.
"We will be appealing. Do you think we would have got at other end it didn't have a player sent off? I don't know, I wouldn't have thought so.
"It does feel like a win for us though scoring so late."
Huddersfield set the tone for their early dominance as Peter Clarke and Lee Novak wasted early chances.
Then came Southern's early bath for a late lunge on Morrison with just 35 minutes played.
Charlton still struggled to make their numerical advantage count as they looked devoid of ideas until the 60th minute breakthrough.
Bradley Pritchard's cross was turned in by on-loan striker Hulse for his third Charlton goal.
But Grayson's men were rewarded when Chris Solly hauled down Clarke, with Clayton striking home with two minutes remaining.
Charlton manager Chris Powell said: "It's not a sending off. It's an honest challenge and both didn't pull out.
'He makes a decision and it's 11-10 and it can be tough breaking them down.
"We got the breakthrough and had a chance to wrap the game up. It wasn't taken and we had to stand firm. We just couldn't hang on.
"We had chances to win it even after that and it wasn't to be.
"People are frustrated, but have to take positives. It's 10 points from the last 12 and we are in a good place. It keeps our run going."
Powell praised new Arsenal loan signing Emmanuel Frimpong, who caught the eye during his hour-long debut at The Valley.
"Frimpong was very good for the hour he was on. He showed strength, passing ability and we knew we were getting a good player.
"For a debut, in those conditions, he should be proud."