Gary Cahill led England against Germany on Friday but the Chelsea defender is suspended for Sunday's match.
And Southgate confirmed on Saturday that Manchester City goalkeeper Hart, currently on loan with Torino, would have the armband for the Wembley contest.
He said: "Joe is our most-capped international and showed maturity since last summer's (European Championship) disappointment."
Southgate also gave more detail about the injury which rules Manchester United defender Chris Smalling out of Sunday's contest.
"He picked up a knock towards the end of the game in Germany. After assessment, he wouldn't be ready for tomorrow."
On Lithuania, the England boss said: "We'll have to pay attention to their wide players. They're good in counter-attack and against Scotland scored with some good interplay with their front players.
"We'll have to make sure our attention is correct on set-plays. They'll be the key areas for us.
"We want to play with a good tempo and press with a good tempo but also have calmness. Against Germany I thought we played with controlled aggression without the ball...sometimes just having that calmness in the opposite half of the pitch is a good reminder for us.
"It's important to make good decisions and we are capable of doing that. There's no need to rush those things if we haven't scored in the first 25-30 minutes.
"We need to make sure we make good chances rather than shots we don't need to take on, move the ball quickly but in a controlled manner."
Southgate was keen to praise the contribution of Jermain Defoe, who is back in the squad at the age of 34 after impressing with Sunderland.
Southgate added: "He's always had the ability to finish but he's in a very good condition.
"He's aware of his diet and prepares correctly so he's got great experience to pass on to the group - and that's been evident on and off the pitch this week."
