After winning the toss and electing to bat, Australia lost captain Aaron Finch for six in the third over, with wicketkeeper-batsman Alex Carey then out for 24 off 31 balls.
Half-centuries from Usman Khawaja (59) and Shaun Marsh (54) helped push the total on, while Peter Handscomb's fine 73 brought up the 250 mark.
Marcus Stoinis added a rapid 47 from just 43 balls and Glenn Maxwell was 11 not out at the end of the 50 overs, with Australia having posted 288 for five. Bhuvneshwar Kumar and Kuldeep Yadav both had two wickets each.
In reply, Australia debutant Jason Behrendorff claimed a first wicket in the opening over as Shikhar Dhawan was out for a duck.
Jhye Richardson then claimed two wickets in three balls to account for skipper Virat Kohli (three) and Ambati Rayudu, out for a seocnd-ball duck, with the tourists rattled at four for three.
India - who had seen opening batsman KL Rahul and all-rounder Hardik Pandya suspended following controversial comments they made on a recent television show - mounted a recovery as they looked to build on a first Test series win Down Under.
Sharma and MS Dhoni (51) helped push India on into three figures, but behind the run-rate, before Dinesh Karthik (12) was out in the 40th over and 113 was needed for victory.
The opener did his best to lead an unlikely victory chase, but ran out of both partners and time. Richardson had picked up a fourth wicket, trapping Dhoni lbw in a decision which would have been overturned on review if India had any referrals left, before Sharma was eventually dismissed for 133, caught by Stoinis off Maxwell.
Sharma's fine innings off 129 balls, including 10 boundaries and a six saw him become only the fifth Indian batsman to pass 10,000 one-day international runs.
Veteran Australian bowler Peter Siddle, 34, marked his first ODI appearance since November 2010 with a late scalp, when Kuldeep Yadav was out for three, more than eight years and two months since his last 50-over wicket success.
Western Australia paceman Richardson finished with four for 26 from his 10 overs as India were 254 for nine. The three-match series continues in Adelaide on January 16.
Related cricket links
