Brandon Jennings added 18 points for the Bucks, who moved to within two games of the idle New York Knicks for the eighth and final playoff spot in the Eastern Conference.
Ellis, who has struggled since coming over in a trade from Golden State on March 14, tied a season low with four points against the Knicks on Monday. On Tuesday, he went 15-of-24 from the field, including 7-of-9 in the fourth quarter, sealing the victory with a 15-foot jumper with 1:27 remaining. He added eight assists.
Josh Smith led the Hawks with 30 points and 18 rebounds. Joe Johnson finished with 11 points on 4-of-13 shooting.
The Hawks had 22 turnovers.
The Bucks were without two key players, forwards Ersan Ilyasova (back spasms) and Carlos Delfino (right groin injury). Before the game, coach Scott Skiles called both “day-to-day.”
The Bucks led 55-46 at halftime, but the Hawks used a 20-6 run over the last 6:58 of the third to take a 79-78 lead. The Hawks made 12 of 17 shots in the period, while the Bucks missed their last five shots of the quarter.
Atlanta, which had been seeking their season-high fifth consecutive win, went up 81-78 with a basket to open the fourth, but Ellis had a steal and a break-away dunk, two jump shots and a free throw in the first five minutes of the period to put Milwaukee up 89-85.
The score was tied at 91 when Mike Dunleavy hit a 3 with 4:31 remaining. The Bucks never trailed again, as Ellis followed with a layup and jumper to make it 98-91.
Rookie Ivan Johnson had a career-high 17 points for Atlanta.
The Bucks’ last seven wins had come against teams with a losing record, and they had not beaten a team above .500 since March 5.
It was the fifth game in six nights for the Bucks, who were coming off an 89-80 defeat to the depleted Knicks. The Hawks were coming off a 139-133 quadruple-overtime victory over Utah on Sunday.
Joe Johnson left Tuesday’s game less than 3 minutes in and received two stitches in his lip, the team said. He returned at the 8:59 mark of the second.
The Bucks led by as many as 17 in the first half and 55-46 at halftime. Smith scored 13 points in the second quarter, and Johnson had just two points in the half on 1-of-5 shooting.