RIO DE JANEIRO (AP) — Michael Phelps closed out the Rio Olympics in the only way imaginable.

Golden.

Phelps put the United States ahead to stay on the butterfly leg of the 4×100-meter medley relay and Nathan Adrian finished it off, giving the most decorated athlete in Olympic history his 23rd career gold medal Saturday night.

If that was the end, and Phelps insists it is, what a way to go.

He has 28 medals overall, having won five golds and a silver at these games.

“This is how I wanted to finish my career,” Phelps said. “Getting off the bus walking into the pool tonight, I pretty much felt myself starting to cry. Last time putting on a suit, last time walking out in front of thousands of people representing my country.”

As Adrian touched the wall to finish off the victory, Phelps gathered the other relay swimmers, Ryan Murphy and Cody Miller, in his arms. One night after his only setback in Rio, an upset loss to Joseph Schooling in the 100 fly, Phelps was back on top.

In the stands, his fiancee, Nicole Johnson, bounced along to the music with their son, 3-month-old Boomer, cradled in her arms.

Phelps is eager to spend a lot more time with them. He plans to marry Johnson after the Olympics and said he wants to watch his son grow, maybe even dole out a swimming lesson or two.

Most of the U.S. swim team was in the stands to watch Phelps’ finale, including the biggest female star of the games, Katie Ledecky.

The 19-year-old Ledecky joked that she was proud to be part of Phelps’ final Olympics — twice. He initially retired after the 2012 London Olympics, only to decide about a year later to return to the pool.

The comeback endured a huge setback with his second drunken-driving arrest in 2014, which led to Phelps being banned from the world championships the following year. But it also sparked a turnaround in his life. He entered six weeks of inpatient therapy, where he got in touch with some of the issues that seemed to lead him astray outside the pool.

He quit drinking, reconnected with his estranged father, got engaged, moved to Arizona with his longtime coach Bob Bowman, and became a father.

At 31, he sounds much more adamant when he says his swimming career really is over.

“These games really showed his growth,” teammate Anthony Ervin said. “That human spirit, that capacity to heal. I think it showed in his swimming, it showed in his demeanor, and it certainly showed in his leadership on the team.”

Phelps was elected a team captain for the first time in his career — this was his fifth Olympics — and truly seemed to enjoy being around his fellow swimmers. He was still the same ruthless competitor, but he was also willing to join in when some of his younger teammates made a carpool karaoke video at their final training camp in Atlanta.

Took a starring role, in fact.

“Being Michael requires such isolation,” Ervin said. “Other people respect that. They give him that space because he is the greatest. But this time around he started reaching out, reaching out to other people, bringing them closer, letting that gap be bridged. That was special.”

Standing atop the medal podium for the 23rd time, listening the “The Star-Spangled Banner” as he’s done so many times before, Phelps teared up a bit and gave a little nod.

Then he and his teammates grabbed a sign that said, “Thank You Rio.”

Two-time gold medalist Murphy put the Americans out front with a world-record split on the backstroke — it counts since he was leading off — but Britain surged ahead on the breaststroke with its own world-record holder, Adam Peaty.

Phelps dove into the pool in second place.

He wouldn’t be for long.

On the return lap, Phelps powered through the water with his whirling butterfly stroke, surging ahead of James Guy to pass off a lead to the anchor Adrian.

It wasn’t in doubt after that. Adrian pulled away on the freestyle to win in an Olympic-record time of 3 minutes, 27.95 seconds. Britain held on for silver, with Australia nabbing bronze.

The victory came just minutes after the women’s medley relay gave the United States its 1,000th Olympic gold medal at the Summer Games.

“A thousandth gold for team USA,” said Simone Manuel, who swam the anchor leg for her second gold of the games and second medal of the night. “It’s a nice number.”

Kathleen Baker, Lilly King and Dana Vollmer joined Manuel in the historic victory, which came with a time of 3:53.13.

“It’s really special,” Manuel said. “Sharing that with three other women is just icing on the cake.”

Australia earned silver, while Denmark took bronze.

Earlier in the night, Manuel took silver in the 50 free. She already became the first African-American woman to win an Olympic swimming title with her win in the 100 free.

Connor Jaeger gave the U.S. another silver in the 1,500 free, leaving the American with 33 swimming medals in Rio — matching the highest total since the boycotted Los Angeles Games in 1984.

The U.S. also won 33 medals at Sydney in 2000.

The final two individual golds of the games went to Pernille Blume of Denmark in the 50 freestyle, her country’s first swimming victory since 1948, and Italy’s Gregorio Paltrinieri in the grueling 1,500 free.

The night, though, belonged to Phelps, who walked out of the arena for final time carrying an American flag handed to him by his mother from her front-row seat, right next to Johnson and little Boomer.

With a gold medal around his neck.

The only way imaginable.