Phelps, who won an unprecedented eight Olympic gold medals at the Beijing Games last August, was just eight-hundredths outside the world record of 50.40 held by compatriot Ian Crocker, in Saturday's competition.
He also improved on his personal best time of 50.58sec, set in winning gold at Beijing.
"I'm really happy," Phelps said. "I had no idea."
Christopher Brady was a distant second in 52.48sec and Joe Bartoch was third in 53.58.
The 100m fly was one of eight events won by Phelps in Beijing - the only one in which he didn't set a world record.
"It was pretty much exactly the same race as Beijing - I was out a tenth slower," Phelps said. "I just need to get a little bit more rest leading up to nationals and hopefully take it out a little bit quicker."
Bartoch, who was on Canada's Olympic team last year, said he knew during the race that someting special was happening.
"It was awesome," he said. "The crowd went wild during the race and I realized he (Phelps) was doing something pretty impressive. It was fun."
Phelps said the performance showed he is on course in his post-Olympic year. His 2009 campaign was disrupted after a published photo of him holding a marijuana pipe prompted a three-month ban by USA Swimming.
"I have a goal in my mind that I want to hit this summer, and we're a few steps closer," Phelps said.
Asked to elaborate, he said the goal was "swimming faster than that."
The Montreal meeting is the 14-time Olympic gold medallist's third since Beijing. Last weekend at Santa Clara, California, he won the 200m butterfly and 400m freestyle but was beaten in the 100m backstroke and 100m freestyle.
"The intensity is still there, the drive to compete is still there, the love to swim is still there, so I think right now it's just getting back into better shape," Phelps said.
"I really feel like I'm in better shape than I was," added Phelps, who will test himself again in the 100m free here on Sunday. "I couldn't ask anything more about where I am and where I stand right now leading into the trials."
The competition is his final warm-up for the US championships that start in Indianapolis on July 7.
The nationals are the selection meeting for the World Championships in Rome later in July.