Secret agents charged with protecting President Barack Obama during a summit in Colombia have instead embarrassed him by getting caught apparently trying to party with prostitutes.

At least one member of the Secret Service -- famous for dark shades, sharp suits and stern demeanor -- tried to take a prostitute up to a hotel room in tropical Cartagena without registering her, a local policeman told Reuters.

The U.S. government said only that some agents were sent home after allegations of "misconduct" and would face the agency's Office of Professional Responsibility. The White House said Obama was briefed on the matter on Friday.

Also Saturday, U.S. military officials confirmed that five service members are accused of misconduct at same hotel -- Cartagena's Caribe Hotel -- as Secret Service, The Associated Press reported.

Ronald Kessler, a former Washington Post reporter and author of the book “In the President’s Secret Service: Behind the Scenes With Agents in the Line of Fire and the Presidents They Protect,” first heard about the investigation and tipped off Post reporters Friday night, Politico reported Saturday.

“It is the biggest scandal in Secret Service history,” Kessler said in an interview Saturday with Politico. “It is all part of this pattern that I wrote about in the book of corner cutting, laxness, cover up.”