Born in Austin, Minnesota, Hormel has been married to Alice Turner and is the father of four daughters and one son. His family are wealthy through the Hormel Meat manufacture of processed food, including "Spam". He was dean of the University of Chicago Law School. He became a member of the US delegations to the UN Human Rights Commission in Geneva and to the 1966 UN General Assembly.

His appointment as the US ambassador to Fiji then to Luxembourg was first announced in October 1997. Some Republican senators blocked his appointment because of objections to his financial and public support of gay rights causes. President Bill Clinton by-passed the senate by making a "recess appointment" during the Memorial Day holiday which allowed James Hormel to take up the post until the end of Bill Clinton's term on 3rd. January 2001.

James Hormel became the first openly gay man to represent the United States as an ambassador in June 1999 when he was sworn in by US Secretary of State Madeleine Albright. He took his oath of office with his hand on a Bible held by his partner, Timothy Wu.

Hormel was known primarily as a philanthropist who gave major gifts to left-wing organizations like the ACLU, and supported groups like the Sisters of Perpetual Indulgence. The San Francisco Library's Gay and Lesbian Center was named for him after his $500,000 donation allowed its construction to be finished.