Jack Klugman was born to a poor Jewish family on April 22, 1922. He lived in South Philadelphia, Pennsylvania, where his father was a painter who died when Klugman was in his early teens. His mother supported the family of six children by making hats during the evenings.

Klugman's childhood led him to gambling as a way of life after a short time in the Army. He said, "This is a true story. The gambling is what got me into it. I owed a loan shark, who was also a friend, some money. I couldn't pay him, so he turned the debt over to a couple of guys who were going to hurt me a little bit. I had to get out of town. Since I had the GI bill, I remembered my brother knew a guy in the Army who had been to Carnegie Mellon University, so I went there." At the university, Klugman enrolled in the drama program, and at first, was a miserable performer. After his first audition, his teacher told him, "you're not suited to be an actor. You're more suited, Mr. Klugman, to be a truck driver. Not that there's anything wrong with truck drivers, but you're really not ready for this." Despite his confidence being shaken by his teacher, he continued to attend Carnegie Mellon.

He also went on to study in New York at the American Theatre Wing. Klugman was not viewed as a "natural" when it came to acting; however, it was something he was passionate about. In 1949, Klugman's stage debut, although non-salaried, was at the Equity Liberty Theatre in Saint Joan. He excelled at acting, but he still was trying to make ends meet, so he did side jobs in order to keep the apartment he rented with roommate Charles Bronson. He also appeared on The US Steel Hour, Alfred Hitchcock Presents, and The Twilight Zone. In 1956, he married Brett Somers, known for being a long-time guest on the popular Match Game television series. Together, Somers and Klugman had two sons, Adam and David. Klugman has three children total; he had a daughter named Leslie from another relationship.

He made a stage appearance in New York for Gypsy, which was his "big break," and he received a Tony nomination for Best Actor in a Musical. He soon landed his most important early role in The Defenders and from there his career took off. In 1964, he won an Emmy for Outstanding Single Performance by an Actor in a Leading Role for his exceptional performance.

In 1970, Klugman played Oscar Madison in the memorable TV series The Odd Couple. Madison was a newly-divorced sports writer who shared his apartment with Felix Unger, a photographer who was also recently divorced. Unger, played by Tony Randall, a neat freak, and Madison, a cigar smoker. They were two polar opposites, which made the television series very popular. Klugman received two Emmy awards, Outstanding Continued Performance by an Actor in a Leading Role in a Comedy Series, in 1971 and 1973. He was also awarded a Golden Globe in 1974 for Best TV Actor in a Musical/Comedy.

After Klugman's many years on The Odd Couple, he starred in another TV series called Quincy M.E., where he played a tough medical examiner who seemed to always find evidence commonly missed by the other examiners. Quincy would usually end up fighting with his boss and the investigating detective about evidence that had potential to change the entire case. The show started as solely solving crimes, but it evolved into a vehicle for telling stories about social injustice.

Klugman did not have problems landing lead roles at this point in his career; however, he did have trouble managing his health. He was an avid smoker and said he was influenced by John Garfield. This bad habit caused him to develop cancer of the larynx in 1974. He had surgery and treatment, which allowed him to continue acting, even though he did not quit his habit of smoking. The cancer came back in 1989, and his right vocal cord had to be removed because of it. This left him unable to speak. He underwent chemotherapy and rehabilitation in order to regain his voice. Klugman received the American Speech and Hearing Association's International Media Award for his struggle to recover his use of speech.

Klugman appeared in an episode on the drama series Crossing Jordan, where he played a medical examiner. In 2005, Klugman wrote a book, Tony and Me: A Story of Friendship, in which he speaks of his friendship with Tony Randall, from their first meeting on the set of The Odd Couple until shortly before Randall's death. The day Tony Randall died a reporter quoted Klugman, who said: "A world without Tony Randall is a world I will never be able to recognize; it is certainly one I will never accept." They were the best of friends and Klugman truly appreciated their friendship.

One of Klugman's more recent movies was When Do We Eat? which was released in spring 2006. In May 2006, he was awarded with the Strusser Award for Public Service from the NYU School Dentistry at Madison Square Garden in New York City. One of his dreams was having the opportunity to play the lead male character Willie Lohan from Arthur Miller's play "Death of a Salesman." He had the opportunity to fulfill this dream in 2008, when he portrayed Willie in a theatrical performance on stage.

He currently is living in Malibu, California, where he spends time with his sons, Adam and David, their wives, Nancy and Carol, and his granddaughters, Katherine and Olivia. At his home in California, races his horses and hopes that somebody he will win the Kentucky Derby. In February 2008, he married his girlfriend of many years, Peggy Crosby. Jack Klugman said, "Poverty can teach lessons that privilege cannot." Klugman's own words speak to his past tribulations and his new found knowledge he gained by succeeded in the harsh business known as acting.