Walter Annenberg was born on March 13, 1908, in Milwaukee, Wisconsin, to Moses and Sadie Annenberg. Annenberg was their only son in a family of eight children. He was deaf in one ear and as a result had a stuttering problem, which led to many visits to speech disorder specialists and continued throughout his college life. Annenberg began his schooling at a Hebrew school, but switched to a German-English academy after a dispute about a donation his father made to the synagogue. In 1920, the family moved to New York City when Moses Annenberg accepted a job from William Randolph Hearst. Annenberg's first year in New York was spent at a small private school in an attempt to improve his stutter, but this had no effect. He then attended a Baptist school in New Jersey, Peddie School, for five years. He graduated from Peddie in 1927. At his father's urging, he then went to the Wharton School of Finance and Commerce at the University of Pennsylvania. In September of 1927, Annenberg moved to Philadelphia, where he joined a Jewish fraternity. Annenberg made his decision to drop out of Wharton at the end of his freshman year, much to his father's disappointment.

Annenberg married his first wife, Ronny Dunkelman, in 1938, and they had two children, Wallis Huberta and Roger. Dunkelman found the mansion in Wynnewood, Pennsylvania, that would become Annenberg's life-long home, but she and Annenberg were divorced in 1949. In 1951 Annenberg married Leonore Rosenstiel, his second wife.

At a very young age, Annenberg showed financial aptitude, proving to be a natural when his father taught him to trade stock. He also developed the idea of selling advertisements in Peddie's prom program. In 1928, after dropping out of the University of Pennsylvania, Annenberg traded stock full time. He made $3 million, only to lose it when the stock market crashed at the onset of the Great Depression. The loss of this first fortune proved to be the beginning of his career in his father's company, Triangle Publications.

Annenberg began his career with minor responsibilities: he signed checks, did some reports, and wrote a column, which he eventually gave up at his father's prodding. He gradually became more interested in assuming responsibilities and was promoted to the position of business manager. At this point, Annenberg frequented his father's estate, Sunnylands, in the Poconos, near Milford, Pennsylvania. In 1936 Walter Annenberg's father bought thePhiladelphia Inquirer, a move that provoked controversy since the paper subscribed to policies of the Republican Party, and Annenberg's father was a Democrat. However, unlike his father, Annenberg was comfortable with stories that had mass appeal. He moved into his own apartment in the Rittenhouse Plaza on Rittenhouse Square, one of the five original city parks. Annenberg gradually assumed control of the business when his father was imprisoned for tax evasion. By the time his father died shortly after his release from prison, Annenberg was fully capable of running the business. He was determined to restore the family finances, after having to repay the government for his father's tax evasion, as well as the family name.

Annenberg foundedSeventeenin 1944, a magazine aimed at teenage girls, andTV Guidein 1953, a publication that included both TV listings and feature stories, both of which immediately proved to be commercial successes. Both of these were a part of Triangle Publications, as were several Philadelphia, Altoona, and Lebanon (all Pennsylvania) radio and television outlets. In 1988 he soldTV Guideand Triangle Publications for around $3 billion—at the time, the largest amount ever received for a publishing company.

Annenberg founded the Annenberg School of Communication at the University of Pennsylvania in Philadelphia, as well as communication schools at other universities. He also established the Annenberg Foundation in 1989 to benefit education. It was to be used as a fund for reforming and restructuring public schools. Among the many awards Annenberg received during his life were the Gold Medal of the Pennsylvania Society, the Award of Greater Philadelphia Chamber of Commerce, and the Medal of Freedom. Annenberg served as the ambassador to Great Britain from 1969 until 1974. He received honorary degrees from the University of Pennsylvania and Temple University.

Annenberg spent his later years still deeply involved in philanthropy, donating money to education ($500 million for public school reform, as well as individual donations to many universities) and art ($11.5 million to the Academy of Music, Philadelphia; $10 million to the Independence Mall Project, Philadelphia; as well as over $1 billion to the Metropolitan Museum of Art), as well as other causes. He died at the age of 94 in his home in Wynnewood, Pennsylvania, of complications from pneumonia on October 1, 2002.