Born in Köln, he studied musicology, sociology and law at the Universities of Köln, Freiburg i. Br. and Grenoble. After he gained his doctorate, the rise of Nazism led Silbermann (a Jewish man) to emigrate to Netherland in 1933, and in 1938 from Amsterdam via Paris to Sydney. In Australia he started as a dishwasher but went soon from rags to riches with his own fast food restaurant Silver's Food Bars. He actually initiated the first fastfood chain of Australia and was a very successful entrepreneur.

His academic career startet 1944 in Sydney at the State Conservatory of Music. As an empirical culture-sociologist he went back to Europe (especially Paris) in 1951. In 1958 René König brought him back to Köln, where he taught at the University. 1964 he was called at the University of Lausanne as a successor of Vilfredo Pareto and later at the University of Bordeaux (1974-1979).

Silbermann was a member of the "Cologne School" (Kölner Schule) along with René König and others. He was one of the editors of the Kölner Zeitschrift für Soziologie und Sozialpsychologie (KZfSS) and foundet the Institut für Massenkommunikation (Institute of Mass Communication). He was an important German pioneer of empirical methods, as against the ideological biases of many contemporal colleagues like his favorite opponent Theodor W. Adorno.

He died in Köln.