Born in Baltimore, MD in 1910, Baltimore native Berger attended McKinley Tech High School in Washington, D.C.. Nicknamed “Bosey” by his grandmother, he attended the University of Maryland to play basketball. At the time, this was considered a coup because Maryland was a small agricultural school compared to the athletic giants recruiting Bosey.
He led Maryland to the 1931 Southern Conference Championship averaging the 19.1 points over the nine-game season. Berger was Maryland's first basketball All-American (2 times) and is regarded as one of the greatest and most likable Maryland athletes.
After graduation, he switched sports signing a professional contract with the Cleveland Indians baseball team. He played in the major leagues for six years with the Indians, White Sox and Red Sox.