David Daniel Kaminsky was born to Ukrainian Jewish immigrants in Brooklyn. Jacob and Clara Nemerovsky Kaminsky and their two sons, Larry and Mac, left Ekaterinoslav two years before his birth; he was the only son born in the United States. He spent his early youth attending Public School 149 in East New York, Brooklyn (on Dumont Avenue, c. Williams Avenue) -- which eventually was renamed to honor him -- where he began entertaining his classmates with songs and jokes, before moving over to Thomas Jefferson High School (on Pennsylvania Ave., by Dumont), though he never graduated. His mother died when he was in his early teens. Clara enjoyed the impressions and humor of her son and always had words of encouragement; her death was a loss for the young Kaye.

Not long after his mother's death, Kaye and his friend Louis ran away to Florida. Kaye sang while Louis played the guitar; the pair eked out a living for a while. When Kaye returned to New York, his father did not pressure him to return to school or work, giving his son the chance to mature and discover his own abilities. Kaye said he had wanted to be a surgeon as a young boy, but there was no chance of the family affording a medical school education.