Little is known about infants’ ability to deal with commonly encountered situations in which speech from one individual occurs simultaneously with that of others. Previous research has shown that while age of the infant and intensity of the background matter, so does the number of background speakers. The read-aloud multi-talker speech used in previous studies is perceptually different from conversational speech typically encountered by infants. To test generalizability, this study used a background of ecologically valid multi-talker speech. Using the head-turn preference procedure, infants were presented with passages of background noise with and without target infant-directed speech at a 10 dB SNR. Results show that while 9-month-olds prefer passages containing target speech with a white noise background, both 9- and 12-month-olds failed to show a preference with a multi-talker background. This inability to segregate speech streams under ecologically valid conditions demonstrates the adversity infants face to learn their community’s language.