Complex behaviors require selectively attending to task-relevant items, and ignoring conspicuous, irrelevant items. For example, driving requires selectively attending to other cars on the road while ignoring flashing billboards. Dominant models of attentional control posit that we avoid distraction by biasing attention towards task-relevant items, and our ability to avoid distraction depends on the strength and specificity of this bias. I find that a strong, specific bias towards task-relevant items is insufficient for preventing distraction. Instead, preventing distraction also requires past experience ignoring distractors. I also find that long-term memory systems, rather than visual short-term memory or priming memory systems, maintain this experience. Based upon these findings, I propose that effective attentional control not only demands a strong, specific bias towards task-relevant items, but also requires that observers learn to ignore conspicuous, irrelevant items.