Background: The current adherence to colorectal cancer screening (CRCS) guidelines is suboptimal. How neighborhood characteristics, e.g., racial residential segregation (RRS), area-level socioeconomic status (SES) and physician composition, affect CRCS adherence are not fully understood. We assessed associations between facility proximity to RRS areas, area-level SES, physician composition, and CRCS adherence. Methods: Data sources included 2013 Minnesota Community Measurement, 2009-2013 American Community Survey, 2012 U.S. and 2012-2013 Washington State Behavioral Risk Factor Surveillance System data, and 2013-2014 Area Health Resource File. Logistic regressions and weighted multilevel logistic regressions were used to assess the association between facility proximity to RRS areas and CRCS adherence, and association between area-level SES, physician composition and CRCS adherence, respectively. Results: Facility proximity to RRS areas was positively associated with low CRCS performance, e.g., facilities located < 2 miles away from Hispanic-segregated areas were 3 times more likely to have low CRCS performance than those at ≥5 miles away (odds ratio (OR): 2.83, 95% confidence interval (CI): 1.29, 6.24). Most area-level SES measures showed negative bivariate associations between deprivation and colonoscopy/overall adherence, and measures such as education had relatively strong associations, although few of fully-adjusted associations remained statistically significant. Further, a one-unit increase in the percentage of gastroenterologists among physicians was associated with 3% increase in the odds of colonoscopy (OR: 1.03, 95% CI: 1.01-1.04) and overall adherence (OR: 1.03, 95% CI: 1.01-1.04) in the rural-metropolitan areas. Conclusions: Developing culturally tailored CRCS programs, increasing percentage of gastroenterologists, and targeting deprived communities may improve CRCS adherence.