The EU Directive on the accessibility requirements for products and services, Directive 2019/882, is an accessibility law that will come into effect in the European Union starting in 2025. The criteria for compliance are based on the World Wide Web Consortium’s Web Content Accessibility Guidelines, WCAG 2.1, the industry standard for accessibility on the web. There are however few empirical studies attempting to ascertain the effectiveness of adhering to the guidelines to prevent accessibility issues faced by real users. The present study utilises usability testing with users with disabilities to identify accessibility problems, to then compare whether adherence to Directive 2019/882 or WCAG 2.1 solves the errors. The results showed that the effectiveness of WCAG 2.1’s ability to prevent accessibility problems on e-commerce websites lies between 41% and 71% and that the effectiveness of Directive 2019/882 lies between 39% and 69%. Blind users, visually impaired users, and users with cognitive impairments were overrepresented when it came to encountered issues left unaddressed by Directive 2019/882 and WCAG 2.1. The results may be used to argue whether Directive 2019/882 and WCAG 2.1 are appropriate guidelines to determine accessibility on the web, which may have societal consequences. We recommend further studies testing the findings of the present study by performing similar research on different websites and different spread of disabilities among participants in the test group.