Artificial intelligence (AI) in the public sector is a debated topic that has the potential to enhance performance. The public sector is complex and characterized by bureaucratic challenges, posing significant procedural and financial difficulties. Using AI systems can substantially enhance the efficiency of administrative processes and the quality of services provided to citizens. However, the use of AI, in general and in the public sector, has mainly been focused on ethical issues such as trust, bias, or surveillance. One crucial factor directly affecting the efficient and ethical use of AI systems is employee competence. Employees' competence is, however, one of the least explored areas when it comes to working with AI systems, especially in the public sector. This study addresses this gap by examining the perceived necessary competences of employees within the Swedish public sector utilizing a qualitative research method and semi-structured interviews. The findings reveal three overarching areas of competences: 1) systemic competence, encompassing AI literacy and general knowledge; 2) Particular competence relating to expertise where the AI is being used; 3) Contextual competence involving data governance, legal implications, and personal data protection.