Drones are unmanned aerial vehicles (UAVs) operated remotely with the help of cameras, GPS, and on-device SD cards. These are used for many applications including civilian as well as military. On the other hand, drone swarms are a fleet of drones that work together to achieve a special goal through swarm intelligence approaches. These provide a lot of advantages such as better coverage, accuracy, increased safety, and improved flexibility when compared to a single drone. However, the deployment of such swarms in an adversarial environment poses significant challenges. This work provides an overview of the current state of research on drone swarms in adversarial environments including algorithms for swarming formation of robotic attack drones with their strengths and weaknesses as well as the attack strategies used by attackers. This work also outlines the common adversarial counter-attack methods to disrupt drone attacks consisting of detection and destruction of drone swarms along with their drawbacks, a counter UAV defense system, and splitting large-scale drones into unconnected clusters. After identifying several challenges, an optimized algorithm is proposed to split the large-scale drone swarms more efficiently.