The Domain Name System is a worldwide global service, considered to be the heart and soul of the internet, that is used for mapping IP addresses to a hostname and vice-versa. Despite the fact that DNS is recognized as a critical internet service, the security aspects concerning its adoption are still highly neglected. This thesis presents the foundations of DNS, investigates vulnerabilities, and enumeration techniques, which are used to locate all DNS servers and records of an organization. In particular, we investigated how attackers can enumerate DNS using an actual data set available for .se and .nu zone files. We analyze such data sets and map their corresponding vulnerabilities to common DNS attacks found in the literature. We show that available information can be exploited to perform security attacks on the DNS infrastructure.

Networks not employing destination-side source address validation (DSAV) expose themselves to a class of pernicious attacks which could be prevented by filtering inbound traffic purporting to originate from within the network. In this work, we survey the pervasiveness of networks vulnerable to infiltration using spoofed addresses internal to the network. We issue recursive Domain Name System (DNS) queries to a large set of known DNS servers world-wide using various spoofed-source addresses. In late 2019, we found that 49% of the autonomous systems we tested lacked DSAV. After a large-scale notification campaign run in late 2020, we repeated our measurements in early 2021 and found that 44% of ASes lacked DSAV--though importantly, as this is an observational study, we cannot conclude causality. As case studies illustrating the dangers of a lack of DSAV, we measure susceptibility of DNS resolvers to cache poisoning attacks and the NXNS attack, two attacks whose attack surface is significantly reduced when DSAV in place. We discover 309K resolvers vulnerable to the NXNS attack and 4K resolvers vulnerable to cache poisoning attacks, 70% and 59% of which would have been protected had DSAV been in place.

DNS over HTTPS (DoH) is a protocol used to send traditional DNS traffic over HTTPS. This causes the DNS name resolving traffic to be encrypted and transmitted over the same port as regular HTTPS traffic. This thesis maps a number of previous vulnerabilities in DNS and compares those risks with the DoH protocol and its implementation, mainly focusing on cache poisoning. A number of attacks from a DoH server to a DoH client are applied. The results show that it is possible to inject incorrect data into the DoH client’s cache. The consequences of this can be extensive, an example of this is a redirect to a malicious webpage, which when using DoH can be difficult to detect because the DNS traffic is encrypted. Further work is needed to mitigate the security holes discovered, as well as to further identify potential threats.

The work analyze current detection methods of anomalies and security incidents in DNS traffic, and than design new obfuscation techniques which are capable of evading anomaly detection. Network attacks, exploiting the DNS protocol for tunneling of other network traffic, were selected for implementation part of the work. Control of botnet is considered as malicious application of tunneling through the DNS protocol. The main result of the work is to emphasize the necessity of discovering new detection principles of anomalies and security incidents in DNS traffic.