With the emergence of multimedia applications and the upcoming age of Voice over IP (VoIP), Voice setup and resources control protocols such as SIP and H.323 over the Internet are becoming increasingly attractive applications. In the last few years as a real competitor in traditional telephony services (PSTN), SIP has gained much attention when compared with H.323. SIP works at presentation and application layer thus it mainly faces security issue at these layers. The objective of this thesis is to describe the most relevant SIP related security issues and then present security mechanisms that can be deployed to overcome the SIP security related issues. This project work demonstrates the tasks necessary to enhance the SIP security both inside and outside of the network. It is divided into three main parts, where the first part describes the SIP architecture, for example, the SIP rivals, SIP components and how a SIP system works. The second part is about some vulnerability issues of concern to SIP, study of the proposed security mechanism and also analysis on how possible threats to the SIP system such as call hijacking, message tempering and DoS attack, affect the SIP based VoIP system. The third and final part describes different steps that have been taken to avoid SIP attacks, by implementing some of the proposed security mechanisms. In order to test the SIP security, a SIP model is designed, which based on security mechanisms such as firewall, IPSec, DMZ and SIP-TLS. The results are conducted into two different scenarios. In the 1st scenario, the SIP system is tested before implementing the security measurements. In this case, the insecure system was vulnerable to several SIP attacks such as call hijacking, DOS and message tampering. In the 2nd scenario, the system is tested after the implementation of the proposed security mechanisms, where by the system now is only accessible to the authorized users and services. The tested results are also compared and discussed at the end.

This thesis focuses on enterprise VoIP telephony network security. Introduces brief comparison of old analog and digital voice networks and IP telephone networks with special focus on VoIP system security. The goal of the thesis is to identify the risks of implementation and operation of VoIP technologies in enterprise environment and so thesis brings some conclusion how to minimalize or avoid these risks. First two chapters briefly introduce the development of telephony technologies with differentiation of enterprise telephone network from public telephone networks. Further it describes individual technologies, digitalization of voice, processing the signal and VoIP protocols and components. Third chapter focuses on infrastructure of telephony networks with special interest for architecture of IP telephony and ways of establishing call processing. It describes data flows for further security risk analysis, which this technology came with. Fifth chapter is about enterprise security standards in common and is trying to describe information security management system (ISMS) adopting VoIP technology. Individual security threats and risks are described in sixth chapter, along with known methods how to avoid them. Final parts of thesis concludes of two real situation studies of threats and risks of VoIP technologies implemented in environment of small commercial enterprise and medium size enterprise, in this example represented by University of economics. These chapters conclude theoretical problems shown on practical examples.