In this thesis we propose a complete solution of authentication and key exchange for digital home environments such that mobile devices can securely access the home devices. Some digital home authentication and key exchange protocols performed between mobile devices and home gateways are assisted by the AAA servers, which are provided by telecommunication companies, but they have some security flaws. In our proposed protocol, the necessary security requirements for digital home secu-rity mechanisms are satisfied, such as mutual authentication, authenticated key ex-change, and forward secrecy. In our digital home security scheme, a mobile device can authenticate his home gateway and exchange a session key with each home de-vice without pre-sharing keys with the home gateway and with the home device. The proposed authentication and key exchange protocol can also cooperate with the AAA server. Furthermore, we propose another authentication and key exchange protocol with forward secrecy between mobile devices and home devices. The computation capabilities of the mobile devices also are considered in our proposed protocols, where we only employ symmetric encryption/decryption and low-cost op-erations in order to reach the aim of light-weight computation cost.

In 1976, Diffie and Hellman proposed the concept of public key cryptosystem (PKC). The application and research of public key cryptography are arisen in the modern cryptography. In 2005, Sun, Hsieh, and Hwang (S.H.H.) proposed an electronic mail protocol based on classic public key cryptography. The technique of the Certificate of Encrypted Message Being a Signature (CEMBS) that Bao proposed in 1998 is applied on session key sharing of their e-mail system. In the same year, Dent pointed out that the first one of S.H.H.¡¦s e-mail protocols cannot suffice the property of forward secrecy. Thus, Kim, Koo, and Lee (K.K.L.) proposed two e-mail protocols based on signcryption concept which is proposed by Zheng in 1997 to overcome the flaw of S.H.H.¡¦s e-mail protocol in 2006. Lin, Lin, and Wang (L.L.W.) pointed out that the second one of S.H.H.¡¦s protocols cannot achieve forward secrecy and then they improved the protocol. In 2007, Yoon and Yoo claimed that the second one of K.K.L.¡¦s protocols is still vulnerable to two possible forgery attacks. In this thesis, we will make deep discussions about secure e-mail protocols based on PKC with providing the property of forward secrecy and then build a novel e-mail protocol to withstand the above attacks. Our proposed e-mail protocol can achieve the properties of authentication, confidentiality, integrity, non-repudiation and forward secrecy.

In recent years, the development of mobile networks is thriving or flourishing from 2G GSM, 2.5G GPRS, 3G UMTS to All-IP 4G, which integrates all heterogeneous networks and becomes mature and popular nowadays. Using mobile devices for voice transferring and multimedia sharing is also a part of our life. Mobile networks provide us an efficient way to exchange messages easily. However, these messages often contain critical personal data or private information. Transferring these messages freely in mobile network is dangerous since they can be eavesdropped easily by malicious mobile users for some illegal purposes, such as committing a crime. Hence, to avoid the exposure of the transmitted messages, robust security mechanisms are required. In this thesis, we will propose a one-round mutual authentication protocol which is computation and communication efficient and secure such that the privacy of mobile users¡¦ identities and the confidentiality of their transmitted data are guaranteed. In computation complexity, the protocol only employs symmetric encryption and hash-mac functions. Due to the possession of forward secrecy, the past encrypted messages are secure, even under the exposure of long-term keys. Furthermore, our scheme achieves the goal of user privacy and location privacy by changing TMSI in every session. Therefore, the third party cannot link two different sessions by eavesdropping the communication. Finally, our scheme also can prevent false base attacks which make use of a powerful base station to redirect mobile users¡¦ messages to a fake base station to obtain certain advantages.

It is necessary to authenticate users when they want to access services in WLANs. Extensible Authentication Protocol (EAP) is an authentication framework widely used in WLANs. Authentication mechanisms built on EAP are called EAP methods. The requirements for EAP methods in WLAN authentication have been defined in RFC 4017. Besides, low computation cost and forward secrecy, excluded in RFC 4017, are noticeable requirements in WLAN authentication. However, all EAP methods and authentication schemes designed for WLANs so far do not satisfy all of the above requirements. Therefore, we will propose an EAP method which utilizes three factors, stored secrets, passwords, and biometrics, to verify users. Our proposed method fully satisfies 1) the requirements of RFC 4017, 2) forward secrecy, and 3) lightweight computation. Moreover, the privacy of biometrics is protected against the authentication server, and the server can flexibly decide whether passwords and biometrics are verified in each round or not.

Information exchange in wireless communication without being blocked by terrain or infrastructure is easier and simpler than that in the traditional wired communication environments. Due to the transmission type, anonymity is urgently required in wireless communications for concealing the footprint of mobile users. Additionally, the mobility of a mobile device may incur possible threats to the past encrypted transmitted data, where the past session keys for the encryptions of wireless communications may be derived by the long-term secret stored the mobile device if it is lost. In this thesis, we propose an efficient solution by using symmetry-based cryptosystems for forward secrecy and anonymity in the standards of mobile networks, such as GSM, UMTS, and LTE, without losing the compatibility. By adopting secret chain (SC) based mechanism, the generation of every session key involves a short-term secret, changed in every session, to achieve forward secrecy and anonymity. Furthermore, synchronization mechanism required for the SC-protocol is also proposed. For more advanced security requirements of truly non-repudiation and strong anonymity, which is additionally anonymous to systems, certificateless signatures and group signatures are applied in the authentication protocols for UMTS and VANETs. Certificateless signatures can eliminate the overhead of using public-key infrastructure (PKI) in wireless communications. Our work proposed a certificateless signature scheme achieving the same security level of non-repudiation as that in the PKI-based signature scheme, that most of the proposed certificateless signatures cannot fulfill. Group signatures practice the privacy of the participants of the authentication protocol by originating the group signatures belonging to their group. However, directly applying group signatures in wireless communications results in inefficiency of computation when a group has a large amount of members. Therefore, we aim at reducing the computation costs of membership revocation on the proposed group signature scheme to constant without being influenced by the amount of members and then apply the scheme to VANETs and UMTS. Eventually, all the proposed schemes in the thesis are theoretically proven secure under the standard reduction.

Authentication protocols are used for two parties to authenticate each other and build a secure channel over wired or wireless public channels. However, the present standards of authentication protocols are either insufficiently secure or inefficient for light weight devices. Therefore, we propose two authentication protocols for improving the security and user efficiency in wired and wireless environments, respectively. Traditionally, TLS/SSL is the standard of authentication and key exchange protocols in wired Internet. It is known that the security of TLS/SSL is not enough due to all sorts of client side attacks. To amend the client side security, multi-factor authentication is an effective solution. However, this solution brings about the issue of biometric privacy which raises public concern of revealing biometric data to an authentication server. Therefore, we propose a truly three factor authentication protocol, where the authentication server can verify their biometric data without the knowledge of users¡¦ templates and samples. In the major wireless technologies, extensible Authentication Protocol (EAP) is an authentication framework widely used in IEEE 802.11 WLANs. Authentication mechanisms built on EAP are called EAP methods. The requirements for EAP methods in WLANs authentication have been defined in RFC 4017. To achieve user efficiency and robust security, lightweight computation and forward secrecy, excluded in RFC 4017, are desired in WLAN authentication. However, all EAP methods and authentication protocols designed for WLANs so far do not satisfy all of the above properties. We will present a complete EAP method that utilizes stored secrets and passwords to verify users so that it can (1) meet the requirements of RFC 4017, (2) provide lightweight computation, and (3) allow for forward secrecy. In order to prove our proposed protocols completely, we apply two different models to examine their security properties: Bellare¡¦s model, a standard reduction based on computational model, that reduces the security properties to the computationally hard problems and the OFMC/AVISPA tool, a model checking approach based on formal model, that uses the concept of the search tree to systematically find the weaknesses of a protocol. Through adopting Bellare¡¦s model and OFMC/AVISPA tool, the security of our work is firmly established.

Many mobile applications in wireless networks such as military battlefield, emergency response, and mobile commerce are based on the notion of secure group communications. Unlike traditional security protocols which concern security properties only, in this dissertation research we design and analyze a class of QoS-aware protocols for secure group communications in wireless networks with the goal to satisfy not only security requirements in terms of secrecy, confidentiality, authentication, availability and data integrity, but also performance requirements in terms of latency, network traffic, response time, scalability and reconfigurability. We consider two elements in the dissertation research: design and analysis. The dissertation research has three major contributions. First, we develop three "threshold-based" periodic batch rekeying protocols to reduce the network communication cost caused by rekeying operations to deal with outsider attacks. Instead of individual rekeying, i.e., performing a rekeying operation right after each group membership change event, these protocols perform batch rekeying periodically. We demonstrate that an optimal rekey interval exists that would satisfy an imposed security requirement while minimizing the network communication cost. Second, we propose and analyze QoS-aware intrusion detection protocols for secure group communications in mobile ad hoc networks to deal with insider attacks. We consider a class of intrusion detection protocols including host-based and voting-based protocols for detecting and evicting compromised nodes and examine their effect on the mean time to security failure metric versus the response time metric. Our analysis reveals that there exists an optimal intrusion detection interval under which the system lifetime metric can be best traded off for the response time performance metric, or vice versa. Furthermore, the intrusion detection interval can be dynamically adjusted based on the attacker behaviors to maximize the system lifetime while satisfying a system-imposed response time or network traffic requirement. Third, we propose and analyze a scalable and efficient region-based group key management protocol for managing mobile groups in mobile ad hoc networks. We take a region-based approach by which group members are broken into region-based subgroups, and leaders in subgroups securely communicate with each other to agree on a group key in response to membership change and member mobility events. We identify the optimal regional area size that minimizes the network communication cost while satisfying the application security requirements, allowing mobile groups to react to network partition/merge events for dynamic reconfigurability and survivability. We further investigate the effect of integrating QoS-aware intrusion detection with region-based group key management and identify combined optimal settings in terms of the optimal regional size and the optimal intrusion detection interval under which the security and performance properties of the system can be best optimized. We evaluate the merits of our proposed QoS-aware security protocols for mobile group communications through model-based mathematical analyses with extensive simulation validation. We perform thorough comparative analyses against baseline secure group communication protocols which do not consider security versus performance tradeoffs, including those based on individual rekeying, no intrusion detection, and/or no-region designs. The results obtained show that our proposed QoS-aware security protocols outperform these baseline algorithms. â / Ph. D.