The cognitive radio network (CRN) concept has been proposed as a solution to the growing demand and underutilization of the radio spectrum. To improve the radio spectrum utilization, CRN technology allows the coexistence of licensed and unlicensed systems over the same spectrum. In an underlay spectrum sharing system, secondary users (SUs) transmit simultaneously with the primary users (PUs) in the same frequency band given that the interference caused by the SU to the PU remains below a tolerable interference limit. Besides the transmission power limitation, a secondary network is subject to distinct channel impairments such as fading and interference from the primary transmissions. Also, CRNs face new security threats and challenges due to their unique cognitive characteristics.This thesis analyzes the performance of underlay CRNs and underlay cognitive relay networks under spectrum sharing constraints and security constraints. Distinct SU transmit power policies are obtained considering various interference constraints such as PU outage constraint or PU peak interference power constraint. The thesis is divided into an introduction and two research parts based on peer-reviewed publications. The introduction provides an overview of radio spectrum management, basic concepts of CRNs, and physical layer security. In the first research part, we study the performance of underlay CRNs with emphasis on a multiuser environment.In Part I-A, we consider a secondary network with delay-tolerant applications and analyze the ergodic capacity. Part I-B analyzes the secondary outage capacity which characterises the maximum data rate that can be achieved over a channel for a given outage probability. In Part I-C, we consider a secondary network with delay constrained applications, and derive expressions of the outage probability and delay-limited throughput. Part I-D presents a queueing model that provides an analytical tool to evaluate the secondary packet-level performance with multiple classes of traffic considering general interarrival and service time distributions. Analytical expressions of the SU average packet transmission time, waiting time in the queue, andtime spent in the system are provided.In the second research part, we analyze the physical layer security for underlay CRNs and underlay cognitive relay networks. Analytical expressions of the probability of non-zero secrecy capacity and secrecy outage probability are derived.Part II-A considers a single hop underlay CRN in the presence of multiple eavesdroppers (EAVs) and multiple SU-Rxs. In Part II-B, an underlay cognitive relay network in the presence of multiple secondary relays and multiple EAVs is studied.Numerical examples illustrate that it is possible to exploit the physical layer characteristics to achieve both security and quality of service in CRNs while satisfying spectrum sharing constraints.

In underlay cognitive radio networks, secondary users can share the spectrum with primary users as long as the interference caused by the secondary users to primary users is below a certain predetermined threshold. It is reasonable to assume that there is always a large pool of secondary users trying to access the channel, which can be occupied by only one secondary user at a given time. As a result, a multi-user scheduling problem arises among the secondary users. In this thesis, by manipulating basic schemes based on selective multi-user diversity, normalized thresholding, transmission power control, and opportunistic round robin, we propose and analyze eight scheduling schemes of secondary users in an underlay cognitive radio set-up. The system performance of these schemes is quantified by using various performance metrics such as the average system capacity, normalized average feedback load, scheduling outage probability, and system fairness of access. In our proposed schemes, the best user out of all the secondary users in the system is picked to transmit at each given time slot in order to maximize the average system capacity. Two thresholds are used in the two rounds of the selection process to determine the best user. The first threshold is raised by the power constraint from the primary user. The second threshold, which can be adjusted by us, is introduced to reduce the feedback load. The overall system performance is therefore dependent on the choice of these two thresholds and the number of users in the system given the channel conditions for all the users. In this thesis, by deriving analytical formulas and presenting numerical examples, we try to provide insights of the relationship between the performance metrics and the involved parameters including two selection thresholds and the number of active users in the system, in an effort to maximize the average system capacity as well as satisfy the requirements of scheduling outage probability and feedback load.

Relay-assisted cooperative communication exploits spatial diversity to combat wireless fading, and is an appealing technology for next generation wireless systems. Several relay cooperation protocols have been proposed in the literature. In amplify-and-forward (AF)relaying, which is the focus of this thesis, the relay amplifies the signal it receives from the source and forwards it to the destination. AF has been extensively studied in the literature on account of its simplicity since the relay does not need to decode the received signal. We propose a novel optimal relaying policy for two-hop AF cooperative relay systems. In this, an average power-constrained relay adapts its gain and transmit power to minimize the fading-averaged symbol error probability (SEP) at the destination. Next, we consider a generalization of the above policy in which the relay operates as an underlay cognitive radio (CR). This mode of communication is relevant because it promises to address the spectrum shortage constraint. Here, the relay adapts its gain as a function of its local channel gain to the source and destination and also the primary such that the average interference it causes to the primary receiver is also constrained. For both the above policies, we also present near-optimal, simpler relay gain adaptation policies that are easy to implement and that provide insights about the optimal policies. The SEPs and diversity order of the policies are analyzed to quantify their performance. These policies generalize the conventional fixed-power and fixed-gain AF relaying policies considered in cooperative and CR literature, and outperform them by 2.0-7.7 dB. This translates into significant energy savings at the source and relay, and motivates their use in next generation wireless systems.