In this thesis, a novel iterative receiver and its improved version are proposed for relay-assisted multiuser communications, in which multiple users transmit to a destination with the help of a relay and using fast frequency-hopping modulation. Each user employs a channel encoder to protect its information and facilitate interference cancellation at the receiver. The signal received at the relay is either amplified, or partially decoded with a simple energy detector, before being forwarded to the destination. Under flat Rayleigh fading channels, the receiver at the destination can be implemented non-coherently, i.e., it does not require the instantaneous channel information to demodulate the users’ transmitted signals. The proposed iterative algorithm at the destination exploits the soft outputs of the channel decoders to successively extract the maximum likelihood symbols of the users and perform interference cancellation. The iterative method is successfully applied for both cases of amplify-and-forward and partial decode-and-forward relaying. The error performance of the proposed iterative receiver is investigated by computer simulation. Under the same spectral efficiency, simulation results demonstrate the excellent performance of the proposed receiver when compared to the performance of decoding without interference cancellation as well as the performance of the maximum likelihood multiuser detection previously developed for uncoded transmission. Simulation results also suggest that a proper selection of channel coding schemes can help to support significant more users without consuming extra system resources. In addition, to further enhance the receiver’s performance in terms of the bit error rate, an improved version of the iterative receiver is presented. Such an improved receiver invokes inner-loop iterations between the channel decoders and the demappers in such a way that the soft outputs of the channel decoders are also used to refine the outputs of the demappers for every outer-loop iteration. Simulation results indicate a performance gain of about 2.5dB by using the two-loop receiver when compared to the performance of the first proposed receiver.

Evolution of wireless services enabled the development of the advanced applications and shifted the paradigms of research in this field from voice to data centric. Such services are spreading like wildfire between users and hence, increasing the demand for large bandwidth. However, the frequency spectrum that is suitable for wireless mobile communications is already assigned to particular services from 400 MHz to several GHz. Also, allocating a large chunk of band continuously from the same part of the spectrum may not be possible due to spectral crowd. Therefore, meeting the demand for high data rate requiring wireless services within the accessible spectrum range becomes a challenging problem. The spectrum allocation policies are discussed by regulatory authorities and academia, and the idea of spectrum sharing systems are addressed as a solution. For instance, heterogeneous networks (HetNets) increase the number of available resources and improve the spectrum accessing capabilities of the wireless communication systems. To achieve this, HetNet nodes are deployed within the coverage of the macrocell regions. Thus, spectral efficiency is boosted via spatial reuse of the same spectral resources. On the contrary, HetNets preclude to fully exploit the resources because of serious interference problems between macrocell and HetNet nodes. Thus, wireless networks of the future will observe interference from even a larger number of sources. Due to co-channel HetNet deployment and denser frequency reuse, interference cancellation is expected to have significant importance for future wireless communication systems. The occupied resources can also be reused as a solution by conducting advanced signal processing algorithms at the receiver to increase the spectral efficiency. While doing so, the proposed approaches are expected to be easily integrated with the existing complementary approaches to improve the capacity further. Besides, new deployment strategies that allow spectrum access for non-licensed users to achieve larger bandwidth become important to increase the spectral efficiency of the HetNets. Within the scope of the dissertation, new solutions are developed for the aforementioned problems of the next-generation wireless communication systems. First, an interference cancellation receiver that exploits the unique characteristics of current waveforms is developed in Chapter 2. Also the unknown model of interference is converted to a known model and new algorithms are proposed to recover the desired signal. Then, another perspective is brought into the subject by transforming the interference problem to an interference advantage in Chapter 3. The idea of co-existence of different types of signals are analyzed to bring another degree of freedom as a solution. The proposed approaches are integrated to the existing complementary approaches, such as interference coordination and power control, to improve the capacity further. Finally, a cooperation mechanism is suggested to facilitate the transmission of signal which has a large bandwidth by integrating the idle bands in Chapter 4. By this way, geo-spatially idle bands within the coverage area are utilized and spectral efficiency is increased.