Massive MU-MIMO (Multiuser-Multiple Input and Multple Output) is a promising technology for 5G wireless communications because of its spectrum and energy efficiency. To combat the distortion from multipath fading channel, the acquisition of channel state information is essential, which generally requires the training signal that lowers the data rate. In addition, coarse quantization can reduce the high computational energy and cost, yet results in the loss of information. In this thesis, an iterative decision feedback receiver, including iterative Channel Estimation (CE) and equalization, is constructed for a Massive MU-MIMO uplink system. The impact of multipath distortion and coarse quantization can be gradually reduced due to the iterative structure that exploits extrinsic feedback to improve the CE and data detection, so that the data rate is improved by reducing training signals for CE and by using low precision quantization. To observe and evaluate the convergence behaviour, an Extrinsic Information Transfer (EXIT) chart method is utilized to visualize the performance of the iterative receiver. / Graduate

Multiple-input multiple-output (MIMO) systems have received much attention due to their multiplexing and diversity capabilities. It is possible to obtain remarkable improvement in spectral efficiency for wireless systems by using MIMO based schemes. However, sophisticated equalization and decoding structures are required for reliable communication at high rates. In this thesis, capacity achieving practical transceiver structures are proposed for MIMO wireless channels depending on the availability of channel state information at the transmitter (CSIT). First, an adaptive MIMO scheme based on the use of quantized CSIT and reduced precoding idea is proposed. With the help of a very tight analytical upper bound obtained for limited rate feedback (LRF) MIMO capacity, it is possible to construct an adaptive scheme varying the number of beamformers used according to the average SNR value. It is shown that this strategy always results in a significantly higher achievable rate than that of the schemes which does not use CSIT, if the number of transmit antennas is greater than that of receive antennas. Secondly, it is known that the use of CSIT does not bring significant improvement over capacity, when similar number of transmit and receive antennas are used / on the other hand, it reduces the complexity of demodulation at the receiver by converting the channel into noninterfering subchannels. However, it is shown in this thesis that it is still possible to achieve a performance very close to the outage probability and exploit the space-frequency diversity benefits of the wireless fading channel without compromising the receiver complexity, even if the CSIT is not used. The proposed receiver structure is based on iterative forward and backward filtering to suppress the interference both in time and space followed by a spacetime decoder. The rotation of multidimensional constellations for block fading channels and the single-carrier frequency domain equalization (SC-FDE) technique for wideband MIMO channels are studied as example applications.