The search for the closest lattice point arises in many communication problems, and is known to be NP-hard. The Maximum Likelihood (ML) Detector is the optimal detector which yields an optimal solution to this problem, but at the expense of high computational complexity. Existing near-optimal methods used to solve the problem are based on the Sphere Decoder (SD), which searches for lattice points confined in a hyper-sphere around the received point. The SD has emerged as a powerful means of finding the solution to the ML detection problem for MIMO systems. However the bottleneck lies in the determination of the initial radius. This thesis is concerned with the detection of transmitted wireless signals in Multiple-Input Multiple-Output (MIMO) digital communication systems as efficiently and effectively as possible. The main objective of this thesis is to design efficient ML detection algorithms for MIMO systems based on the depth-first search (DFS) algorithms whilst taking into account complexity and bit error rate performance requirements for advanced digital communication systems. The increased capacity and improved link reliability of MIMO systems without sacrificing bandwidth efficiency and transmit power will serve as the key motivation behind the study of MIMO detection schemes. The fundamental principles behind MIMO systems are explored in Chapter 2. A generic framework for linear and non-linear tree search based detection schemes is then presented Chapter 3. This paves way for different methods of improving the achievable performance-complexity trade-off for all SD-based detection algorithms. The suboptimal detection schemes, in particular the Minimum Mean Squared Error-Successive Interference Cancellation (MMSE-SIC), will also serve as pre-processing as well as comparison techniques whilst channel capacity approaching Low Density Parity Check (LDPC) codes will be employed to evaluate the performance of the proposed SD. Numerical and simulation results show that non-linear detection schemes yield better performance compared to linear detection schemes, however, at the expense of a slight increase in complexity. The first contribution in this thesis is the design of a near ML-achieving SD algorithm for MIMO digital communication systems that reduces the number of search operations within the sphere-constrained search space at reduced detection complexity in Chapter 4. In this design, the distance between the ML estimate and the received signal is used to control the lower and upper bound radii of the proposed SD to prevent NP-complete problems. The detection method is based on the DFS algorithm and the Successive Interference Cancellation (SIC). The SIC ensures that the effects of dominant signals are effectively removed. Simulation results presented in this thesis show that by employing pre-processing detection schemes, the complexity of the proposed SD can be significantly reduced, though at marginal performance penalty. The second contribution is the determination of the initial sphere radius in Chapter 5. The new initial radius proposed in this thesis is based on the variable parameter α which is commonly based on experience and is chosen to ensure that at least a lattice point exists inside the sphere with high probability. Using the variable parameter α, a new noise covariance matrix which incorporates the number of transmit antennas, the energy of the transmitted symbols and the channel matrix is defined. The new covariance matrix is then incorporated into the EMMSE model to generate an improved EMMSE estimate. The EMMSE radius is finally found by computing the distance between the sphere centre and the improved EMMSE estimate. This distance can be fine-tuned by varying the variable parameter α. The beauty of the proposed method is that it reduces the complexity of the preprocessing step of the EMMSE to that of the Zero-Forcing (ZF) detector without significant performance degradation of the SD, particularly at low Signal-to-Noise Ratios (SNR). More specifically, it will be shown through simulation results that using the EMMSE preprocessing step will substantially improve performance whenever the complexity of the tree search is fixed or upper bounded. The final contribution is the design of the LRAD-MMSE-SIC based SD detection scheme which introduces a trade-off between performance and increased computational complexity in Chapter 6. The Lenstra-Lenstra-Lovasz (LLL) algorithm will be utilised to orthogonalise the channel matrix H to a new near orthogonal channel matrix H ̅.The increased computational complexity introduced by the LLL algorithm will be significantly decreased by employing sorted QR decomposition of the transformed channel H ̅ into a unitary matrix and an upper triangular matrix which retains the property of the channel matrix. The SIC algorithm will ensure that the interference due to dominant signals will be minimised while the LDPC will effectively stop the propagation of errors within the entire system. Through simulations, it will be demonstrated that the proposed detector still approaches the ML performance while requiring much lower complexity compared to the conventional SD.

The research reported in this thesis is concerned with robust precoder and transceiver optimization in multiuser multi-antenna wireless communication systems in the presence of imperfect channel state information(CSI). Precoding at the transmit side, which utilizes the CSI, can improve the system performance and simplify the receiver design. Transmit precoding is essential for inter-user interference cancellation in multiuser downlink where users do not cooperate. Linear and non-linear precoding have been widely investigated as low-complexity alternatives to dirty paper coding-based transmission scheme for multiuser multiple-input multiple-output(MU-MIMO)downlink. Similarly, in relay-assisted networks, precoding at the relay nodes have been shown to improve performance. The precoder and joint precoder/receive filter (transceiver) designs usually assume perfect knowledge of the CSI. In practical systems, however, the CSI will be imperfect due to estimation errors, feedback errors and feedback delays. Such imperfections in CSI will lead to deterioration of performance of the precoders/transceivers designed assuming perfect CSI. In such situations, designs which are robust to CSI errors are crucial to realize the potential of multiuser multi-antenna systems in practice. This thesis focuses on the robust designs of precoders and transceivers for MU-MIMO downlink, and for non-regenerative relay networks in the presence of errors in the CSI. We consider a norm-bounded error(NBE) model, and a stochastic error(SE) model for the CSI errors. These models are suitable for commonly encountered errors, and they allow mathematically and computationally tractable formulations for the robust designs. We adopt a statistically robust design in the case of stochastic error, and a minimax or worst-case robust design in the case of norm-bounded error. We have considered the robust precoder and transceiver designs under different performance criteria based on transmit power and quality-of-service(QoS) constraints. The work reported in this thesis can be grouped into three parts, namely,i ) robust linear pre-coder and transceiver designs for multiuser downlink, ii)robust non-linear precoder and transceiver designs for multiuser downlink, and iii)robust precoder designs for non-regenerative relay networks. Linear precoding: In this part, first, a robust precoder for multiuser multiple-input single-output(MU-MISO)downlink that minimizes the total base station(BS)transmit power with constraints on signal-to-interference-plus-noise ratio(SINR) at the user terminals is considered. We show that this problem can be reformulated as a second order cone program(SOCP) with the same order of computational complexity as that of the non-robust precoder design. Next, a robust design of linear transceiver for MU-MIMO downlink is developed. This design is based on the minimization of sum mean square error(SMSE) with a constraint on the total BS transmit power, and assumes that the error in the CSI at the transmitter(CSIT) follows the stochastic error model. For this design, an iterative algorithm based on the associated Karush-Kuhn-Tucker(KKT) conditions is proposed. Our numerical results demonstrate the robust performance of the propose designs. Non-linear precoding: In this part, we consider robust designs of Tomlinson-Harashima precoders(THP) for MU-MISO and MU-MIMO downlinks with different performance criteria and CSI error models. For MU-MISO systems with imperfect CSIT, we investigate the problem of designing robust THPs under MSE and total BS transmit power constraints. The first design is based on the minimization of total BS transmit power under constraints on the MSE at the individual user receivers. We present an iterative procedure to solve this problem, where each iteration involves the solution of a pair of convex optimization problems. The second design is based on the minimization of a stochastic function of the SMSE under a constraint on the total BS transmit power. We solve this problem efficiently by the method of alternating optimization. For MU-MIMO downlink, we propose robust THP transceiver designs that jointly optimize the TH precoder and receiver filters. We consider these transceiver designs under stochastic and norm-bounded error models for CSIT. For the SE model, we propose a minimum SMSE transceiver design. For the NBE model, we propose three robust designs, namely, minimum SMSE design, MSE-constrained design, and MSE-balancing design. Our proposed solutions to these robust design problems are based on iteratively solving a pair of sub-problems, one of which can be solved analytically, and the other can be formulated as a convex optimization problem that can be solved efficiently. Robust precoder designs for non-regenerative relay networks: In this part, we consider robust designs for two scenarios in the case of relay-assisted networks. First, we consider a non-regenerative relay network with a source-destination node pair assisted by multiple relay nodes, where each node is equipped with a single antenna. The set of the cooperating relay nodes can be considered as a distributed antenna array. For this scenario, we present a robust distributed beam former design that minimizes the total relay transmit power with a constraint on the SNR at the destination node. We show that this robust design problem can be reformulated as a semi-definite program (SDP)that can be solved efficiently. Next, we consider a non-regenerative relay network, where a set of source-destination node pairs are assisted by a MIMO-relay node, which is equipped with multiple transmit and multiple receive antennas. For this case, we consider robust designs in the presence of stochastic and norm-bounded CSI errors. We show that these problems can be reformulated as convex optimization problems. In the case of norm-bounded error, we use an approximate expression for the MSE in order to obtain a tractable solution.

Στο πλαίσιο αυτής της διατριβής μελετάται η επίδοση συστημάτων διαφορισμού MIMO σε γενικευμένα κανάλια διαλείψεων. Αρχικά, εξετάζεται η επίδοση των OSTBC σε περιβάλλοντα διαλείψεων Hoyt. Αποδεικνύεται ότι, στην περίπτωση τέτοιων συστημάτων, ο σηματοθορυβικός λόγος (signal to noise ratio, SNR) εκφράζεται ως μία τετραγωνική μορφή κανονικών τυχαίων μεταβλητών και γίνεται χρήση της συνάρτησης πυκνότητας πιθανότητας και της αθροιστικής συνάρτησης κατανομής αυτής της μορφής για τον υπολογισμό των μετρικών επίδοσης. Επιπλέον, μελετάται η σύγκλιση των σειρών που χρησιμοποιούνται για τον υπολογισμό των δύο αυτών συναρτήσεων και κατασκευάζονται νέα άνω φράγματα για το σφάλμα αποκοπής των σειρών. Τα φράγματα αυτά είναι σαφώς πιο αυστηρά από τα ήδη γνωστά από τη βιβλιογραφία. Στη συνέχεια, εισάγεται ένα γενικευμένο μοντέλο διαλείψεων για την ανάλυση επίδοσης των OSTBC και των δεκτών MRC και υπολογίζονται όλες οι μετρικές επίδοσης των δύο συστημάτων για το συγκεκριμένο μοντέλο διαλείψεων. Το μοντέλο αυτό περιλαμβάνει ως ειδικές περιπτώσεις τα πλέον διαδεδομένα μοντέλα καναλιών διαλείψεων, ενώ επιπλέον, επιτρέπει την ανάλυση επίδοσης σε μικτά περιβάλλοντα διαλείψεων όπου τα πολλαπλά κανάλια μπορούν να ακολουθούν διαφορετικές κατανομές. Στη συνέχεια, μελετάται η επίδοση συστημάτων συνεργατικού διαφορισμού με χρήση αναμεταδοτών ανίχνευσης και προώθησης (Detect and Forward, DaF) σε περιβάλλοντα διαλείψεων Rayleigh. Εξετάζονται τρεις διαφορετικοί δέκτες και υπολογίζεται η πιθανότητα σφάλματος ανά bit γι' αυτούς. Τέλος προτείνεται ένας νέος δέκτης για συνεργατικά συστήματα DaF και αποδεικνύεται η ανωτερότητά του σε σύγκριση με τους υπόλοιπους μελετώμενους δέκτες. Όλα τα θεωρητικά αποτελέσματα που παρουσιάζονται στο πλαίσιο της διατριβής συγκρίνονται με αποτελέσματα προσομοιώσεων Monte Carlo που αποδεικνύουν την ορθότητα της ανάλυσης. / This thesis studies the performance of MIMO diversity systems in generalized fading channels. First, we examine the performance of OSTBC in Hoyt fading channels. It is proven that, for this fading model, and when an OSTBC is employed, the signal-to-noise ratio (SNR) of the OSTBC can be expressed as a quadratic form in normal random variables. Therefore, the performance analysis for OSTBC over Hoyt fading channels is performed using the PDF and the CDF of such quadratic forms. In the statistical literature, these functions are expressed in terms of infinite series. The convergence of the series is thoroughly studied and new expressions for the truncation error bound of these series are proposed. The proposed bounds are much tighter than the bounds that can be found in the literature. The expressions for the PDF and the CDF are then used for the performance analysis of OSTBC over Hoyt fading and several performance metrics are calculated. Then, a generalized fading model for the performance analysis of OSTBC and MRC is proposed and the theoretical performance analysis of both MRC and OSTBC is carried out. The main advantage of this model is the fact that it includes as special cases most of the widely used fading models. Furthermore, the performance of cooperative diversity systems employing Detect and Forward (DaF) relays is studied for Rayleigh fading channels. More specifically, three low complexity detection algorithms for these channels are examined and closed-form expressions of the bit error probability (BEP) for these receivers are derived. Finally, a new low complexity receiver for cooperative systems with DaF relays is proposed. Using Monte Carlo Simulations it is shown that this receiver outperforms the three receivers that have been studied. For the systems studied in the thesis, the performance analysis results that have been derived theoretically are compared with Monte Carlo simulations that prove the validity of the analysis.

Τα μελλοντικά ασύρματα δίκτυα και συστήματα επικοινωνιών αναμένεται να παρέχουν αξιόπιστα υπηρεσίες δεδομένων με απαιτήσεις ρυθμού μετάδοσης δεδομένων οι οποίες κυμαίνονται από λίγα kbps μέχρι μερικά Mbps και εξαιτίας του υψηλού κόστους του φάσματος συχνοτήτων, αυτά τα συστήματα χρειάζεται να είναι εξαιρετικά αποτελεσματικά όσον αφορά στη χρησιμοποίηση του φάσματος. Συγκεκριμένα, η εφαρμογή τεχνικών μετάδοσης δεδομένων οι οποίες βασίζονται σε MIMO και OFDMA θεωρείται ως μια πολλά υποσχόμενη λύση για να ικανοποιήσει αυτές τις απαιτήσεις. Από την άλλη μεριά, τα συστήματα MIMO-OFDMA είναι εύκαμπτα και φασματικά αποτελεσματικά αλλά ο αξιοσημείωτα μεγάλος αριθμός υποφορέων και ο συνυπολογισμός της διάστασης χώρου καθιστούν την κατανομή ραδιοπόρων πολύ πολύπλοκη. Στην πραγματικότητα, η βέλτιστη κατανομή ραδιοπόρων η οποία μεγιστοποιεί το συνολικό ρυθμό μετάδοσης δεδομένων των χρηστών είναι συχνά πάρα πολύ πολύπλοκη για πρακτικές εφαρμογές. Συνεπώς, απαιτούνται υποβέλτιστες σχετικά αποτελεσματικές και χαμηλής πολυπλοκότητας στρατηγικές κατανομής ραδιοπόρων ώστε να κατανείμουν τους ραδιοπόρους συχνότητας, ισχύος και χώρου του συστήματος στους χρήστες του συστήματος. Η παρούσα ΔΔ διαπραγματεύεται στρατηγικές κατανομής ραδιοπόρων στην κατερχόμενη και στην ανερχόμενη ζεύξη συστημάτων OFDMA, στην κατερχόμενη ζεύξη συστημάτων MISO-OFDMA και στην κατερχόμενη ζεύξη συστημάτων MIMO-OFDMA στοχεύοντας στη μεγιστοποίηση του συνολικού ρυθμού μετάδοσης δεδομένων των χρηστών εγγυώντας οι ρυθμοί μετάδοσης δεδομένων των χρηστών να τηρούν μια προκαθορισμένη αναλογία μεταξύ τους ή να ξεπερνούν προκαθορισμένους ελάχιστους ρυθμούς μετάδοσης δεδομένων. Στο πλαίσιο της επίλυσης του προβλήματος της μεγιστοποίησης του συνολικού ρυθμού μετάδοσης δεδομένων των χρηστών με ανεκτή πολυπλοκότητα για κάθε μία από τις προαναφερθείσες περιπτώσεις, προτείνονται νέοι υποβέλτιστοι αλγόριθμοι. Στην κατερχόμενη ζεύξη των συστημάτων SISO, στόχος είναι η μεγιστοποίση του συνολικού ρυθμού μετάδοσης δεδομένων των χρηστών με περιορισμό στη συνολική διαθέσιμη ισχύ και με αναλογικούς ρυθμούς μετάδοσης δεδομένων μεταξύ των χρηστών. Η προτεινόμενη μέθοδος, η οποία είναι αποτελεσματική όσον αφορά στην πολυπλοκότητα, αποτελείται από τρεις αλγόριθμους: έναν αλγόριθμο ο οποίος προσδιορίζει τον αριθμό των υποφορέων για κάθε χρήστη, έναν αλγόριθμο κατανομής υποφορέων διαιρώντας τους χρήστες σε δύο ομάδες και τον αλγόριθμο water-filling. Οι πρώτοι δύο αλγόριθμοι αναθέτουν τους διαθέσιμους υποφορείς στους χρήστες του συστήματος και ο τρίτος αλγόριθμος κατανέμει τη διαθέσιμη ισχύ με βέλτιστο τρόπο για μεγιστοποίηση του συνολικού ρυθμού μετάδοσης δεδομένων. Στην ανερχόμενη ζεύξη των συστημάτων SISO, στόχος είναι η μεγιστοποίηση του συνολικού ρυθμού μετάδοσης δεδομένων των χρηστών με περιορισμό στην ισχύ κάθε χρήστη και σε ελάχιστους ρυθμούς μετάδοσης δεδομένων μεταξύ των χρηστών. Η προτεινόμενη τεχνική, η οποία είναι αποτελεσματική όσον αφορά στην πολυπλοκότητα, αποτελείται από τρεις αλγόριθμους: έναν αλγόριθμο ο οποίος προσδιορίζει τον αριθμό των υποφορέων για κάθε χρήστη, έναν αλγόριθμο κατανομής υποφορέων διαιρώντας τους χρήστες σε δύο ομάδες και τον αλγόριθμο water-filling. Οι πρώτοι δύο αλγόριθμοι αναθέτουν τους διαθέσιμους υποφορείς στους χρήστες του συστήματος και ο τρίτος αλγόριθμος κατανέμει τη διαθέσιμη ισχύ. Στην κατερχόμενη ζεύξη των συστημάτων MISO αναπτύσσονται τρεις αλγόριθμοι επιλογής χρηστών και κατανομής πόρων για πολυχρηστικά συστήματα κατερχόμενης ζεύξης οι οποίοι είναι λιγότερο πολύπλοκοι από άλλες προσεγγίσεις και ενσωματώνουν τη δικαιοσύνη. Στους πρώτους δύο αλγόριθμους επιβάλλονται αναλογικοί περιορισμοί μεταξύ των ρυθμών μετάδοσης δεδομένων των χρηστών και στον τρίτο αλγόριθμο περιορισμοί στους ελάχιστους ρυθμούς μετάδοσης δεδομένων λαμβάνονται υπόψη. Επίσης, πραγματοποιείται επέκταση του αλγόριθμου μεγιστοποίησης του συνολικού ρυθμού μετάδοσης δεδομένων με αναλογικούς περιορισμούς δικαιοσύνης σε ΣΚΚ και για μείωση της πολυπλοκότητας οι υποφορείς ομαδοποιούνται σε τεμάχια. Τα αποτελέσματα της προσομοίωσης επιβεβαιώνουν την αποτελεσματικότητα τους στη διανομή του συνολικού ρυθμού μετάδοσης δεδομένων δίκαια μεταξύ των χρηστών αλλά και ότι σε ΣΚΚ επιτυγχάνονται μεγαλύτεροι συνολικοί ρυθμοί μετάδοσης δεδομένων. Τέλος, στην κατερχόμενη ζεύξη των συστημάτων MIMO, το πρόβλημα διατυπώνεται με στόχο τη μεγιστοποίηση του συνολικού ρυθμού μετάδοσης δεδομένων των χρηστών με περιορισμό στη συνολική διαθέσιμη ισχύ και ελέγξιμο εύρος ζώνης στο σύστημα εισάγοντας την παράμετρο α. Αφού αυτό το πρόβλημα βελτιστοποίησης πρέπει να εκτελεστεί σε πραγματικό χρόνο, προτείνεται ένας αλγόριθμος αποδοτικός, υποβέλτιστος και αποτελεματικός όσον αφορά στην πολυπλοκότητα ο οποίος παρουσιάζει λογική απώλεια όσον αφορά στην περίπτωση χωρίς περιορισμούς όπου ο μόνος στόχος είναι η μεγιστοποίηση του συνολικού ρυθμού μετάδοσης δεδομένων και εντυπωσιακό όφελος συγκρινόμενος με τη στατική τεχνική TDMA. Πέραν της θεωρητικής ανάλυσης των παραπάνω αλγόριθμων, ο προσομοιωτικός κώδικας που δημιουργήθηκε βασισμένος σε ρεαλιστικές υποθέσεις και απλουστεύσεις, μάς έδωσε τα αποτελέσματα εκείνα τα οποία μετρούν το συνολικό ρυθμό μετάδοσης δεδομένων των χρηστών ο οποίος παρέχεται από κάθε έναν από τους προαναφερθέντες αλγόριθμους και εξετάζουν την πιθανή καταλληλότητα για χρήση τους σε συγκεκριμένα περιβάλλοντα. Τα τελικά συμπεράσματα είναι ότι τα συστήματα MIMO-OFDMA είναι ικανά να προσφέρουν πραγματικές ευρυζωνικές υπηρεσίες πάνω από το ασύρματο κανάλι επικοινωνίας. / Future wireless communication networks and systems are expected to reliably provide data services with data rate requirements ranging from a few kbps up to some Mbps and, due to the high costs of frequency spectrum, these systems also need to be extremely efficient in terms of the spectrum usage. In particular, the application of transmission schemes based on OFDMA and on MIMO is considered as a promising solution to meet these requirements. On the one hand, MIMO-OFDMA systems are flexible and spectrally efficient but the considerably large number of subcarriers and the inclusion of the space dimension make the RRA in such systems very complex. In fact, the optimum RRA that maximizes the sum of the users' data rates is often too complex for practical application. Consequently, suboptimal rather efficient and low-complexity RRA strategies are required in order to allocate the frequency, power, and space radio resources of the system to the users of the system. This doctoral thesis deals with RRA strategies in the downlink and uplink of OFDMA systems, the downlink of MISO-OFDMA systems, and the downlink of MIMO-OFDMA systems aiming at the maximization of the sum of the users' data rates guaranteeing proportional data rates or minimum data rates among users. In order to solve the problem of maximizing the sum of the users' data rates with affordable complexity in each one of the aforementioned cases, new suboptimal algorithms are proposed. In the SISO downlink the objective is to maximize the sum of the users' data rates subject to constraints on the total available power and proportional data rates among users. The proposed method, which is also complexity effective, consists of three algorithms; an algorithm that determines the number of subcarriers for each user, a subcarrier allocation algorithm by dividing the users in two groups and the water-filling algorithm. The first two algorithms assign the available subcarriers to the users of the system and the third one allocates the available power optimally in order to maximize the sum of the users' data rates. In the SISO uplink the objective is to maximize the sum of the users' data rates subject to constraints on per user power and minimum data rates among users. The proposed scheme, which is also complexity effective, consists of three algorithms; an algorithm that determines the number of subcarriers for each user, a subcarrier allocation algorithm by dividing the users in two groups and the water-filling algorithm. The first two algorithms assign the available subcarriers to the users of the system and the third one allocates the available power. In the MISO downlink three user selection and resource allocation algorithms for multiuser downlink systems are developed that are less complex than other approaches and incorporate fairness. In the first two algorithms proportional constraints among the users' data rates are imposed and in the third algorithm minimum data rate constraints are taken into account. The proposed algorithm that maximizes the sum of the users' data rates with proportional data rate constraints is also applied to DAS and subcarriers are grouped to chunks. Simulation results sustain their effectiveness in distributing the sum data rate fairly and flexibly among users and that in DAS higher sum of the users' data rates are obtained. Finally, in the MIMO downlink the problem is formulated in order to maximize the sum of the users' data rates subject to total available power constraint with controllable bandwidth introducing system parameter α. Since this optimization should be performed in real time, an efficient, suboptimal and complexity effective algorithm is proposed which shows reasonable loss with respect to the unconstrained case where the only target is the maximization of the sum data rate and impressive profit compared to static TDMA scheme. Apart from the theoretical analysis of the above algorithms, simulation code, which was created based on realistic assumptions and simplifications, gave us results which measure the sum of the users' data rates that provide each one of the aforementioned algorithms and examine the possible appropriateness for use in specific environments. The final concluding results are that MIMO-OFDMA systems are able to offer real broadband services over the wireless communication channel.

Multi-antenna wireless communication systems that employ a large number of antennas have recently stirred a lot of research interest. This is mainly due to the possibility of achieving very high spectral efficiency, power efficiency, and link reliability in such large-scale multiple-input multiple-output (MIMO) systems. An emerging architecture for large-scale multiuser MIMO communications is one where each base station (BS) is equipped with a large number of antennas (tens to hundreds of antennas) and the user terminals are equipped with fewer antennas (one to four antennas) each. The backhaul communication between base stations is also carried out using large number of antennas. Because of the high dimensionality of large-scale MIMO signals, the computational complexity of various transceiver operations can be prohibitively large. Therefore, low complexity techniques that scale well for transceiver signal processing in such large-scale MIMO systems are crucial. The transceiver operations of interest include signal encoding at the transmitter, and channel estimation, detection and decoding at the receiver. This thesis focuses on the design and analysis of novel low-complexity transceiver signal processing schemes for large-scale MIMO systems. In this thesis, we consider two types of large-scale MIMO systems, namely, massive MIMO systems and generalized spatial modulation MIMO (GSM-MIMO) systems. In massive MIMO, the mapping of information bits to modulation symbols is done using conventional modulation alphabets (e.g., QAM, PSK). In GSM-MIMO, few of the avail-able transmit antennas are activated in a given channel use, and information bits are conveyed through the indices of these active antennas, in addition to the bits conveyed through conventional modulation symbols. We also propose a novel modulation scheme named as precoder index modulation, where information bits are conveyed through the index of the chosen precoder matrix as well as the modulation symbols transmitted. Massive MIMO: In this part of the thesis, we propose a novel MIMO receiver which exploits channel hardening that occurs in large-scale MIMO channels. Channel hardening refers to the phenomenon where the off-diagonal terms of HH H become much weaker compared to the diagonal terms as the size of the channel gain matrix H increases. We exploit this phenomenon to devise a low-complexity channel estimation scheme and a message passing algorithm for signal detection at the BS receiver in massive MIMO systems. We refer to the proposed receiver as the channel hardening-exploiting message passing (CHEMP) receiver. The key novelties in the proposed CHEMP receiver are: (i) operation on the matched filtered system model, (ii) Gaussian approximation on the off-diagonal terms of the HH H matrix, and (iii) direct estimation of HH H instead of H and use of this estimate of HH H for detection The performance and complexity results show that the proposed CHEMP receiver achieves near-optimal performance in large-scale MIMO systems at complexities less than those of linear receivers like minimum mean squared error (MMSE) receiver. We also present a log-likelihood ratio (LLR) analysis that provides an analytical reasoning for this better performance of the CHEMP receiver. Further, the proposed message passing based detection algorithm enables us to combine it with low density parity check (LDPC) decoder to formulate a joint message passing based detector-decoder. For this joint detector-decoder, we design optimized irregular binary LDPC codes specific to the massive MIMO channel and the proposed receiver through EXIT chart matching. The LDPC codes thus obtained are shown to achieve improved coded bit error rate (BER) performance compared to off-the-shelf irregular LDPC codes. The performance of the CHEMP receiver degrades when the system loading factor (ratio of the number of users to the number of BS antennas) and the modulation alpha-bet size are large. It is of interest to devise receiver algorithms that work well for high system loading factors and modulation alphabet sizes. For this purpose, we propose a low-complexity factor-graph based vector message passing algorithm for signal detection. This algorithm uses a scalar Gaussian approximation of interference on the basic sys-tem model. The performance results show that this algorithm performs well for large modulation alphabets and high loading factors. We combine this detection algorithm with a non-binary LDPC decoder to obtain a joint detector-decoder, where the field size of the non-binary LDPC code is same as the size of the modulation alphabet. For this joint message passing based detector-decoder, we design optimized non-binary irregular LDPC codes tailored to the massive MIMO channel and the proposed detector. GSM-MIMO: In this part of the thesis, we consider GSM-MIMO systems in point-to-point as well as multiuser communication settings. GSM-MIMO has the advantage of requiring only fewer transmit radio frequency (RF) chains than the number of transmit antennas. We analyze the capacity of point-to-point GSM-MIMO, and obtain lower and upper bounds on the GSM-MIMO system capacity. We also derive an upper bound on the BER performance of maximum likelihood detection in GSM-MIMO systems. This bound is shown to be tight at moderate to high signal-to-noise ratios. When the number of transmit and receive antennas are large, the complexity of en-coding and decoding of GSM-MIMO signals can be prohibitively high. To alleviate this problem, we propose a low complexity GSM-MIMO encoding technique that utilizes com-binatorial number system for bits-to-symbol mapping. We also propose a novel layered message passing (LaMP) algorithm for decoding GSM-MIMO signals. Low computational complexity is achieved in the LaMP algorithm by detecting the modulation bits and the antenna index bits in two deferent layers. We then consider large-scale multiuser GSM-MIMO systems, where multiple users employ GSM at their transmitters to communicate with a BS having a large number of receive antennas. For this system, we develop computationally efficient message passing algorithms for signal detection using vector Gaussian approximation of interference. The performance results of these algorithms show that the GSM-MIMO system outperforms the massive MIMO system by several dBs for the same spectral efficiency. Precoder index modulation: It is known that the performance of a communication link can be enhanced by exploiting time diversity without reducing the rate of transmission using pseudo random phase preceding (PRPP). In order to further improve the performance of GSM-MIMO, we apply PRPP technique to GSM-MIMO systems. PRPP provides additional diversity advantage at the receiver and further improves the performance of GSM-MIMO systems. For PRPP-GSM systems, we propose methods to simultaneously precode both the antenna index bits and the modulation symbols using rectangular precoder matrices. Finally, we extend the idea of index modulation to pre-coding and propose a new modulation scheme referred to as precoder index modulation (PIM). In PIM, information bits are conveyed through the index of a prehared PRPP matrix, in addition to the information bits conveyed through the modulation symbols. PIM is shown to increase the achieved spectral efficiency, in addition to providing diver-sity advantages.

Bayesian approaches for sparse signal recovery have enjoyed a long-standing history in signal processing and machine learning literature. Among the Bayesian techniques, the expectation maximization based Sparse Bayesian Learning(SBL) approach is an iterative procedure with global convergence guarantee to a local optimum, which uses a parameterized prior that encourages sparsity under an evidence maximization frame¬work. SBL has been successfully employed in a wide range of applications ranging from image processing to communications. In this thesis, we propose novel, efficient and low-complexity SBL-based algorithms that exploit structured sparsity in the presence of fully/partially known measurement matrices. We apply the proposed algorithms to the problem of channel estimation and data detection in Orthogonal Frequency Division Multiplexing(OFDM) systems. Further, we derive Cram´er Rao type lower Bounds(CRB) for the single and multiple measurement vector SBL problem of estimating compressible vectors and their prior distribution parameters. The main contributions of the thesis are as follows: We derive Hybrid, Bayesian and Marginalized Cram´er Rao lower bounds for the problem of estimating compressible vectors drawn from a Student-t prior distribution. We derive CRBs that encompass the deterministic or random nature of the unknown parameters of the prior distribution and the regression noise variance. We use the derived bounds to uncover the relationship between the compressibility and Mean Square Error(MSE) in the estimates. Through simulations, we demonstrate the dependence of the MSE performance of SBL based estimators on the compressibility of the vector. OFDM is a well-known multi-carrier modulation technique that provides high spectral efficiency and resilience to multi-path distortion of the wireless channel It is well-known that the impulse response of a wideband wireless channel is approximately sparse, in the sense that it has a small number of significant components relative to the channel delay spread. In this thesis, we consider the estimation of the unknown channel coefficients and its support in SISO-OFDM systems using a SBL framework. We propose novel pilot-only and joint channel estimation and data detection algorithms in block-fading and time-varying scenarios. In the latter case, we use a first order auto-regressive model for the time-variations, and propose recursive, low-complexity Kalman filtering based algorithms for channel estimation. Monte Carlo simulations illustrate the efficacy of the proposed techniques in terms of the MSE and coded bit error rate performance. • Multiple Input Multiple Output(MIMO) combined with OFDM harnesses the inherent advantages of OFDM along with the diversity and multiplexing advantages of a MIMO system. The impulse response of wireless channels between the Nt transmit and Nr receive antennas of a MIMO-OFDM system are group approximately sparse(ga-sparse),i.e. ,the Nt Nr channels have a small number of significant paths relative to the channel delay spread, and the time-lags of the significant paths between transmit and receive antenna pairs coincide. Often, wire¬less channels are also group approximately-cluster sparse(ga-csparse),i.e.,every ga-sparse channel consists of clusters, where a few clusters have all strong components while most clusters have all weak components. In this thesis, we cast the problem of estimating the ga-sparse and ga-csparse block-fading and time-varying channels using a multiple measurement SBL framework. We propose a bouquet of novel algorithms for MIMO-OFDM systems that generalize the algorithms proposed in the context of SISO-OFDM systems. The efficacy of the proposed techniques are demonstrated in terms of MSE and coded bit error rate performance.

Space-Time Block Coding is an efficient communication technique used in multiple-input multiple-output wireless systems. The complexity with which a Space-Time Block Code (STBC) can be decoded is important from an implementation point of view since it directly affects the receiver complexity and speed. In this thesis, we address the problem of designing low complexity decoding techniques for STBCs, and constructing STBCs that achieve high rate and full-diversity with these decoders. This thesis is divided into two parts; the first is concerned with the optimal decoder, viz. the maximum-likelihood (ML) decoder, and the second with non-ML decoders. An STBC is said to be multigroup ML decodable if the information symbols encoded by it can be partitioned into several groups such that each symbol group can be ML decoded independently of the others, and thereby admitting low complexity ML decoding. In this thesis, we first give a new framework for constructing low ML decoding complexity STBCs using codes over the Klein group, and show that almost all known low ML decoding complexity STBCs can be obtained by this method. Using this framework we then construct new full-diversity STBCs that have the least known ML decoding complexity for a large set of choices of number of transmit antennas and rate. We then introduce the notion of Asymptotically-Good (AG) multigroup ML decodable codes, which are families of multigroup ML decodable codes whose rate increases linearly with the number of transmit antennas. We give constructions for full-diversity AG multigroup ML decodable codes for each number of groups g > 1. For g > 2, these are the first instances of g-group ML decodable codes that are AG or have rate more than 1. For g = 2 and identical delay, the new codes match the known families of AG codes in terms of rate. In the final section of the first part we show that the upper triangular matrix R encountered during the sphere-decoding of STBCs can be rank-deficient, thus leading to higher sphere-decoding complexity, even when the rate is less than the minimum of the number of transmit antennas and the number receive antennas. We show that all known AG multigroup ML decodable codes suffer from such rank-deficiency, and we explicitly derive the sphere-decoding complexities of most known AG multigroup ML decodable codes. In the second part of this thesis we first study a low complexity non-ML decoder introduced by Guo and Xia called Partial Interference Cancellation (PIC) decoder. We give a new full-diversity criterion for PIC decoding of STBCs which is equivalent to the criterion of Guo and Xia, and is easier to check. We then show that Distributed STBCs (DSTBCs) used in wireless relay networks can be full-diversity PIC decoded, and we give a full-diversity criterion for the same. We then construct full-diversity PIC decodable STBCs and DSTBCs which give higher rate and better error performance than known multigroup ML decodable codes for similar decoding complexity, and which include other known full-diversity PIC decodable codes as special cases. Finally, inspired by a low complexity essentially-ML decoder given by Sirianunpiboon et al. for the two and three antenna Perfect codes, we introduce a new non-ML decoder called Adaptive Conditional Zero-Forcing (ACZF) decoder which includes the technique of Sirianunpiboon et al. as a special case. We give a full-diversity criterion for ACZF decoding, and show that the Perfect codes for two, three and four antennas, the Threaded Algebraic Space-Time code, and the 4 antenna rate 2 code of Srinath and Rajan satisfy this criterion. Simulation results show that the proposed decoder performs identical to ML decoding for these five codes. These STBCs along with ACZF decoding have the best error performance with least complexity among all known STBCs for four or less transmit antennas.

In this thesis, our focus is on low-complexity, high-performance detection algorithms for multi-antenna communication receivers. A key contribution in this thesis is the demonstration that efficient algorithms from metaheuristics and machine learning can be gainfully adapted for signal detection in multi- antenna communication receivers. We first investigate a popular metaheuristic known as the reactive tabu search (RTS), a combinatorial optimization technique, to decode the transmitted signals in large-dimensional communication systems. A basic version of the RTS algorithm is shown to achieve near-optimal performance for 4-QAM in large dimensions. We then propose a method to obtain a lower bound on the BER performance of the optimal detector. This lower bound is tight at moderate to high SNRs and is useful in situations where the performance of optimal detector is needed for comparison, but cannot be obtained due to very high computational complexity. To improve the performance of the basic RTS algorithm for higher-order modulations, we propose variants of the basic RTS algorithm using layering and multiple explorations. These variants are shown to achieve near-optimal performance in higher-order QAM as well. Next, we propose a new receiver called linear regression of minimum mean square error (MMSE) residual receiver (referred to as LRR receiver). The proposed LRR receiver improves the MMSE receiver by learning a linear regression model for the error of the MMSE receiver. The LRR receiver uses pilot data to estimate the channel, and then uses locally generated training data (not transmitted over the channel) to find the linear regression parameters. The LRR receiver is suitable for applications where the channel remains constant for a long period (slow-fading channels) and performs well. Finally, we propose a receiver that uses a committee of linear receivers, whose parameters are estimated from training data using a variant of the AdaBoost algorithm, a celebrated supervised classification algorithm in ma- chine learning. We call our receiver boosted MMSE (B-MMSE) receiver. We demonstrate that the performance and complexity of the proposed B-MMSE receiver are quite attractive for multi-antenna communication receivers.

Abstract Multiple-input multiple-output transmitted signal design for the partially coherent Rayleigh fading channels with discrete inputs under a given average transmit power constraint is consider in this thesis. The objective is to design the space-time constellations and linear precoders to adapt to the degradation caused by the imperfect channel estimation at the receiver and the transmit-receive antenna correlation. The system is partially coherent so that the multiple-input multiple-output channel coefficients are estimated at the receiver and its error covariance matrix is fed back to the transmitter. Two constellation design criteria, one for the single and another for the multiple transmit antennae are proposed. An upper bound on the average bit error probability for the single transmit antenna and cutoff rate, i.e., a lower bound on the mutual information, for multiple transmit antennae are derived. Both criteria are functions of channel estimation error covariance matrix. The designed constellations are called as partially coherent constellation. Additionally, to use the resulting constellations together with forward error control codes requires efficient bit mapping schemes. Because these constellations lack geometrical symmetry in general, the Gray mapping is not always possible in the majority of the constellations obtained. Moreover, different mapping schemes may lead to highly different bit error rate performances. Thus, an efficient bit mapping algorithm called the modified binary switching algorithm is proposed. It minimizes an upper bound on the average bit error probability. It is shown through computer simulations that the designed partially coherent constellation and their optimized bit mapping algorithm together with turbo codes outperform the conventional constellations. Linear precoder design was also considered as a simpler, suboptimal alternative. The cutoff rate expression is again used as a criterion to design the linear precoder. A linear precoder is obtained by numerically maximizing the cutoff rate with respect to the precoder matrix with a given average transmit power constraint. Furthermore, the precoder matrix is decomposed using singular-value-decomposition into the input shaping, power loading, and beamforming matrices. The beamforming matrix is found to coincide with the eigenvectors of the transmit correlation matrix. The power loading and input shaping matrices are solved numerically using the difference of convex functions programming algorithm and optimization under the unitary constraint, respectively. Computer simulations show that the performance gains of the designed precoders are significant compared to the cutoff rate optimized partially coherent constellations without precoding. / Tiivistelmä Väitöskirjassa tarkastellaan lähetyssignaalien suunnittelua osittain koherenteissa Rayleigh-häipyvissä kanavissa toimiviin monitulo-monilähtöjärjestelmiin (MIMO). Lähettimen keskimääräinen lähetysteho oletetaan rajoitetuksi ja lähetyssignaali diskreetiksi. Tavoitteena on suunnitella tila-aikakonstellaatioita ja lineaarisia esikoodereita jotka mukautuvat epätäydellisen kanavaestimoinnin aiheuttamaan suorituskyvyn heikkenemiseen sekä lähetin- ja vastaanotinantennien väliseen korrelaatioon. Tarkasteltavien järjestelmien osittainen koherenttisuus tarkoittaa sitä, että MIMO-kanavan kanavakertoimet estimoidaan vastaanottimessa, josta niiden virhekovarianssimatriisi lähetetään lähettimelle. Työssä esitetään kaksi konstellaatiosuunnittelukriteeriä, toinen yhdelle lähetinantennille ja toinen moniantennilähettimelle. Molemmat kriteerit ovat kanavan estimaatiovirheen kovarianssimatriisin funktioita. Työssä johdetaan yläraja keskimääräiselle bittivirhetodennäköisyydelle yhden lähetinantennin tapauksessa sekä rajanopeus (cutoff rate), joka on alaraja keskinäisinformaatiolle, usean lähetinantennin tapauksessa. Konstellaatioiden käyttö yhdessä virheenkorjauskoodien kanssa edellyttää tehokaita menetelmiä, joilla bitit kuvataan konstellaatiopisteisiin. Koska tarvittavat konstellaatiot eivät ole tyypillisesti geometrisesti symmetrisiä, Gray-kuvaus ei ole yleensä mahdollinen.Lisäksi erilaiset kuvausmenetelmät voivat johtaa täysin erilaisiin bittivirhesuhteisiin. Tästä johtuen työssä esitetään uusi kuvausalgoritmi (modified bit switching algorithm), joka minimoi keskimääräisen bittivirhetodennäköisyyden ylärajan. Simulointitulokset osoittavat, että työssä kehitetyt konstellaatiot antavat paremman suorituskyvyn turbokoodatuissa järjestelmissä kuin perinteiset konstellaatiot. Työssä tarkastellaan myös lineaarista esikoodausta yksinkertaisena, alioptimaalisena vaihtoehtona uusille konstellaatioille. Esikoodauksen suunnittelussa käytetään samaa kriteeriä kuin konstellaatioiden kehityksessä eli rajanopeutta. Lineaarinen esikooderi löydetään numeerisesti maksimoimalla rajanopeus kun rajoitusehtona on lähetysteho. Esikoodausmatriisi hajotetaan singulaariarvohajotelmaa käyttäen esisuodatus, tehoallokaatio ja keilanmuodostusmatriiseiksi, jonka havaitaan vastaavan lähetyskorrelaatiomatriisin ominaisvektoreita. Tehoallokaatiomatriisi ratkaistaan numeerisesti käyttäen difference of convex functions -optimointia ja esisuodatusmatriisi optimoinnilla unitaarista rajoitusehtoa käyttäen. Simulaatiotulokset osoittavat uusien esikoodereiden tarjoavan merkittävän suorituskykyedun sellaisiin rajanopeusoptimoituihin osittain koherentteihin konstellaatioihin nähden, jotka eivät käytä esikoodausta.

One of the most popular ways to exploit the advantages of a multiple-input multiple-output (MIMO) system is using space time block coding. A space time block code (STBC) is a finite set of complex matrices whose entries consist of the information symbols to be transmitted. A linear STBC is one in which the information symbols are linearly combined to form a two-dimensional code matrix. A well known method of maximum-likelihood (ML) decoding of such STBCs is using the sphere decoder (SD). In this thesis, new constructions of STBCs with low sphere decoding complexity are presented and various ways of characterizing and reducing the sphere decoding complexity of an STBC are addressed. The construction of low sphere decoding complexity STBCs is tackled using irreducible matrix representations of Clifford algebras, cyclic division algebras and crossed-product algebras. The complexity reduction algorithms for the STBCs constructed are explored using tree based search algorithms. Considering an STBC as a vector space over the set of weight matrices, the problem of characterizing the sphere decoding complexity is addressed using quadratic form representations. The main results are as follows. A sub-class of fast decodable STBCs known as Block Orthogonal STBCs (BOSTBCs) are explored. A set of sufficient conditions to obtain BOSTBCs are explained. How the block orthogonal structure of these codes can be exploited to reduce the SD complexity of the STBC is then explained using a depth first tree search algorithm. Bounds on the SD complexity reduction and its relationship with the block orthogonal structure are then addressed. A set of constructions to obtain BOSTBCs are presented next using Clifford unitary weight designs (CUWDs), Coordinate-interleaved orthogonal designs (CIODs), cyclic division algebras and crossed product algebras which show that a lot of codes existing in literature exhibit the block orthogonal property. Next, the dependency of the ordering of information symbols on the SD complexity is discussed following which a quadratic form representation known as the Hurwitz-Radon quadratic form (HRQF) of an STBC is presented which is solely dependent on the weight matrices of the STBC and their ordering. It is then shown that the SD complexity is only a function of the weight matrices defining the code and their ordering, and not of the channel realization (even though the equivalent channel when SD is used depends on the channel realization). It is also shown that the SD complexity is completely captured into a single matrix obtained from the HRQF. Also, for a given set of weight matrices, an algorithm to obtain a best ordering of them leading to the least SD complexity is presented using the HRQF matrix.