Multi-antenna wireless systems have become very popular due to their theoretically predicted higher spectral efficiencies and improved performance compared to single-antenna systems. Large-scale multiple-input multiple-output (MIMO) systems refer to wireless systems where communication terminals employ tens to hundreds of antennas to achieve in-creased spectral efficiencies/sum rates, reliability, and power efficiency. Large-scale multi-antenna systems are attractive to meet the increasing wireless data rate requirements, without compromising on the bandwidth. This thesis addresses key signal processing issues in large-scale MIMO systems. Specifically, the thesis investigates efficient algorithms for signal detection and channel estimation in large-scale MIMO systems. It also investigates ‘spatial modulation,’ a multi-antenna modulation scheme that can reduce the number of transmit radio frequency (RF) chains, without compromising much on the spectral efficiency. The work reported in this thesis is comprised of the following two parts: 1 investigation of low-complexity receiver algorithms based on Markov chain Monte Carlo (MCMC) technique, tabu search, and belief propagation for large-scale uplink multiuser MIMO systems, and 2 investigation of achievable rates and signal detection in generalized spatial modulation. 1. Receiver algorithms for large-scale multiuser MIMO systems on the uplink In this part of the thesis, we propose low-complexity algorithms based on MCMC techniques, Gaussian sampling based lattice decoding (GSLD), reactive tabu search (RTS), and factor graph based belief propagation (BP) for signal detection on the uplink in large-scale multiuser MIMO systems. We also propose an efficient channel estimation scheme based on Gaussian sampling. Markov chain Monte Carlo (MCMC) sampling: We propose a novel MCMC based detection algorithm, which achieves near-optimal performance in large dimensions at low complexities by the joint use of a mixed Gibbs sampling (MGS) strategy and a multiple restart strategy with an efficient restart criterion. The proposed mixed Gibbs sampling distribution is a weighted mixture of the target distribution and uniform distribution. The presence of the uniform component in the sampling distribution allows the algorithm to exit from local traps quickly and alleviate the stalling problem encountered in conventional Gibbs sampling. We present an analysis for the optimum choice of the mixing ratio. The analysis approach is to define an absorbing Markov chain and use its property regarding the expected number of iterations needed to reach the global minima for the first time. We also propose an MCMC based algorithm which exploits the sparsity in uplink multiuser MIMO transmissions, where not all users are active simultaneously. Gaussian sampling based lattice decoding: Next, we investigate the problem of searching the closest lattice point in large dimensional lattices and its use in signal detection in large-scale MIMO systems. Specifically, we propose a Gaussian sampling based lattice decoding (GSLD) algorithm. The novelty of this algorithm is that, instead of sampling from a discrete distribution as in Gibbs sampling, the algorithm iteratively generates samples from a continuous Gaussian distribution, whose parameters are obtained analytically. This makes the complexity of the proposed algorithm to be independent of the size of the modulation alpha-bet. Also, the algorithm is able to achieve near-optimal performance for different antenna and modulation alphabet settings at low complexities. Random restart reactive tabu search (R3TS): Next, we study receiver algorithms based on reactive tabu search (RTS) technique in large-scale MIMO systems. We propose a multiple random restarts based reactive tabu search (R3TS) algorithm that achieves near-optimal performance in large-scale MIMO systems. A key feature of the proposed R3TS algorithm is its performance based restart criterion, which gives very good performance-complexity tradeoff in large-dimension systems. Lower bound on maximum likelihood (ML) bit error rate (BER) performance: We propose an approach to obtain lower bounds on the ML performance of large-scale MIMO systems using RTS simulation. In the proposed approach, we run the RTS algorithm using the transmitted vector as the initial vector, along with a suitable neighborhood definition, and find a lower bound on number of errors in ML solution. We demonstrate that the proposed bound is tight (within about 0.5 dB of the optimal performance in a 16×16MIMO system) at moderate to high SNRs. Factor graph using Gaussian approximation of interference (FG-GAI): Multiuser MIMO channels can be represented by graphical models that are fully/densely connected (loopy graphs), where conventional belief propagation yields suboptimal performance and requires high complexity. We propose a solution to this problem that uses a simple, yet effective, Gaussian approximation of interference (GAI) approach that carries out a linear per-symbol complexity message passing on a factor graph (FG) based graphical model. The proposed algorithm achieves near-optimal performance in large dimensions in frequency-flat as well as frequency-selective channels. Gaussian sampling based channel estimation: Next, we propose a Gaussian sampling based channel estimation technique for large-scale time-division duplex (TDD) MIMO systems. The proposed algorithm refines the initial estimate of the channel by iteratively detecting the data block and using that knowledge to improve the estimated channel knowledge using a Gaussian sampling based technique. We demonstrate that this algorithm achieves near-optimal performance both in terms of mean square error of the channel estimates and BER of detected data in both frequency-flat and frequency-selective channels. 2. Generalized spatial modulation In the second part of the thesis, we investigate generalized spatial modulation (GSM) in point-to point MIMO systems. GSM is attractive because of its ability to work with less number of transmit RF chains compared to traditional spatial multiplexing, without com-promising much on spectral efficiency. In this work, we show that, by using an optimum combination of number of transmit antennas and number of transmit RF chains, GSM can achieve better throughput and/or BER than spatial multiplexing. We compute tight bounds on the maximum achievable rate in a GSM system, and quantify the percentage savings in the number of transmit RF chains as well as the percentage increase in the rate achieved in GSM compared to spatial multiplexing. We also propose a Gibbs sampling based algorithm suited to detect GSM signals, which yields impressive BER performance and complexity results.