It is well known that multiple antennas at the transmitter and receiver are imperative for reliable and high data-rate communication over wireless channels. However, these systems essentially need multiple radio frequency (RF) chains owing to multiple antennas, and hence pose challenges for applications with limited form-factor. Antenna Selection (AS) techniques alleviate this problem by using only a subset of the total available antennas and hence require only a few RF chains compared to the number of antennas. These systems operate in a closed-loop scenario, where the information fed back from the receiver is used for the transmit antenna subset selection. In contrast to this, a novel open-loop technique known as spatial modulation (SM) was recently proposed that uses a single RF-chain at the transmitter and achieves a higher spectral efficiency compared to single-input and AS based systems. The work in the thesis mainly focuses on the following aspects of SM system: Study of Mutual Information in SM systems operating in open-loop and closed-loop scenarios: We study the achievable mutual information in the SM system operating with finite and Gaussian input alphabet, and compare the results with that of the SIMO and AS based systems. Reduced-complexity maximum-likelihood (ML) decoding algorithms for SM systems: We propose ML-optimal sphere decoders for SM systems with arbitrary number of transmit antennas. Furthermore, a reduced-complexity ML detector is also proposed whose computational complexity is lowest among the known existing detectors in the literature. Transmit diversity techniques for SM systems: The conventional SM system achieves a transmit diversity order of one. We propose a complex interleaved orthogonal design baaed SM scheme that achieves a transit diversity order of two, while offering symbol-by- symbol ML decodability. Transmit antenna subset selection algorithms for SM systems: The SM system is considered in the closed-loop scenario, where only a subset of the total number of transmit antennas is chosen based on the information fed back by the receiver. Specifically, the Euclidean distance and capacity optimized antenna selection algorithms are studied in comparison with the conventional AS based systems. SM system operating in dispersive channels: The SM system operating in a dispersive channel with the aid of zero-padding is studied. It is shown that the SM system achieves full receive-diversity and multipath-diversity with ML decoding, but offers a decoding complexity that is exponential in the number of multipaths. Furthermore, a reduced complexity linear receiver is proposed that achieves achieves full multipath as well as receive-diversity, while offering a decoding complexity order same as that of the SM system operating in a frequency-flat channel.