We study the problem of communicating a discrete time analog source over a channel such that the resulting distortion is minimized. For ergodic channels, Shannon showed that separate source and channel coding is optimal. In this work we study this problem for non-ergodic channels. Although not much can be said about the general problem of transmitting any analog sources over any non-ergodic channels with any distortion metric, for many practical problems like video broadcast and voice transmission, we can gain insights by studying the transmission of a Gaussian source over a wireless channel with mean square error as the distortion measure. Motivated by different applications, we consider three different non-ergodic channel models - (1) Additive white Gaussian noise (AWGN) channel whose signal-to-noise ratio (SNR) is unknown at the transmitter; (2) Rayleigh fading multiple-input multiple-output MIMO channel whose SNR is known at the transmitter; and (3) Rayleigh fading MIMO channel whose SNR is unknown at the transmitter. The traditional approach to study these problems has been to fix certain SNRs of interest and study the corresponding achievable distortion regions. However, the problems formulated this way have not been solved even for simple setups like 2 SNRs for the AWGN channel. We are interested in performance over a wide range of SNR and hence we use the distortion SNR exponent metric to study this problem. Distortion SNR exponent is defined as the rate of decay of distortion with SNR in the high SNR limit. We study several layered transmissions schemes where the source is first compressed in layers and then the layers are transmitted using channel codes that provide variable error protection. Results show that in several cases such layered transmission schemes are optimal in terms of the distortion SNR exponent. Specifically, if the band- width expansion (number of channel uses per source sample) is b, we show that the optimal distortion SNR exponent for the AWGN channel is b and it is achievable using a superposition based layered scheme. For the L-block Rayleigh fading M x N MIMO channel the optimal exponent is characterized for b < (|N - M|+1)= min(M;N) and b > MNL2. This corresponds to the entire range of b when min(M;N) = 1 and L = 1. The results also show that the exponents obtained using layered schemes which are a small subclass of joint source channel coding (JSCC) schemes are, surprisingly, as good as and better in some cases than achievable exponent of all other JSCC schemes reported so far.

Traditionally, source coding (data compression) and channel coding (error protection) are performed separately and sequentially, resulting in what we call a tandem (separate) coding system. In practical implementations, however, tandem coding might involve a large delay and a high coding/decoding complexity, since one needs to remove the redundancy in the source coding part and then insert certain redundancy in the channel coding part. On the other hand, joint source-channel coding (JSCC), which coordinates source and channel coding or combines them into a single step, may offer substantial improvements over the tandem coding approach. This thesis deals with the fundamental Shannon-theoretic limits for a variety of communication systems via JSCC. More specifically, we investigate the reliability function (which is the largest rate at which the coding probability of error vanishes exponentially with increasing blocklength) for JSCC for the following discrete-time communication systems: (i) discrete memoryless systems; (ii) discrete memoryless systems with perfect channel feedback; (iii) discrete memoryless systems with source side information; (iv) discrete systems with Markovian memory; (v) continuous-valued (particularly Gaussian) memoryless systems; (vi) discrete asymmetric 2-user source-channel systems. For the above systems, we establish upper and lower bounds for the JSCC reliability function and we analytically compute these bounds. The conditions for which the upper and lower bounds coincide are also provided. We show that the conditions are satisfied for a large class of source-channel systems, and hence exactly determine the reliability function. We next provide a systematic comparison between the JSCC reliability function and the tandem coding reliability function (the reliability function resulting from separate source and channel coding). We show that the JSCC reliability function is substantially larger than the tandem coding reliability function for most cases. In particular, the JSCC reliability function is close to twice as large as the tandem coding reliability function for many source-channel pairs. This exponent gain provides a theoretical underpinning and justification for JSCC design as opposed to the widely used tandem coding method, since JSCC will yield a faster exponential rate of decay for the system error probability and thus provides substantial reductions in complexity and coding/decoding delay for real-world communication systems. / Thesis (Ph.D, Mathematics & Statistics) -- Queen's University, 2008-05-13 22:31:56.425