A converse method is developed for the source broadcast problem. Specifically, it is shown that the separation architecture is optimal for a variant of the source broadcast problem and the associated source-channel separation theorem can be leveraged, via a reduction argument, to establish a necessary condition for the original problem, which uni es several existing results in the literature. Somewhat surprisingly, this method, albeit based on the source-channel separation theorem, can be used to prove the optimality of non-separation based schemes and determine the performance limits in certain scenarios where the separation architecture is suboptimal. / Thesis / Master of Applied Science (MASc)

Efficient multimedia communication over mobile or wireless channels remains a challenging problem. To deal with that problem so far, the industry has followed mostly a divide and conquer approach, by considering separately the source of data (text, image, video, etc.) and the communication channel (electromagnetic waves across the air, a telephone line, a coaxial cable, etc.). The goal is always the same: to transmit (or store) more data reliably per unit of time, of energy, of physical medium, etc. With today's applications, the divide and conquer approach has, in a sense, started to show its limits. Let us consider, for example, the digital transmission of an image. At the transmitter, the first main step is data compression, at the source level. The number of bits that are necessary to represent the image with a given level of quality is reduced, usually by removing details in the image that are invisible (or less visible) to the human eye. The second main step is data protection, at the channel level. The transmission is made ideally resistant to deteriorations caused by the channel, by implementing techniques such as time/frequency/space expansions. In a sense, the two steps are quite antagonistic --- we first compress then expand the original signal --- and have different goals --- compression enables to transfer more data per unit of time/energy/medium while protection enables to transfer data reliably. At the receiver, the "reversed" operations are implemented. This separation in two steps dates back to Shannon's source and channel coding separation theorem in 1948 and has encouraged the division of the research community in two groups, one focusing on data compression, the other on data protection. This separation has also seduced the industry for the design, thereby supported by theory, of layered communication protocols. But this theorem holds only under asymptotic conditions that are rarely satisfied with today's multimedia content and mobile channels. Therefore, it is usually wise in practice to drop this strict separation and to allow at least some cross-layer cooperation between the source and channel layers. This is what lies behind the words joint source-channel techniques. As the name suggests, these techniques are optimized jointly, without a strict separation. Intuitively, since the optimization is less constrained from a mathematical standpoint, the solution can only be better or equivalent. In this thesis, we investigate a promising subset of these techniques, based on the turbo principle and on variable length codes. The potential of this subset has been illustrated for the first time in 2000, with an example that, since then, has been successfully improved in several directions. Unfortunately, most decoding algorithms have been so far developed on an ad hoc basis, without a unified view and often without specifying the approximations made. Besides, most code-related conclusions are based on simulations or on extrinsic information analysis. A theoretical framework on the error correcting properties of variable length codes in turbo systems is lacking. The purpose of this work, in three parts, is to fill in these gaps up to a certain extent. The first part presents the literature in this field and attempts to give a unified overview. The second part proposes a transmission system that generalizes previous systems from the literature, with the simple addition of a repetition code. While most previous systems are designed for bit streams with a high level of residual redundancy, the proposed system has the interesting flexibility to handle easily different levels of redundancy. Its performance is then analyzed for small levels of redundancy, which is a case not tackled extensively in the literature. This analysis leads notably to the discovery of surprising interleaving gains with reversible variable length codes. The third part develops the mathematical framework that was motivated during the second part but skipped on purpose for the sake of clarity. We first clarify several issues that arise with non-uniform bits and the extrinsic information charts, and propose and discuss two methods to compute these charts. Next, several theoretical results are stated on the robustness of variable length codes concatenated with linear error correcting codes. Notably, an approximate average distance spectrum of the concatenated code is rigorously developed. Together with the union bound, this spectrum provides upper bounds on the symbol and frame/packet error rates. These bounds are then analyzed from an interleaving gain standpoint and it is proved that the variable length code improves the interleaving gain if its spectrum is bounded.

The aim of source coding is to represent information as accurately as possible using as few bits as possible and in order to do so redundancy from the source needs to be removed. The aim of channel coding is in some sense the contrary, namely to introduce redundancy that can be exploited to protect the information when being transmitted over a nonideal channel. Combining these two techniques leads to the area of joint source-channel coding which in general makes it possible to achieve a better performance when designing a communication system than in the case when source and channel codes are designed separately. In this thesis four particular areas in joint source-channel coding are studied: analog (i.e. continuous) bandwidth expansion, distributed source coding over noisy channels, multiple description coding (MDC) and soft decoding. A general analog bandwidth expansion code based on orthogonal polynomials is proposed and analyzed. The code has a performance comparable with other existing schemes. However, the code is more general in the sense that it is implementable for a larger number of source distributions. The problem of distributed source coding over noisy channels is studied. Two schemes are proposed and analyzed for this problem which both work on a sample by sample basis. The first code is based on scalar quantization optimized for a certain channel characteristics. The second code is nonlinear and analog. Two new MDC schemes are proposed and investigated. The first is based on sorting a frame of samples and transmitting, as side-information/redundancy, an index that describes the resulting permutation. In case that some of the transmitted descriptors are lost during transmission this side information (if received) can be used to estimate the lost descriptors based on the received ones. The second scheme uses permutation codes to produce different descriptions of a block of source data. These descriptions can be used jointly to estimate the original source data. Finally, also the MDC method multiple description coding using pairwise correlating transforms as introduced by Wang et al. is studied. A modi fication of the quantization in this method is proposed which yields a performance gain. A well known result in joint source-channel coding is that the performance of a communication system can be improved by using soft decoding of the channel output at the cost of a higher decoding complexity. An alternative to this is to quantize the soft information and store the pre-calculated soft decision values in a lookup table. In this thesis we propose new methods for quantizing soft channel information, to be used in conjunction with soft-decision source decoding. The issue on how to best construct finite-bandwidth representations of soft information is also studied. / QC 20100920

Error Correction and Concealment of Block Based, Motion-Compensated Temporal Prediction, Transform Coded Video David L. Robie 133 Pages Directed by Dr. Russell M. Mersereau The use of the Internet and wireless networks to bring multimedia to the consumer continues to expand. The transmission of these products is always subject to corruption due to errors such as bit errors or lost and ill-timed packets; however, in many cases, such as real time video transmission, retransmission request (ARQ) is not practical. Therefore receivers must be capable of recovering from corrupted data. Errors can be mitigated using forward error correction in the encoder or error concealment techniques in the decoder. This thesis investigates the use of forward error correction (FEC) techniques in the encoder and error concealment in the decoder in block-based, motion-compensated, temporal prediction, transform codecs. It will show improvement over standard FEC applications and improvements in error concealment relative to the Motion Picture Experts Group (MPEG) standard. To this end, this dissertation will describe the following contributions and proofs-of-concept in the area of error concealment and correction in block-based video transmission. A temporal error concealment algorithm which uses motion-compensated macroblocks from previous frames. A spatial error concealment algorithm which uses the Hough transform to detect edges in both foreground and background colors and using directional interpolation or directional filtering to provide improved edge reproduction. A codec which uses data hiding to transmit error correction information. An enhanced codec which builds upon the last by improving the performance of the codec in the error-free environment while maintaining excellent error recovery capabilities. A method to allocate Reed-Solomon (R-S) packet-based forward error correction that will decrease distortion (using a PSNR metric) at the receiver compared to standard FEC techniques. Finally, under the constraints of a constant bit rate, the tradeoff between traditional R-S FEC and alternate forward concealment information (FCI) is evaluated. Each of these developments is compared and contrasted to state of the art techniques and are able to show improvements using widely accepted metrics. The dissertation concludes with a discussion of future work.

<p>The aim of source coding is to represent information as accurately as possible using as few bits as possible and in order to do so redundancy from the source needs to be removed. The aim of channel coding is in some sense the contrary, namely to introduce redundancy that can be exploited to protect the information when being transmitted over a nonideal channel. Combining these two techniques leads to the area of joint source–channel coding which in general makes it possible to achieve a better performance when designing a communication system than in the case when source and channel codes are designed separately. In this thesis two particular areas in joint source–channel coding are studied: multiple description coding (MDC) and soft decoding. Two new MDC schemes are proposed and investigated. The first is based on sorting a frame of samples and transmitting, as side-information/redundancy, an index that describes the resulting permutation. In case that some of the transmitted descriptors are lost during transmission this side information (if received) can be used to estimate the lost descriptors based on the received ones. The second scheme uses permutation codes to produce different descriptions of a block of source data. These descriptions can be used jointly to estimate the original source data. Finally, also the MDC method multiple description coding using pairwise correlating transforms as introduced by Wang et al is studied. A modification of the quantization in this method is proposed which yields a performance gain. A well known result in joint source–channel coding is that the performance of a communication system can be improved by using soft decoding of the channel output at the cost of a higher decoding complexity. An alternative to this is to quantize the soft information and store the pre-calculated soft decision values in a lookup table. In this thesis we propose new methods for quantizing soft channel information, to be used in conjunction with soft-decision source decoding. The issue on how to best construct finite-bandwidth representations of soft information is also studied.</p>

Linear programming (LP) decoding of low-density parity-check codes was introduced by Feldman et al. in [1]. In his formulation it is assumed that communication takes place over a memoryless channel and that the source is uniform. Here, we extend the LP decoding paradigm by studying its application to scenarios with source non-uniformity and to decoding over channels with memory. We develop two decoders for the scenario of non-uniform memoryless sources transmitted over memoryless channels. The first decoder uses a modified linear cost function which incorporates the a-priori source information and works with systematic codes. The second decoder differs by using non-systematic codes obtained by puncturing lower rate systematic codes and using an “extended decoding polytope.” Simulations show that the modified decoders yield gains over the standard LP decoder. Next, LP decoding is considered for two channels with memory: the binary additive Markov noise channel and the infinite-memory non-ergodic Polya-contagion channel. For the Markov channel, no linear cost function corresponding to maximum likelihood (ML) decoding could be obtained and hence it is unclear how to proceed. For the Polya channel, two LP-based decoders are developed. The first is derived in a straightforward manner from the ML decoding rule of [2]. The second decoder relies on a simplification of the same ML decoding rule which holds for codes containing the all-ones codeword. Simulations are performed for both decoders with regular and irregular LDPC codes and demonstrate relatively good performance with respect to the channel epsilon-capacity. / Thesis (Master, Mathematics & Statistics) -- Queen's University, 2008-12-08 16:24:43.358

This thesis studies cooperative sensing and transmission in the context ofwireless sensor networks (WSNs). We especially focus on two means of cooperative sensing and transmission, namely, distributed source coding and relaying. We consider systems where the usefulness of the measured data is dependent on how old the data is and we therefore need low-delay transmission schemes. At first sight, the low-delay criterion may seem to be of little relevance, but it is this aspect in particular that distinguishes this thesis from many of the existing communication theoretic results, which often are asymptotic in the block lengths. The thesis is composed of an introductory part, discussing the fundamentals of communication theory and how these are related to the requirements of WSNs, followed by a part where the results of the thesis are reported in Papers A-H. Papers A-D study different scenarios for distributed source-channel coding. In Paper A, we consider transmission of correlated continuous sources and propose an iterative algorithm for designing simple and energy-efficient sensor nodes. In particular the cases of the binary symmetric channel as well as the additive white Gaussian noise channel are studied. In Paper B, the work is extended to channels with interference and it is shown that also in this case there can be significant power savings by performing a joint optimization of the system.Papers C and D use a more structured approach and propose side-information-aware source-channel coding strategies using lattices and sinusoids. In Paper E, we apply the methods we have used in joint source-channel coding to the famous Witsenhausen counterexample. By using a relatively simple iterative algorithm, we are able to demonstrate the best numerical performance known to date. For the case of systems with relays, we study the transmission of a continuous Gaussian source and the transmission of an uniformly distributed discrete source. In both situations, we propose algorithms to design low-delay source-channel and relay mappings. By studying the structure of the optimized source-channel and relay mappings, we provide useful insights into how the optimized systems work. These results are reported in Papers F and G. In Paper H, we finally consider sum-MSE minimization for the Gaussian multiple-input, multiple-output broadcast channel. By using recently discovered properties of this problem, we derive a closed-form expression for the optimal power allocation in the two-user scenario and propose a conceptually simple and efficient algorithm that handles an arbitrary number of users. Throughout the thesis we show that there are significant gains if the parts of the system are jointly optimized for the source and channel statistics. All methods that are considered in this thesis yield very low coding and decoding delays. In general, nonlinear mappings outperform linear mappings for problems where there is side-information available. Another contribution of this thesis is visualization of numerically optimized systems that can be used as inspiration when structured low-delay systems are designed. / The author changed name from Johannes Karlsson to Johannes Kron in January 2011. QC 20110512

The aim of source coding is to represent information as accurately as possible using as few bits as possible and in order to do so redundancy from the source needs to be removed. The aim of channel coding is in some sense the contrary, namely to introduce redundancy that can be exploited to protect the information when being transmitted over a nonideal channel. Combining these two techniques leads to the area of joint source–channel coding which in general makes it possible to achieve a better performance when designing a communication system than in the case when source and channel codes are designed separately. In this thesis two particular areas in joint source–channel coding are studied: multiple description coding (MDC) and soft decoding. Two new MDC schemes are proposed and investigated. The first is based on sorting a frame of samples and transmitting, as side-information/redundancy, an index that describes the resulting permutation. In case that some of the transmitted descriptors are lost during transmission this side information (if received) can be used to estimate the lost descriptors based on the received ones. The second scheme uses permutation codes to produce different descriptions of a block of source data. These descriptions can be used jointly to estimate the original source data. Finally, also the MDC method multiple description coding using pairwise correlating transforms as introduced by Wang et al is studied. A modification of the quantization in this method is proposed which yields a performance gain. A well known result in joint source–channel coding is that the performance of a communication system can be improved by using soft decoding of the channel output at the cost of a higher decoding complexity. An alternative to this is to quantize the soft information and store the pre-calculated soft decision values in a lookup table. In this thesis we propose new methods for quantizing soft channel information, to be used in conjunction with soft-decision source decoding. The issue on how to best construct finite-bandwidth representations of soft information is also studied. / QC 20101124

[pt] Este trabalho apresenta um estudo sobre compressão de imagens de sensoriamento remoto para serem transmitidas através de um canal com ruído. As imagens são capturadas por um satélite de sensoriamento remoto e transmitidas a uma estação terrestre. A compreensão das imagens é necessária para se economizar banda e potência de transmissão. Algumas técnicas muito boas de compressão de imagens apresentam sérios problemas quando na presença de ruído. Assim, a técnica de quantização vetorial foi escolhida para ser utilizada neste trabalho. Utilizando-se a idéia de quantização vetorial multi-estágios, propões-se um esquema de compressão com remoção de médias, onde separa-se a informação contida na imagem para tratá-la de forma diferenciada, de acordo com a sua importância. É feita então uma análise sobre o projeto do enlace do satélite do sensoriamento remoto comparando-se o esquema utilizado atualmente com o esquema proposto. / [en] This thesis presents a study of remote sensing image compression to be transmitted over a noisy channel. The images are obtained by a remote sensing satellite and transmitting to an earth station. The compression is due to savings in bandwidth and transmitting power. Some of the most efficient image codecs presents serious problems in the presence of noise. So, the vector quantization technique was chosen to be used. Using the multi-stage vector quantization idea, a compression scheme with mean remove is proposed as a manner to separate and treat unequally the image information as its importance. An analysis on the design of the remote sensing satellite link is done with a comparison between the current scheme used the proposed one.

[pt] Este trabalho estuda o problema de compressão de imagens de sensoriamento remoto segundo a ótica da codificação conjunta fonte-canal. É analisado o desempenho de métodos baseados em quantização vetorial segundo o algoritmo LBG, principalmente o COVQ (Channel Optimized Vector Quantizer) bem como a quantização vetorial estruturada em árvore. Dentro desse contexto, são propostos 2 novos métodos para a resolução do problema: (1)Uma quantização vetorial estruturada em árvores que leva em conta a transmissão através de canais ruidosos, solução denominada COTSVQ (Channel-Design Tree Strutured Vecotr Quantizer), bem como (2) uma classe de métodos que se utiliza de códigos corretores de erro sobre a estrutura progressiva do TSVQ, de forma a proteger os dados de forma ativa durante a transmissão. Os dois métodos propostos podem ser combinados no mesmo compressor, de forma a originar uma classe ampla de compressores adaptados à transmissão por canais com ruído. São apresentados resultados que comparam os desempenhos dos métodos propostos com aqueles já existentes para uma análise de desempenho, na situação de transmissão via satélite de imagens captadas e comprimidas para uma taxa de 1,5bpp. Os resultados mostram que os métodos propostos são muito menos complexos que os já existentes, porém conseguindo atingir uma qualidade de imagem equivalente, ou, em alguns casos, superior. / [en] This work studies the problem of remote sensorng image compression by joint source-channel coding. The vector quantizer methods evaluated are those designed with the LBG algorithm, the COVQ (channel-optimized vector quantizer) algorithm as well as tree-structured vector quantizer. The noisy channel is modelled as a BSC. In this context, two news methods are proposed: (1) A tree- structures vector quantizer that considers the transmission through noisy channels (denominated CD-TSVQ), and (2) a new class of compressors that uses forward error- correcting codes over the TSVQ structure, as a way to actively protect data during the transmission. The twoproposed methods can be combined on the same compressor architecture, resulting in a vast class of compressors well-adapted to the transmission through noisy channels. Results allowing the comparision of the proposed methods with existing ones are presented. Performance evaluated in a scenery where images are compressed to be transmited at a rate of 1.5bpp. Results yield to the conclusion that the porposed methods are much less complex than the existing methods, yet achieve equivalent or, in some situations, improved performance.