Shannon's classical separation result holds only in the limit of infinite source code dimension and infinite channel code block length. In addition, Shannon theory does not address the design of good source codes when the probability of channel error is nonzero, which is inevitable for finite-length channel codes. Thus, for practical systems, a joint source and channel code design could improve performance for finite dimension source code and finite block length channel code, as well as complexity and delay. Consider a multicast system over a broadcast channel, where different end users typically have different capacities. To support such user or capacity diversity, it is desirable to encode the source to be broadcasted into a scalable bit stream along which multiple resolutions of the source can be reconstructed progressively from left to right. Such source coding technique is called multiresolution source coding. In wireless communications, joint source channel coding (JSCC) has attracted wide attention due to its adaptivity to time-varying channels. However, there are few works on joint source channel coding for network multicast, especially for the optimal source coding over broadcast channels. In this work, we aim at designing and analyzing the optimal multiresolution vector quantization (MRVQ) in conjunction with the subsequent broadcast channel over which the coded scalable bit stream would be transmitted. By adopting random index assignment (RIA) to link MRVQ for the source with superposition coding for the broadcast channel, we establish a closed-form formula of end-to-end distortion for a tandem system of MRVQ and a broadcast channel. From this formula we analyze the intrinsic structure of end-to-end distortion (EED) in a communication system and derive two necessary conditions for optimal multiresolution vector quantization over broadcast channels with random index assignment. According to the two necessary conditions, we propose a greedy iterative algorithm for jointly designed MRVQ with channel conditions, which depends on the channel only through several types of average channel error probabilities rather than the complete knowledge of the channel. Experiments show that MRVQ designed by the proposed algorithm significantly outperforms conventional MRVQ designed without channel information. By building an closed-form formula for the weighted EED with RIA, it also makes the computational complexity incurred during the performance analysis feasible. In comparison with MRVQ design for a fixed index assignment, the computation complexity for quantization design is significantly reduced by using random index assignment. In addition, simulations indicate that our proposed algorithm shows better robustness against channel mismatch than MRVQ design with a fixed index assignment, simply due to the nature of using only the average channel information. Therefore, we conclude that our proposed algorithm is more appropriate in both wireless communications and applications where the complete knowledge of the channel is hard to obtain. Furthermore, we propose two novel algorithms for MRVQ over broadcast channels. One aims to optimize the two corresponding quantizers at two layers alternatively and iteratively, and the other applies under the constraint that each encoding cell is convex and contains the reconstruction point. Finally, we analyze the asymptotic performance of weighted EED for the optimal joint MRVQ. The asymptotic result provides a theoretically achievable quantizer performance level and sheds light on the design of the optimal MRVQ over broadcast channel from a different aspect.

The paradigm of separate source-channel coding is inspired by Shannon's separation result, which implies the asymptotic optimality of designing source and channel coding independently from each other. The result exploits the fact that channel error probabilities can be made arbitrarily small, as long as the block length of the channel code can be made arbitrarily large. However, this is not possible in practice, where the block length is either fixed or restricted to a range of finite values. As a result, the optimality of source and channel coding separation becomes unknown, leading researchers to consider joint source-channel coding (JSCC) to further improve the performance of practical systems that must operate in the finite block length regime. With this motivation, this thesis investigates the application of JSCC principles for multimedia communications over point-to-point, broadcast, and relay channels. All analyses are conducted from the perspective of end-to-end distortion (EED) for results that are applicable to channel codes with finite block lengths in pursuing insights into practical design. The thesis first revisits the fundamental open problem of the separation of source and channel coding in the finite block length regime. Derived formulations and numerical analyses for a source-channel coding system reveal many scenarios where the EED reduction is positive when pairing the channel-optimized source quantizer (COSQ) with an optimal channel code, hence establishing the invalidity of the separation theorem in the finite block length regime. With this, further improvements to JSCC systems are considered by augmenting error detection codes with the COSQ. Closed-form EED expressions for such system are derived, from which necessary optimality conditions are identified and used in proposed algorithms for system design. Results for both the point-to-point and broadcast channels demonstrate significant reductions to the EED without sacrificing bandwidth when considering a tradeoff between quantization and error detection coding rates. Lastly, the JSCC system is considered under relay channels, for which a computable measure of the EED is derived for any relay channel conditions with nonzero channel error probabilities. To emphasize the importance of analyzing JSCC systems under finite block lengths, the large sub-optimality in performance is demonstrated when solving the power allocation configuration problem according to capacity-based formulations that disregard channel errors, as opposed to those based on the EED. Although this thesis only considers one JSCC setup of many, it is concluded that consideration of JSCC systems from a non-asymptotic perspective not only is more meaningful, but also reveals more relevant insight into practical system design. This thesis accomplishes such by maintaining the EED as a measure of system performance in each of the considered point-to-point, broadcast, and relay cases.

Most of the conventional communication systems use channel interleaving as well as hard decision decoding in their designs, which lead to discarding channel memory and soft-decision information. This simplification is usually done since the complexity of handling the memory or soft-decision information is rather high. In this work, we design two lossy joint source-channel coding (JSCC) schemes that do not use explicit algebraic channel coding for a recently introduced channel model, in order to take advantage of both channel memory and soft-decision information. The channel model, called the non-binary noise discrete channel with queue based noise (NBNDC-QB), obtains closed form expressions for the channel transition distribution, correlation coefficient, and many other channel properties. The channel has binary input and $2^q$-ary output and the noise is a $2^q$-ary Markovian stationary ergodic process, based on a finite queue, where $q$ is the output's soft-decision resolution. We also numerically show that the NBNDC-QB model can effectively approximate correlated Rayleigh fading channels without losing its analytical tractability. The first JSCC scheme is the so called channel optimized vector quantizer (COVQ) and the second scheme consists of a scalar quantizer, a proper index assignment, and a sequence maximum a posteriori (MAP) decoder, designed to harness the redundancy left in the quantizer's indices, the channel's soft-decision output, and noise time correlation. We also find necessary and sufficient condition when the sequence MAP decoder is reduced to an instantaneous symbol-by-symbol decoder, i.e., a simple instantaneous mapping. / Thesis (Master, Electrical & Computer Engineering) -- Queen's University, 2011-09-25 19:43:28.785

Motivated by potential applications in wireless sensor networks, this thesis considers the problem of communicating a large number of correlated analog sources over a Gaussian multiple-access channel using non-orthogonal code-division multiple-access (CDMA). A joint source-channel decoder is presented which can exploit the inter-source correlation for interference reduction in the CDMA channel. This decoder uses a linear minimum mean square error (MMSE) multi-user detector (MUD) in tandem with a MMSE joint source decoder (JSD) for multiple sources to achieve a computational complexity that scales with the number of sources. The MUD and the JSD, then iteratively exchange extrinsic information to improve the interference cancellation. Experimental results show that, compared to a non-iterative decoder, the proposed iterative decoder is more robust against potential performance degradation due to correlated channel interference and offers better near far resistance.

Radio spectrum has become a scarce and priced resource due to the rapid growth of wireless networks. However, recent surveys conducted by the FCC indicate that a large part of the allotted frequency spectrum lies unused. Cognitive radio systems, built on the software defined radios, allow the efficient usage of these unused frequency spectrum. Cognitive radio systems can be modeled as a multiple access channel in which certain users have the priority (primary users) while others (cognitive or secondary users) are allowed to access the channels without causing any interference to the primary users. However a secondary user’s transmissions not only encounter high levels of uncertainty and variability in the number of channels available to them, but they also suffer data losses if a primary user activity occurs. Under such rigid constraints, the reliable transmission of real time multimedia of a secondary user with an acceptable quality of service becomes challenging. Multimedia transmission in a cognitive system requires channel adaptive source and channel coding schemes. In order to address this problem, this thesis investigates and develops a novel joint source-channel coding (JSCC) approach. The proposed JSCC allows the dynamic generation of codes, which minimizes the end-to-end distortion. This JSCC is based on quantized frame expansions to introduce redundancy into transmitted data. An algorithm has been developed to determine the optimal trade-off between redundancy and quantization rate, under a constraint on channel capacity. The proposed approach does not require the communication of any overhead data between the transmitter and receiver. When compared to codes commonly used to deal with packet losses, simulation results indicate that the proposed JSCC can achieve lower distortion for secondary user’s transmissions in cognitive radio systems.

Motivated by potential applications in wireless sensor networks, this thesis considers the problem of communicating a large number of correlated analog sources over a Gaussian multiple-access channel using non-orthogonal code-division multiple-access (CDMA). A joint source-channel decoder is presented which can exploit the inter-source correlation for interference reduction in the CDMA channel. This decoder uses a linear minimum mean square error (MMSE) multi-user detector (MUD) in tandem with a MMSE joint source decoder (JSD) for multiple sources to achieve a computational complexity that scales with the number of sources. The MUD and the JSD, then iteratively exchange extrinsic information to improve the interference cancellation. Experimental results show that, compared to a non-iterative decoder, the proposed iterative decoder is more robust against potential performance degradation due to correlated channel interference and offers better near far resistance.

With the increasing popularity and relevance of ad-hoc wireless sensor networks, cooperative transmission is more relevant than ever. In this thesis, we consider methods for optimization of cooperative transmission schemes in wireless sensor networks. We are in particular interested in communication schemes that can be used in applications that are critical to low-delays, such as networked control, and propose suitable candidates of joint source-channel coding schemes. We show that, in many cases, there are significant gains if the parts of the system are jointly optimized for the current source and channel. We especially focus on two means of cooperative transmission, namely distributed source coding and relaying. In the distributed source coding case, we consider transmission of correlated continuous sources and propose an algorithm for designing simple and energy-efficient sensor nodes. In particular the cases of the binary symmetric channel as well as the additive white Gaussian noise channel are studied. The system works on a sample by sample basis yielding a very low encoding complexity, at an insignificant delay. Due to the source correlation, the resulting quantizers use the same indices for several separated intervals in order to reduce the quantization distortion. For the case of relaying, we study the transmission of a continuous Gaussian source and the transmission of an uniformly distributed discrete source. In both situations, we propose design algorithms to design low-delay source-channel and relay mappings. We show that there can be significant power savings if the optimized systems are used instead of more traditional systems. By studying the structure of the optimized source-channel and relay mappings, we provide useful insights on how the optimized systems work. Interestingly, the design algorithm generally produces relay mappings with a structure that resembles Wyner-Ziv compression.

A converse method is developed for the source broadcast problem. Specifically, it is shown that the separation architecture is optimal for a variant of the source broadcast problem and the associated source-channel separation theorem can be leveraged, via a reduction argument, to establish a necessary condition for the original problem, which uni es several existing results in the literature. Somewhat surprisingly, this method, albeit based on the source-channel separation theorem, can be used to prove the optimality of non-separation based schemes and determine the performance limits in certain scenarios where the separation architecture is suboptimal. / Thesis / Master of Applied Science (MASc)

Efficient multimedia communication over mobile or wireless channels remains a challenging problem. To deal with that problem so far, the industry has followed mostly a divide and conquer approach, by considering separately the source of data (text, image, video, etc.) and the communication channel (electromagnetic waves across the air, a telephone line, a coaxial cable, etc.). The goal is always the same: to transmit (or store) more data reliably per unit of time, of energy, of physical medium, etc. With today's applications, the divide and conquer approach has, in a sense, started to show its limits. Let us consider, for example, the digital transmission of an image. At the transmitter, the first main step is data compression, at the source level. The number of bits that are necessary to represent the image with a given level of quality is reduced, usually by removing details in the image that are invisible (or less visible) to the human eye. The second main step is data protection, at the channel level. The transmission is made ideally resistant to deteriorations caused by the channel, by implementing techniques such as time/frequency/space expansions. In a sense, the two steps are quite antagonistic --- we first compress then expand the original signal --- and have different goals --- compression enables to transfer more data per unit of time/energy/medium while protection enables to transfer data reliably. At the receiver, the "reversed" operations are implemented. This separation in two steps dates back to Shannon's source and channel coding separation theorem in 1948 and has encouraged the division of the research community in two groups, one focusing on data compression, the other on data protection. This separation has also seduced the industry for the design, thereby supported by theory, of layered communication protocols. But this theorem holds only under asymptotic conditions that are rarely satisfied with today's multimedia content and mobile channels. Therefore, it is usually wise in practice to drop this strict separation and to allow at least some cross-layer cooperation between the source and channel layers. This is what lies behind the words joint source-channel techniques. As the name suggests, these techniques are optimized jointly, without a strict separation. Intuitively, since the optimization is less constrained from a mathematical standpoint, the solution can only be better or equivalent. In this thesis, we investigate a promising subset of these techniques, based on the turbo principle and on variable length codes. The potential of this subset has been illustrated for the first time in 2000, with an example that, since then, has been successfully improved in several directions. Unfortunately, most decoding algorithms have been so far developed on an ad hoc basis, without a unified view and often without specifying the approximations made. Besides, most code-related conclusions are based on simulations or on extrinsic information analysis. A theoretical framework on the error correcting properties of variable length codes in turbo systems is lacking. The purpose of this work, in three parts, is to fill in these gaps up to a certain extent. The first part presents the literature in this field and attempts to give a unified overview. The second part proposes a transmission system that generalizes previous systems from the literature, with the simple addition of a repetition code. While most previous systems are designed for bit streams with a high level of residual redundancy, the proposed system has the interesting flexibility to handle easily different levels of redundancy. Its performance is then analyzed for small levels of redundancy, which is a case not tackled extensively in the literature. This analysis leads notably to the discovery of surprising interleaving gains with reversible variable length codes. The third part develops the mathematical framework that was motivated during the second part but skipped on purpose for the sake of clarity. We first clarify several issues that arise with non-uniform bits and the extrinsic information charts, and propose and discuss two methods to compute these charts. Next, several theoretical results are stated on the robustness of variable length codes concatenated with linear error correcting codes. Notably, an approximate average distance spectrum of the concatenated code is rigorously developed. Together with the union bound, this spectrum provides upper bounds on the symbol and frame/packet error rates. These bounds are then analyzed from an interleaving gain standpoint and it is proved that the variable length code improves the interleaving gain if its spectrum is bounded.

The aim of source coding is to represent information as accurately as possible using as few bits as possible and in order to do so redundancy from the source needs to be removed. The aim of channel coding is in some sense the contrary, namely to introduce redundancy that can be exploited to protect the information when being transmitted over a nonideal channel. Combining these two techniques leads to the area of joint source-channel coding which in general makes it possible to achieve a better performance when designing a communication system than in the case when source and channel codes are designed separately. In this thesis four particular areas in joint source-channel coding are studied: analog (i.e. continuous) bandwidth expansion, distributed source coding over noisy channels, multiple description coding (MDC) and soft decoding. A general analog bandwidth expansion code based on orthogonal polynomials is proposed and analyzed. The code has a performance comparable with other existing schemes. However, the code is more general in the sense that it is implementable for a larger number of source distributions. The problem of distributed source coding over noisy channels is studied. Two schemes are proposed and analyzed for this problem which both work on a sample by sample basis. The first code is based on scalar quantization optimized for a certain channel characteristics. The second code is nonlinear and analog. Two new MDC schemes are proposed and investigated. The first is based on sorting a frame of samples and transmitting, as side-information/redundancy, an index that describes the resulting permutation. In case that some of the transmitted descriptors are lost during transmission this side information (if received) can be used to estimate the lost descriptors based on the received ones. The second scheme uses permutation codes to produce different descriptions of a block of source data. These descriptions can be used jointly to estimate the original source data. Finally, also the MDC method multiple description coding using pairwise correlating transforms as introduced by Wang et al. is studied. A modi fication of the quantization in this method is proposed which yields a performance gain. A well known result in joint source-channel coding is that the performance of a communication system can be improved by using soft decoding of the channel output at the cost of a higher decoding complexity. An alternative to this is to quantize the soft information and store the pre-calculated soft decision values in a lookup table. In this thesis we propose new methods for quantizing soft channel information, to be used in conjunction with soft-decision source decoding. The issue on how to best construct finite-bandwidth representations of soft information is also studied. / QC 20100920