Source coding, a central concept in information theory, is the study of encoding and decoding data. Depending on the topological structure of the sources, i.e. how the sources are connected with encoders, different rate distortion functions are used. In this thesis two different encoding schemes---distributed and decentralized---are discussed and compared with a benchmark (centralized) coding structure. Specifically, all structures for two and three sources are discussed and a special case for the multi-source (more than three sources) is calculated. This work gives a pathway to characterize the generalized multiterminal source coding systems by finding the difference in the rate distortion limits from the optimal centralized coding system. It is shown that in specific cases, some decentralized systems can achieve the Shannon lower bound in a high resolution regime. / Thesis / Master of Science (MSc)

Recent advances in micro-electro-mechanical systems (MEMS) fabrication have made it possible to construct miniature devices containing an embedded system with strong computing capabilities. New generations of low cost sensor nodes can be created small with powerful computing and sensing capabilities. The small sensor nodes together with distributed wireless networking techniques enable the creation of innovative self-organized and peer-to-peer large scale wireless sensor networks (WSNs). A coordinated network of sensor nodes can perform distributed sensing of environmental phenomena over large-scale physical spaces and enable reliable monitoring and control in various applications. WSNs provide bridges between the virtual world of information technology and the real physical world. They represent a fundamental paradigm shift from traditional inter-human personal communications to autonomous inter-device communications. This thesis investigates the problems of target detection and tracking in WSNs. WSNs have some unique advantages over traditional sensor networks. However, the severe scarcity of power, communication and computation resources imposes some major challenges on the design and applications of distributed protocols for WSNs. In particular, this thesis focuses on two aspects of remote target detection and tracking in WSNs: distributed source coding (DSC) and sensor node localization. The primary purpose is to improve the application performance while minimizing energy consumption and bandwidth overhead.

Driven by a host of emerging applications (e.g., sensor networks and wireless video), distributed source coding (i.e., Slepian-Wolf coding, Wyner-Ziv coding and various other forms of multiterminal source coding), has recently become a very active research area. In this thesis, we first design a practical coding scheme for the quadratic Gaussian Wyner-Ziv problem, because in this special case, no rate loss is suffered due to the unavailability of the side information at the encoder. In order to approach the Wyner-Ziv distortion limit D??W Z(R), the trellis coded quantization (TCQ) technique is employed to quantize the source X, and irregular LDPC code is used to implement Slepian-Wolf coding of the quantized source input Q(X) given the side information Y at the decoder. An optimal non-linear estimator is devised at the joint decoder to compute the conditional mean of the source X given the dequantized version of Q(X) and the side information Y . Assuming ideal Slepian-Wolf coding, our scheme performs only 0.2 dB away from the Wyner-Ziv limit D??W Z(R) at high rate, which mirrors the performance of entropy-coded TCQ in classic source coding. Practical designs perform 0.83 dB away from D??W Z(R) at medium rates. With 2-D trellis-coded vector quantization, the performance gap to D??W Z(R) is only 0.66 dB at 1.0 b/s and 0.47 dB at 3.3 b/s. We then extend the proposed Wyner-Ziv coding scheme to the quadratic Gaussian multiterminal source coding problem with two encoders. Both direct and indirect settings of multiterminal source coding are considered. An asymmetric code design containing one classical source coding component and one Wyner-Ziv coding component is first introduced and shown to be able to approach the corner points on the theoretically achievable limits in both settings. To approach any point on the theoretically achievable limits, a second approach based on source splitting is then described. One classical source coding component, two Wyner-Ziv coding components, and a linear estimator are employed in this design. Proofs are provided to show the achievability of any point on the theoretical limits in both settings by assuming that both the source coding and the Wyner-Ziv coding components are optimal. The performance of practical schemes is only 0.15 b/s away from the theoretical limits for the asymmetric approach, and up to 0.30 b/s away from the limits for the source splitting approach.

Following recent theoretical works on successive Wyner-Ziv coding, we propose a practical layered Wyner-Ziv video coder using the DCT, nested scalar quantiza- tion, and irregular LDPC code based Slepian-Wolf coding (or lossless source coding with side information at the decoder). Our main novelty is to use the base layer of a standard scalable video coder (e.g., MPEG-4/H.26L FGS or H.263+) as the decoder side information and perform layered Wyner-Ziv coding for quality enhance- ment. Similar to FGS coding, there is no performance di®erence between layered and monolithic Wyner-Ziv coding when the enhancement bitstream is generated in our proposed coder. Using an H.26L coded version as the base layer, experiments indicate that Wyner-Ziv coding gives slightly worse performance than FGS coding when the channel (for both the base and enhancement layers) is noiseless. However, when the channel is noisy, extensive simulations of video transmission over wireless networks conforming to the CDMA2000 1X standard show that H.26L base layer coding plus Wyner-Ziv enhancement layer coding are more robust against channel errors than H.26L FGS coding. These results demonstrate that layered Wyner-Ziv video coding is a promising new technique for video streaming over wireless networks. For scalable video transmission over the Internet and 3G wireless networks, we propose a system for receiver-driven layered multicast based on layered Wyner-Ziv video coding and digital fountain coding. Digital fountain codes are near-capacity erasure codes that are ideally suited for multicast applications because of their rate- less property. By combining an error-resilient Wyner-Ziv video coder and rateless fountain codes, our system allows reliable multicast of high-quality video to an arbi- trary number of heterogeneous receivers without the requirement of feedback chan- nels. Extending this work on separate source-channel coding, we consider distributed joint source-channel coding by using a single channel code for both video compression (via Slepian-Wolf coding) and packet loss protection. We choose Raptor codes - the best approximation to a digital fountain - and address in detail both encoder and de- coder designs. Simulation results show that, compared to one separate design using Slepian-Wolf compression plus erasure protection and another based on FGS coding plus erasure protection, the proposed joint design provides better video quality at the same number of transmitted packets.

Rapid advances in sensor technologies have fueled massive torrents of data streaming across networks. Such large volume of information, indeed, restricts the operational performance of data processing, causing inefficiency in sensing, computation, communication and control. Hence, classical data processing techniques need to be re-analyzed and re-designed prior to be applied to modern networked data systems. This thesis aims to understand and characterize fundamental principles and interactions in and among sensing, compression, communication, computation and control, involved in networked data systems. In this regard, the thesis investigates four problems. The common theme is the design and analysis of optimized low-delay transmission strategies with affordable complexity for reliable communication of acquired data over networks with the objective of providing high quality of service for users. In the first three problems considered in the thesis, an emerging framework for data acquisition, namely, compressed sensing, is used which performs acquisition and compression simultaneously. The first problem considers the design of iterative encoding schemes, based on scalar quantization, for transmission of compressed sensing measurements over rate-limited links. Our approach is based on an analysis-by-synthesis principle, where the motivation is to reflect non-linearity in reconstruction, raised by compressed sensing, via synthesis, on choosing the best quantized value for encoding, via analysis. Our design shows significant reconstruction performance compared to schemes that only consider direct quantization of compressed sensing measurements. In the second problem, we investigate the design and analysis of encoding--decoding schemes, based on vector quantization, for transmission of compressed sensing measurements over rate-limited noisy links. In so realizing, we take an approach adapted from joint source-channel coding framework. We show that the performance of the studied system can approach the fundamental theoretical bound by optimizing the encoder-decoder pair. The price, however, is increased complexity at the encoder. To address the encoding complexity of the vector quantizer, we propose to use low-complexity multi-stage vector quantizer whose optimized design shows promising performance. In the third problem considered in the thesis, we take one step forward, and study joint source-channel coding schemes, based on vector quantization, for distributed transmission of compressed sensing measurements over noisy rate-limited links. We design optimized distributed coding schemes, and analyze theoretical bounds for such topology. Under certain conditions, our results reveal that the performance of the optimized schemes approaches the analytical bounds. In the last problem and in the context of control under communication constraints, we bring the notion of system dynamicity into the picture. Particularly, we study relations among stability in dynamical networked control systems, performance of real-time coding schemes and the coding complexity. For this purpose, we take approaches adapted from separate source-channel coding, and derive theoretical bounds on the performance of two types of coding schemes: dynamic repetition codes, and dynamic Fountain codes. We analytically and numerically show that the dynamic Fountain codes, over binary-input symmetric channels, with belief propagation decoding, are able to provide system stability in a networked control system. The results in the thesis evidently demonstrate that impressive performance gain is feasible by employing tools from communication and information theory to control and sensing. The insights offered through the design and analysis will also reveal fundamental pieces for understanding real-world networked data puzzle. / <p>QC 20140407</p>

No description available.

With the progress of computer graphics and computer vision technologies, 3D/multiview video applications such as 3D-TV and tele-immersive conference become more and more popular and are very likely to emerge as a prime application in the near future. A successful 3D/multiview video system needs synergistic integration of various technologies such as 3D/multiview video acquisition, compression, transmission and rendering. In this thesis, we focus on addressing the challenges for multiview video compression. In particular, we have made 5 major contributions: (1) We propose a novel neighbor-based multiview video compression system which helps remove the inter-view redundancies among multiple video streams and improve the performance. An optimal stream encoding order algorithm is designed to enable the encoder to automatically decide the stream encoding order and find the best reference streams. (2) A novel multiview video transcoder is designed and implemented. The proposed multiview video transcoder can be used to encode multiple compressed video streams and reduce the cost of multiview video acquisition system. (3) A learning-based multiview video compression scheme is invented. The novel multiview video compression algorithms are built on the recent advances on semi-supervised learning algorithms and achieve compression by finding a sparse representation of images. (4) Two novel distributed source coding algorithms, EETG and SNS-SWC, are put forward. Both EETG and SNS-SWC are capable to achieve the whole Slepian-Wolf rate region and are syndrome-based schemes. EETG simplifies the code construction algorithm for distributed source coding schemes using extended Tanner graph and is able to handle mismatched bits at the encoder. SNS-SWC has two independent decoders and thus can simplify the decoding process. (5) We propose a novel distributed multiview video coding scheme which allows flexible rate allocation between two distributed multiview video encoders. SNS-SWC is used as the underlying Slepian-Wolf coding scheme. It is the first work to realize simultaneous Slepian-Wolf coding of stereo videos with the help of a distributed source code that achieves the whole Slepian-Wolf rate region. The proposed scheme has a better rate-distortion performance than the separate H.264 coding scheme in the high-rate case. / Computer Networks and Multimedia Systems

La séparation de sources est la tâche qui consiste à récupérer plusieurs signaux dont on observe un ou plusieurs mélanges. Ce problème est particulièrement difficile et de manière à rendre la séparation possible, toute information supplémentaire connue sur les sources ou le mélange doit pouvoir être prise en compte. Dans cette thèse, je propose un formalisme général permettant d’inclure de telles connaissances dans les problèmes de séparation, où une source est modélisée comme la réalisation d’un processus gaussien. L’approche a de nombreux intérêts : elle généralise une grande partie des méthodes actuelles, elle permet la prise en compte de nombreux a priori et les paramètres du modèle peuvent être estimés efficacement. Ce cadre théorique est appliqué à la séparation informée de sources audio, où la séparation est assistée d'une information annexe calculée en amont de la séparation, lors d’une phase préliminaire où à la fois le mélange et les sources sont disponibles. Pour peu que cette information puisse se coder efficacement, cela rend possible des applications comme le karaoké ou la manipulation des différents instruments au sein d'un mix à un coût en débit bien plus faible que celui requis par la transmission séparée des sources. Ce problème de la séparation informée s’apparente fortement à un problème de codage multicanal. Cette analogie permet de placer la séparation informée dans un cadre théorique plus global où elle devient un problème de codage particulier et bénéficie à ce titre des résultats classiques de la théorie du codage, qui permettent d’optimiser efficacement les performances. / Source separation consists in recovering different signals that are only observed through their mixtures. To solve this difficult problem, any available prior information about the sources must be used so as to better identify them among all possible solutions. In this thesis, I propose a general framework, which permits to include a large diversity of prior information into source separation. In this framework, the sources signals are modeled as the outcomes of independent Gaussian processes, which are powerful and general nonparametric Bayesian models. This approach has many advantages: it permits the separation of sources defined on arbitrary input spaces, it permits to take many kinds of prior knowledge into account and also leads to automatic parameters estimation. This theoretical framework is applied to the informed source separation of audio sources. In this setup, a side-information is computed beforehand on the sources themselves during a so-called encoding stage where both sources and mixtures are available. In a subsequent decoding stage, the sources are recovered using this information and the mixtures only. Provided this information can be encoded efficiently, it permits popular applications such as karaoke or active listening using a very small bitrate compared to separate transmission of the sources. It became clear that informed source separation is very akin to a multichannel coding problem. With this in mind, it was straightforwardly cast into information theory as a particular source-coding problem, which permits to derive its optimal performance as rate-distortion functions as well as practical coding algorithms achieving these bounds.

We consider the asymptotic minimum rate under the logarithmic loss distortion constraint. More specifically, we find the asymptotic minimum rate expression when given distortions get close to 0. The problem under consideration is separate encoding and joint decoding of correlated two information sources, subject to a logarithmic loss distortion constraint. We introduce a test channel, whose transition probability (conditional probability mass function) captures the encoding and decoding process. Firstly, we find the expression for the special case of doubly symmetric binary sources with binary-output test channels. Then the result is extended to the case where the test channels are arbitrary. When given distortions get close to 0, the asymptotic rate coincides with that for the aforementioned special case. Finally, we consider the general case and show that the key findings for the special case continue to hold. / Thesis / Master of Applied Science (MASc)

The realization of high data rate wireless communication and large-scale connectivity with seamless coverage has been enabled by the introduction of various advanced transmission technologies, such as multiple access (MAC) technology and relay-assisted communications. However, beyond the accurate representation and successful transmission of information, in many applications it is the semantic aspect of that information that is really of interest. This thesis makes contributions to both the technology of conventional wireless communications and the theory of semantic communication. The main work is summarized as follows: We first consider an uplink system with K single-antenna users and one base station equipped with a single antenna, where each user utilizes a binary constellation to carry data. By maximizing the minimum Euclidean distance of the received sum constellation, the optimal user constellations and sum constellation are obtained for K=3 users. Using the principle of lattice coding, that design is extended to the K-user case. In both settings, the sum constellation belongs to additively uniquely decomposable constellation group (AUDCG). That property enables us to reduce the maximum likelihood multi-user detector to a single-user quantization based receiver. The symbol error probability (SEP) formula is derived, showing that our proposed non-orthogonal multiple access (NOMA) scheme outperforms the existing time division multiple access (TDMA) designs for the same system. Our design also sheds light on the general complex constellation designs for the MAC channel with arbitrary user constellation size. Specifically, K-user constellations with any 2^Mk size can be obtained using combinations of the proposed binary constellations. Next we concentrate on a multi-hop relay network with two time slots, consisting of single-antenna source and amplify-and-forward relay nodes and a destination node with M antennas. We develop a novel uniquely-factorable constellation set (UFCS) based on a PSK constellation for such system to allow the source and relay nodes to transmit their own information concurrently at the symbol level. By taking advantage of the uniquely-factorable property, the optimal maximum likelihood (ML) detection was equivalently reduced to a symbol-by-symbol detection based on phase quantization. In addition, the SEP formula was given, while enable us to show that the diversity gain of the system is one. For semantic communication, a new source model is considered, which consists of an intrinsic state part and an extrinsic observation part. The intrinsic state corresponds to the semantic feature of the source. It is not observable, and can only be inferred from the extrinsic observation. As an instance of the general model, the case of Gaussian distributed extrinsic observations is studied, where we assume a linear relationship between the intrinsic and extrinsic parts. We derive the rate-distortion function (in both centralized encoding and distributed encoding) of semantic-aware source coding under quadratic distortion structure by converting the semantic distortion constraint of the source to a surrogate distortion constraint on the observations. With proposed AUDCG and UFCS-based designs, high data rates as well as low detection latency can be achieved. Our modulation division method will be one of the promising technologies for the next generation communication and the analysis of the source coding with semantic information constraints also provides some insights that will guide the future development of semantic communication systems. / Thesis / Doctor of Philosophy (PhD) / The proliferation of smart phones and electronic devices has spurred explosive growth in high-speed multimedia services over the next generation of wireless cellular networks. Indeed, high data rates and large-scale connectivity with seamless coverage are the dominant themes of wireless communication system design. Moreover, beyond the accurate representation and successful transmission of information, the interpretation of its meaning is being paid more attention nowadays, which requires the development of approaches to semantic communication. The goal of this thesis is to contribute to the development of both conventional and semantic communication systems. Two advanced transmission technologies, namely, multiple access and relay-assisted communications are considered. By taking advantage of the special structures of digital communication signals, new approaches to multiple access and relay-assisted communications are developed. These designs enable high data rates, while simultaneously facilitating low-latency detection. Since there has been very limited analysis of the source coding of a vector source subject to semantic information constraints, we also study the rate distortion to trade-off for vector sources in both the case of centralized encoding and the case of distributed encoding, and we establish some insights that will guide the future development of semantic communication systems.