This thesis studies cooperative sensing and transmission in the context ofwireless sensor networks (WSNs). We especially focus on two means of cooperative sensing and transmission, namely, distributed source coding and relaying. We consider systems where the usefulness of the measured data is dependent on how old the data is and we therefore need low-delay transmission schemes. At first sight, the low-delay criterion may seem to be of little relevance, but it is this aspect in particular that distinguishes this thesis from many of the existing communication theoretic results, which often are asymptotic in the block lengths. The thesis is composed of an introductory part, discussing the fundamentals of communication theory and how these are related to the requirements of WSNs, followed by a part where the results of the thesis are reported in Papers A-H. Papers A-D study different scenarios for distributed source-channel coding. In Paper A, we consider transmission of correlated continuous sources and propose an iterative algorithm for designing simple and energy-efficient sensor nodes. In particular the cases of the binary symmetric channel as well as the additive white Gaussian noise channel are studied. In Paper B, the work is extended to channels with interference and it is shown that also in this case there can be significant power savings by performing a joint optimization of the system.Papers C and D use a more structured approach and propose side-information-aware source-channel coding strategies using lattices and sinusoids. In Paper E, we apply the methods we have used in joint source-channel coding to the famous Witsenhausen counterexample. By using a relatively simple iterative algorithm, we are able to demonstrate the best numerical performance known to date. For the case of systems with relays, we study the transmission of a continuous Gaussian source and the transmission of an uniformly distributed discrete source. In both situations, we propose algorithms to design low-delay source-channel and relay mappings. By studying the structure of the optimized source-channel and relay mappings, we provide useful insights into how the optimized systems work. These results are reported in Papers F and G. In Paper H, we finally consider sum-MSE minimization for the Gaussian multiple-input, multiple-output broadcast channel. By using recently discovered properties of this problem, we derive a closed-form expression for the optimal power allocation in the two-user scenario and propose a conceptually simple and efficient algorithm that handles an arbitrary number of users. Throughout the thesis we show that there are significant gains if the parts of the system are jointly optimized for the source and channel statistics. All methods that are considered in this thesis yield very low coding and decoding delays. In general, nonlinear mappings outperform linear mappings for problems where there is side-information available. Another contribution of this thesis is visualization of numerically optimized systems that can be used as inspiration when structured low-delay systems are designed. / The author changed name from Johannes Karlsson to Johannes Kron in January 2011. QC 20110512

info:eu-repo/semantics/nonPublished

Cette thèse porte sur quelques problèmes de codage de source dans lesquels on souhaite préserver la confidentialité vis à vis d’une écoute du canal. Dans la première partie, nous fournissons des nouveaux résultats fondamentaux sur le codage de source pour la détection (utilisateur légitime) et la confidentialité (vis à vis d’une écoute du canal) en présence d'informations secondaires aux terminaux de réception. Nous proposons plusieurs nouveaux résultats d'optimisation de la région de débit-erreur-équivocation réalisable, et proposons des algorithmes pratiques pour obtenir des solutions aussi proches que possible de l'optimal, ce qui nécessite la conception de quantificateurs en présence d'un eavesdropper. Dans la deuxième partie, nous étudions le problème de l'estimation sécurisée dans un cadre d'utilité-confidentialité où l'utilisateur recherche soit à extraire les aspects pertinents de données complexes ou bien à les cacher vis à vis d'un eavesdropper potentiel. L'objectif est principalement axé sur l'élaboration d'un cadre général qui combine la théorie de l'information et la théorie de la communication, visant à fournir un nouvel outil pour la confidentialité dans les Smart Grids. D'un point de vue théorique, cette recherche a permis de quantifier les limites fondamentales et donc le compromis entre sécurité et performance (estimation / détection). / This thesis investigates source coding problems in which some secrecy should be ensured with respect to eavesdroppers. In the first part, we provide some new fundamental results on both detection and secrecy oriented source coding in the presence of side information at the receiving terminals. We provide several new results of optimality and single-letter characterization of the achievable rate-error-equivocation region, and propose practical algorithms to obtain solutions that are as close as possible to the optimal, which requires the design of optimal quantization in the presence of an eavesdropper In the second part, we study the problem of secure estimation in a utility-privacy framework where the user is either looking to extract relevant aspects of complex data or hide them from a potential eavesdropper. The objective is mainly centered on the development of a general framework that combines information theory with communication theory, aiming to provide a novel and powerful tool for security in Smart Grids. From a theoretical perspective, this research was able to quantify fundamental limits and thus the tradeoff between security and performance (estimation/detection).

The aim of source coding is to represent information as accurately as possible using as few bits as possible and in order to do so redundancy from the source needs to be removed. The aim of channel coding is in some sense the contrary, namely to introduce redundancy that can be exploited to protect the information when being transmitted over a nonideal channel. Combining these two techniques leads to the area of joint source–channel coding which in general makes it possible to achieve a better performance when designing a communication system than in the case when source and channel codes are designed separately. In this thesis two particular areas in joint source–channel coding are studied: multiple description coding (MDC) and soft decoding. Two new MDC schemes are proposed and investigated. The first is based on sorting a frame of samples and transmitting, as side-information/redundancy, an index that describes the resulting permutation. In case that some of the transmitted descriptors are lost during transmission this side information (if received) can be used to estimate the lost descriptors based on the received ones. The second scheme uses permutation codes to produce different descriptions of a block of source data. These descriptions can be used jointly to estimate the original source data. Finally, also the MDC method multiple description coding using pairwise correlating transforms as introduced by Wang et al is studied. A modification of the quantization in this method is proposed which yields a performance gain. A well known result in joint source–channel coding is that the performance of a communication system can be improved by using soft decoding of the channel output at the cost of a higher decoding complexity. An alternative to this is to quantize the soft information and store the pre-calculated soft decision values in a lookup table. In this thesis we propose new methods for quantizing soft channel information, to be used in conjunction with soft-decision source decoding. The issue on how to best construct finite-bandwidth representations of soft information is also studied. / QC 20101124

Ces dernières années, l'intérêt scientifique porté aux différents aspects des systèmes autonomes est en pleine croissance. Des voitures autonomes jusqu'à l'Internet des objets, il est clair que la capacité de systèmes à prendre des décision de manière autonome devient cruciale. De plus, ces systèmes opéreront avec des ressources limitées. Dans cette thèse, ces systèmes sont étudiés sous l'aspect de la théorie de l'information, dans l'espoir qu'une compréhension fondamentale de leurs limites et de leurs utilisations pourrait aider leur conception par les futures ingénieurs.Dans ce travail, divers problèmes de décision binaire distribuée et collaborative sont considérés. Deux participants doivent "déclarer" la mesure de probabilité de deux variables aléatoires, distribuées conjointement par un processus sans mémoire et désignées par $vct{X}^n=(X_1,dots,X_n)$ et $vct{Y}^n=(Y_1,dots,Y_n)$. Cette décision et prise entre deux mesures de probabilité possibles sur un alphabet fini, désignés $P_{XY}$ et $P_{bar{X}bar{Y}}$. Les prélèvements marginaux des variables aléatoires, $vct{X}^n$ et $vct{Y}^n$ sont supposés à être disponibles aux différents sites .Il est permis aux participants d'échanger des quantités limitées d'information sur un canal parfait avec un contraint de débit maximal. Durant cette thèse, la nature de cette communication varie. La communication unidirectionnelle est considérée d'abord, suivie par la considération de communication bidirectionnelle, qui permet des échanges interactifs entre les participants. / In recents years, interest has been growing in research of different autonomous systems. From the self-dring car to the Internet of Things (IoT), it is clear that the ability of automated systems to make autonomous decisions in a timely manner is crucial in the 21st century. These systems will often operate under stricts constains over their resources. In this thesis, an information-theoric approach is taken to this problem, in hope that a fundamental understanding of the limitations and perspectives of such systems can help future engineers in designing them.Throughout this thesis, collaborative distributed binary decision problems are considered. Two statisticians are required to declare the correct probability measure of two jointly distributed memoryless process, denoted by $vct{X}^n=(X_1,dots,X_n)$ and $vct{Y}^n=(Y_1,dots,Y_n)$, out of two possible probability measures on finite alphabets, namely $P_{XY}$ and $P_{bar{X}bar{Y}}$. The marginal samples given by $vct{X}^n$ and $vct{Y}^n$ are assumed to be available at different locations.The statisticians are allowed to exchange limited amounts of data over a perfect channel with a maximum-rate constraint. Throughout the thesis, the nature of communication varies. First, only unidirectional communication is allowed. Using its own observations, the receiver of this communication is required to first identify the legitimacy of its sender by declaring the joint distribution of the process, and then depending on such authentication it generates an adequate reconstruction of the observations satisfying an average per-letter distortion. Bidirectional communication is subsequently considered, in a scenario that allows interactive communication between the participants.

[pt] Códigos de permutação são uma interessante ferramenta matemática que pode ser empregada para construir tanto esquemas de compressão com perdas quanto esquemas de modulação em um sistema de transmissão digital. Códigos de permutação vetorial, uma extensão mais poderosa dos códigos de permutação escalar, foram recentemente introduzidos no contexto de compressão de fontes. Este trabalho apresenta novas contribuições a essa teoria e introduz os códigos de permutação vetorial no contexto de modulação. Para compressão de fontes, é demonstrado matematicamente que os códigos de permutação vetorial (VPC) têm desempenho assintótico idêntico ao do quantizador vetorial com restrição de entropia (ECVQ). Baseado neste desenvolvimento, é proposto um método eficiente para o projeto de VPC s. O bom desempenho dos códigos projetados com esse método é verificado através de resultados experimentais para as fontes uniforme e gaussiana: são exibidos VPC s cujo desempenho é semelhante ao do ECVQ e superior ao de sua versão escalar. Para o propósito de transmissão digital, é verificado que também a modulação baseada em códigos de permutação vetorial (VPM) possui desempenho superior ao de sua versão escalar. São desenvolvidas as expressões para o projeto ótimo de VPM, e um método é apresentado para detecção ótima de VPM em canais AWGN e com desvanecimento. / [en] Permutation codes are an interesting mathematical tool which can be used to devise both lossy compression schemes and modulation schemes for digital transmission systems. Vector permutation codes, a more powerful extension of scalar permutation codes, were recently introduced for the purpose of source compression. This work presents new contributions to this theory and also introduces vector permutation codes for the purpose of modulation. For source compression, it is proved that vector permutation codes (VPC) have an asymptotical performance equal to that of an entropy-constrained vector quantizer (ECVQ). Based on this development, an efficient method is proposed for VPC design. Experimental results for Gaussian and uniform sources show that the codes designed by this method have indeed a good performance: VPC s are exhibited whose performances are similar to that of ECVQ and superior to those of their scalar counterparts. In the context of digital transmission, it is verified that also vector permutation modulation (VPM) is superior in performance to scalar permutation modulation. Expressions are developed for the optimal design of VPM, and a method is presented for maximum-likelihood detection of VPM in AWGN and fading channels.

Research Doctorate - Doctor of Philosophy (PhD) / This thesis presents results on optimal coding and decoding of discrete-time stochastic signals, in the sense of minimizing a distortion metric subject to a constraint on the bit-rate and on the signal transfer function from source to reconstruction. The first (preliminary) contribution of this thesis is the introduction of new distortion metric that extends the mean squared error (MSE) criterion. We give this extension the name Weighted-Correlation MSE (WCMSE), and use it as the distortion metric throughout the thesis. The WCMSE is a weighted sum of two components of the MSE: the variance of the error component uncorrelated to the source, on the one hand, and the remainder of the MSE, on the other. The WCMSE can take account of signal transfer function constraints by assigning a larger weight to deviations from a target signal transfer function than to source-uncorrelated distortion. Within this framework, the second contribution is the solution of a family of feedback quantizer design problems for wide sense stationary sources using an additive noise model for quantization errors. These associated problems consist of finding the frequency response of the filters deployed around a scalar quantizer that minimize the WCMSE for a fixed quantizer signal-to-(granular)-noise ratio (SNR). This general structure, which incorporates pre-, post-, and feedback filters, includes as special cases well known source coding schemes such as pulse coded modulation (PCM), Differential Pulse-Coded Modulation (DPCM), Sigma Delta converters, and noise-shaping coders. The optimal frequency response of each of the filters in this architecture is found for each possible subset of the remaining filters being given and fixed. These results are then applied to oversampled feedback quantization. In particular, it is shown that, within the linear model used, and for a fixed quantizer SNR, the MSE decays exponentially with oversampling ratio, provided optimal filters are used at each oversampling ratio. If a subtractively dithered quantizer is utilized, then the noise model is exact, and the SNR constraint can be directly related to the bit-rate if entropy coding is used, regardless of the number of quantization levels. On the other hand, in the case of fixed-rate quantization, the SNR is related to the number of quantization levels, and hence to the bit-rate, when overload errors are negligible. It is shown that, for sources with unbounded support, the latter condition is violated for sufficiently large oversampling ratios. By deriving an upper bound on the contribution of overload errors to the total WCMSE, a lower bound for the decay rate of the WCMSE as a function of the oversampling ratio is found for fixed-rate quantization of sources with finite or infinite support. The third main contribution of the thesis is the introduction of the rate-distortion function (RDF) when WCMSE is the distortion metric, denoted by WCMSE-RDF. We provide a complete characterization for Gaussian sources. The resulting WCMSE-RDF yields, as special cases, Shannon's RDF, as well as the recently introduced RDF for source-uncorrelated distortions (RDF-SUD). For cases where only source-uncorrelated distortion is allowed, the RDF-SUD is extended to include the possibility of linear-time invariant feedback between reconstructed signal and coder input. It is also shown that feedback quantization schemes can achieve a bit-rate only 0.254 bits/sample above this RDF by using the same filters that minimize the reconstruction MSE for a quantizer-SNR constraint. The fourth main contribution of this thesis is to provide a set of conditions under which knowledge of a realization of the RDF can be used directly to solve encoder-decoder design optimization problems. This result has direct implications in the design of subband coders with feedback, as well as in the design of encoder-decoder pairs for applications such as networked control. As the fifth main contribution of this thesis, the RDF-SUD is utilized to show that, for Gaussian sta-tionary sources with memory and MSE distortion criterion, an upper bound on the information-theoretic causal RDF can be obtained by means of an iterative numerical procedure, at all rates. This bound is tighter than 0:5 bits/sample. Moreover, if there exists a realization of the causal RDF in which the re-construction error is jointly stationary with the source, then the bound obtained coincides with the causal RDF. The iterative procedure proposed here to obtain Ritc(D) also yields a characterization of the filters in a scalar feedback quantizer having an operational rate that exceeds the bound by less than 0:254 bits/sample. This constitutes an upper bound on the optimal performance theoretically attainable by any causal source coder for stationary Gaussian sources under the MSE distortion criterion.

In digital watermarking, a watermark is embedded into a covertext in such a way that the resulting watermarked signal is robust to certain distortion caused by either standard data processing in a friendly environment or malicious attacks in an unfriendly environment. The watermarked signal can then be used for different purposes ranging from copyright protection, data authentication,fingerprinting, to information hiding. In this thesis, digital watermarking will be investigated from both an information theoretic viewpoint and a numerical computation viewpoint. <br /><br /> From the information theoretic viewpoint, we first study a new digital watermarking scenario, in which watermarks and covertexts are generated from a joint memoryless watermark and covertext source. The configuration of this scenario is different from that treated in existing digital watermarking works, where watermarks are assumed independent of covertexts. In the case of public watermarking where the covertext is not accessible to the watermark decoder, a necessary and sufficient condition is determined under which the watermark can be fully recovered with high probability at the end of watermark decoding after the watermarked signal is disturbed by a fixed memoryless attack channel. Moreover, by using similar techniques, a combined source coding and Gel'fand-Pinsker channel coding theorem is established, and an open problem proposed recently by Cox et al is solved. Interestingly, from the sufficient and necessary condition we can show that, in light of the correlation between the watermark and covertext, watermarks still can be fully recovered with high probability even if the entropy of the watermark source is strictly above the standard public watermarking capacity. <br /><br /> We then extend the above watermarking scenario to a case of joint compression and watermarking, where the watermark and covertext are correlated, and the watermarked signal has to be further compressed. Given an additional constraint of the compression rate of the watermarked signals, a necessary and sufficient condition is determined again under which the watermark can be fully recovered with high probability at the end of public watermark decoding after the watermarked signal is disturbed by a fixed memoryless attack channel. <br /><br /> The above two joint compression and watermarking models are further investigated under a less stringent environment where the reproduced watermark at the end of decoding is allowed to be within certain distortion of the original watermark. Sufficient conditions are determined in both cases, under which the original watermark can be reproduced with distortion less than a given distortion level after the watermarked signal is disturbed by a fixed memoryless attack channel and the covertext is not available to the watermark decoder. <br /><br /> Watermarking capacities and joint compression and watermarking rate regions are often characterized and/or presented as optimization problems in information theoretic research. However, it does not mean that they can be calculated easily. In this thesis we first derive closed forms of watermarking capacities of private Laplacian watermarking systems with the magnitude-error distortion measure under a fixed additive Laplacian attack and a fixed arbitrary additive attack, respectively. Then, based on the idea of the Blahut-Arimoto algorithm for computing channel capacities and rate distortion functions, two iterative algorithms are proposed for calculating private watermarking capacities and compression and watermarking rate regions of joint compression and private watermarking systems with finite alphabets. Finally, iterative algorithms are developed for calculating public watermarking capacities and compression and watermarking rate regions of joint compression and public watermarking systems with finite alphabets based on the Blahut-Arimoto algorithm and the Shannon's strategy.

In digital watermarking, a watermark is embedded into a covertext in such a way that the resulting watermarked signal is robust to certain distortion caused by either standard data processing in a friendly environment or malicious attacks in an unfriendly environment. The watermarked signal can then be used for different purposes ranging from copyright protection, data authentication,fingerprinting, to information hiding. In this thesis, digital watermarking will be investigated from both an information theoretic viewpoint and a numerical computation viewpoint. <br /><br /> From the information theoretic viewpoint, we first study a new digital watermarking scenario, in which watermarks and covertexts are generated from a joint memoryless watermark and covertext source. The configuration of this scenario is different from that treated in existing digital watermarking works, where watermarks are assumed independent of covertexts. In the case of public watermarking where the covertext is not accessible to the watermark decoder, a necessary and sufficient condition is determined under which the watermark can be fully recovered with high probability at the end of watermark decoding after the watermarked signal is disturbed by a fixed memoryless attack channel. Moreover, by using similar techniques, a combined source coding and Gel'fand-Pinsker channel coding theorem is established, and an open problem proposed recently by Cox et al is solved. Interestingly, from the sufficient and necessary condition we can show that, in light of the correlation between the watermark and covertext, watermarks still can be fully recovered with high probability even if the entropy of the watermark source is strictly above the standard public watermarking capacity. <br /><br /> We then extend the above watermarking scenario to a case of joint compression and watermarking, where the watermark and covertext are correlated, and the watermarked signal has to be further compressed. Given an additional constraint of the compression rate of the watermarked signals, a necessary and sufficient condition is determined again under which the watermark can be fully recovered with high probability at the end of public watermark decoding after the watermarked signal is disturbed by a fixed memoryless attack channel. <br /><br /> The above two joint compression and watermarking models are further investigated under a less stringent environment where the reproduced watermark at the end of decoding is allowed to be within certain distortion of the original watermark. Sufficient conditions are determined in both cases, under which the original watermark can be reproduced with distortion less than a given distortion level after the watermarked signal is disturbed by a fixed memoryless attack channel and the covertext is not available to the watermark decoder. <br /><br /> Watermarking capacities and joint compression and watermarking rate regions are often characterized and/or presented as optimization problems in information theoretic research. However, it does not mean that they can be calculated easily. In this thesis we first derive closed forms of watermarking capacities of private Laplacian watermarking systems with the magnitude-error distortion measure under a fixed additive Laplacian attack and a fixed arbitrary additive attack, respectively. Then, based on the idea of the Blahut-Arimoto algorithm for computing channel capacities and rate distortion functions, two iterative algorithms are proposed for calculating private watermarking capacities and compression and watermarking rate regions of joint compression and private watermarking systems with finite alphabets. Finally, iterative algorithms are developed for calculating public watermarking capacities and compression and watermarking rate regions of joint compression and public watermarking systems with finite alphabets based on the Blahut-Arimoto algorithm and the Shannon's strategy.

Wireless multimedia sensor networks (WMSNs) are networks of interconnected devices that allow retrieving video and audio streams, still images, and scalar data from the environment. In a densely deployed WMSN, there exists correlation among the observations of camera sensors with overlapped coverage areas, which introduces substantial data redundancy in the network. In this dissertation, efficient communication schemes are designed for WMSNs by leveraging the correlation of visual information observed by camera sensors. First, a spatial correlation model is developed to estimate the correlation of visual information and the joint entropy of multiple correlated camera sensors. The compression performance of correlated visual information is then studied. An entropy-based divergence measure is proposed to predict the compression efficiency of performing joint coding on the images from correlated cameras. Based on the predicted compression efficiency, a clustered coding technique is proposed that maximizes the overall compression gain of the visual information gathered in WMSNs. The correlation of visual information is then utilized to design a network scheduling scheme to maximize the lifetime of WMSNs. Furthermore, as many WMSN applications require QoS support, a correlation-aware QoS routing algorithm is introduced that can efficiently deliver visual information under QoS constraints. Evaluation results show that, by utilizing the correlation of visual information in the communication process, the energy efficiency and networking performance of WMSNs could be improved significantly.