Consider the problem of designing capacity-achieving codes for multi-terminal communication scenarios. For point-to-point communication problems, one can optimize a single code to approach capacity, but for multi-terminal problems this translates to optimizing a single code to perform well over the entire region of channel parameters. A coding scheme is called universal if it allows reliable communication over the entire achievable region promised by information theory. It was recently shown that terminated low-density parity-check convolutional codes (also known as spatially-coupled low-density parity-check ensembles) have belief-propagation thresholds that approach their maximum a-posteriori thresholds. This phenomenon, called "threshold saturation via spatial-coupling," was proven for binary erasure channels and then for binary memoryless symmetric channels. This approach provides us with a new paradigm for constructing capacity approaching codes. It was also conjectured that the principle of spatial coupling is very general and that the phenomenon of threshold saturation applies to a very broad class of graphical models. In this work, we consider a noisy Slepian-Wolf problem (with erasure and binary symmetric channel correlation models) and the binary-input Gaussian multiple access channel, which deal with correlation between sources and interference at the receiver respectively. We derive an area theorem for the joint decoder and empirically show that threshold saturation occurs for these multi-user scenarios. We also show that the outer bound derived using the area theorem is tight for the erasure Slepian-Wolf problem and that this bound is universal for regular LDPC codes with large left degrees. As a result, we demonstrate near-universal performance for these problems using spatially-coupled coding systems.

The hard-disk storage industry is at a critical time as the current technologies are incapable of achieving densities beyond 500 Gb/in2, which will be reached in a few years. Many radically new storage architectures have been proposed, which along with advanced signal processing algorithms are expected to achieve much higher densities. In this dissertation, various signal processing algorithms are developed to improve the performance of current and next-generation magnetic storage systems.Low-density parity-check (LDPC) error correction codes are known to provide excellent performance in magnetic storage systems and are likely to replace or supplement currently used algebraic codes. Two methods are described to improve their performance in such systems. In the first method, the detector is modified to incorporate auxiliary LDPC parity checks. Using graph theoretical algorithms, a method to incorporate maximum number of such checks for a given complexity is provided. In the second method, a joint detection and decoding algorithm is developed that, unlike all other schemes, operates on the non-binary channel output symbols rather than input bits. Though sub-optimal, it is shown to provide the best known decoding performance for channels with memory more than 1, which are practically the most important.This dissertation also proposes a ternary magnetic recording system from a signal processing perspective. The advantage of this novel scheme is that it is capable of making magnetic transitions with two different but predetermined gradients. By developing optimal signal processing components like receivers, equalizers and detectors for this channel, the equivalence of this system to a two-track/two-head system is determined and its performance is analyzed. Consequently, it is shown that it is preferable to store information using this system, than to store using a binary system with inter-track interference. Finally, this dissertation provides a number of insights into the unique characteristics of heat-assisted magnetic recording (HAMR) and two-dimensional magnetic recording (TDMR) channels. For HAMR channels, the effects of laser spot on transition characteristics and non-linear transition shift are investigated. For TDMR channels, a suitable channel model is developed to investigate the two-dimensional nature of the noise.

The unifying theme of this dissertation is the development of new solutions for decoding and inference problems based on convex optimization methods. Th first part considers the joint detection and decoding problem for low-density parity-check (LDPC) codes on finite-state channels (FSCs). Hard-disk drives (or magnetic recording systems), where the required error rate (after decoding) is too low to be verifiable by simulation, are most important applications of this research. Recently, LDPC codes have attracted a lot of attention in the magnetic storage industry and some hard-disk drives have started using iterative decoding. Despite progress in the area of reduced-complexity detection and decoding algorithms, there has been some resistance to the deployment of turbo-equalization (TE) structures (with iterative detectors/decoders) in magnetic-recording systems because of error floors and the difficulty of accurately predicting performance at very low error rates. To address this problem for channels with memory, such as FSCs, we propose a new decoding algorithms based on a well-defined convex optimization problem. In particular, it is based on the linear-programing (LP) formulation of the joint decoding problem for LDPC codes over FSCs. It exhibits two favorable properties: provable convergence and predictable error-floors (via pseudo-codeword analysis). Since general-purpose LP solvers are too complex to make the joint LP decoder feasible for practical purposes, we develop an efficient iterative solver for the joint LP decoder by taking advantage of its dual-domain structure. The main advantage of this approach is that it combines the predictability and superior performance of joint LP decoding with the computational complexity of TE. The second part of this dissertation considers the matrix completion problem for the recovery of a data matrix from incomplete, or even corrupted entries of an unknown matrix. Recommender systems are good representatives of this problem, and this research is important for the design of information retrieval systems which require very high scalability. We show that our IMP algorithm reduces the well-known cold-start problem associated with collaborative filtering systems in practice.

abstract: There are many wireless communication and networking applications that require high transmission rates and reliability with only limited resources in terms of bandwidth, power, hardware complexity etc.. Real-time video streaming, gaming and social networking are a few such examples. Over the years many problems have been addressed towards the goal of enabling such applications; however, significant challenges still remain, particularly, in the context of multi-user communications. With the motivation of addressing some of these challenges, the main focus of this dissertation is the design and analysis of capacity approaching coding schemes for several (wireless) multi-user communication scenarios. Specifically, three main themes are studied: superposition coding over broadcast channels, practical coding for binary-input binary-output broadcast channels, and signalling schemes for two-way relay channels. As the first contribution, we propose an analytical tool that allows for reliable comparison of different practical codes and decoding strategies over degraded broadcast channels, even for very low error rates for which simulations are impractical. The second contribution deals with binary-input binary-output degraded broadcast channels, for which an optimal encoding scheme that achieves the capacity boundary is found, and a practical coding scheme is given by concatenation of an outer low density parity check code and an inner (non-linear) mapper that induces desired distribution of "one" in a codeword. The third contribution considers two-way relay channels where the information exchange between two nodes takes place in two transmission phases using a coding scheme called physical-layer network coding. At the relay, a near optimal decoding strategy is derived using a list decoding algorithm, and an approximation is obtained by a joint decoding approach. For the latter scheme, an analytical approximation of the word error rate based on a union bounding technique is computed under the assumption that linear codes are employed at the two nodes exchanging data. Further, when the wireless channel is frequency selective, two decoding strategies at the relay are developed, namely, a near optimal decoding scheme implemented using list decoding, and a reduced complexity detection/decoding scheme utilizing a linear minimum mean squared error based detector followed by a network coded sequence decoder. / Dissertation/Thesis / Ph.D. Electrical Engineering 2011

La généralisation de l’usage des systèmes de dialogue homme-machine accroît la nécessité du développement rapide des différents composants de ces systèmes. Les systèmes de dialogue peuvent être conçus pour différents domaines d’application et dans des langues différentes. La nécessité d’une production rapide pour de nouvelles langues reste un problème ouvert et crucial auquel il est nécessaire d’apporter des solutions efficaces.Nos travaux s’intéressent particulièrement au module de compréhension de la parole et proposent des approches pour la portabilité rapide peu coûteuse de ce module.Les méthodes statistiques ont montré de bonnes performances pour concevoir les modules de compréhension de la parole pour l’étiquetage sémantique de tours de dialogue.Cependant ces méthodes nécessitent de larges corpus pour être apprises. La collecte de ces corpus est aussi coûteuse en temps et en expertise humaine.Dans cette thèse, nous proposons plusieurs approches pour porter un système de compréhension d’une langue vers une autre en utilisant les techniques de la traduction automatique. Les premiers travaux consistent à appliquer la traduction automatique à plusieurs niveaux du processus de portabilité du système de compréhension afin de réduire le coût lié à production de nouvelles données d’apprentissage. Les résultats expérimentaux montrent que l’utilisation de la traduction automatique permet d’obtenir des systèmes performant avec un minimum de contribution humaine.Cette thèse traite donc à la fois de la traduction automatique et de la compréhension de la parole. Nous avons effectué une comparaison approfondie entre les méthodes utilisées pour chacune des tâches et nous avons proposé un décodage conjoint basé sur une méthode discriminante qui à la fois traduit une phrase et lui attribue ses étiquettes sémantiques. Ce décodage est obtenu par une approche à base de graphe qui permet de composer un graphe de traduction avec un graphe de compréhension. Cette représentation peut être généralisée pour permettre des transmissions d’informations riches entre les composants du système de dialogue / The generalisation of human-machine dialogue system increases the need for a rapid development of the various components of these systems. Dialogue systems can be designed for different applications and in different languages. The need for a fast production of systems for new languages ​​is still an open and crucial issue which requires effective solutions. Our work is particularly interested in speech understanding module and propose approaches for language portability of this module. The statistical methods showed good performance to design modules for speech understanding. However, these methods require large corpora to be trained. The collection of these corpora is expensive in time and human expertise. In this thesis, we propose several approaches to port an understanding system from one language to another using machine translation techniques. The experimental results show that the use of machine translation allows to produce efficient systems with minimal human effort. This thesis addresses both machine translation and speech understanding domain. We conducted a comparison between the methods used for each task and we have proposed a joint decoding between translation and understanding based on a discriminant method. This decoding is achieved by a graph-based approach which allows to compose a translation graph with an understanding graph. This representation can be generalized to allow a rich transmission of information between the components of the dialogue system