In network information theory, the relay channel models a communication scenario where there is one or more relay nodes that can help the information transmission between the source and the destination. Although the capacity of the relay channel is still unknown even in the single-relay case, two fundamentally different relay schemes have been developed by (Cover and El Gamal, 1979) for such channels, which, depending on whether the relay decodes the information or not, are generally known as Decode-and-Forward (D-F) and Compress-and-Forward (C-F). In the D-F relay scheme, the relay first decodes the message sent by the source and then forwards it to the destination, and the destination decodes the message taking into account the inputs of both the source and the relay. In contrast, the C-F relay scheme is used when the relay cannot decode the message sent by the source, but still can help by compressing its observation into some compressed version, and forwarding this compression into the destination; the destination then either successively or jointly decodes the compression of the relay's observation and the original message of the source. For the single-relay case, it is known that joint compression-message decoding, although providing more freedom in choosing the compression at the relay, cannot achieve higher rates for the original message than successive decoding. This thesis addresses some fundamental issues in generalizing and unifying the above D-F and C-F relay schemes to the multiple-relay case. We first generalize the C-F scheme to multiple-relay channels, and investigate the question of whether compression-message joint decoding can improve the achievable rate compared to successive decoding in the multiple-relay case. It is demonstrated that in the case of multiple relays, there is no improvement on the achievable rate by joint decoding either. More interestingly, it is discovered that any compressions not supporting successive decoding will actually lead to strictly lower achievable rates for the original message. Therefore, to maximize the achievable rate for the original message, the compressions should always be chosen to support successive decoding. Furthermore, it is shown that any compressions not completely decodable even with joint decoding will not provide any contribution to the decoding of the original message. We also develop a new C-F relay scheme with block-by-block backward decoding. This new scheme improves the original C-F relay scheme to achieve higher rates in the multiple-relay case as the recently proposed noisy network coding scheme. However, compared to noisy network coding which uses repetitive encoding/all blocks united decoding, our new coding scheme is not only simpler, but also reveals the essential reason for the improvement of the achievable rate, that is, delayed decoding until all the blocks have been finished. Finally, to allow each relay node the freedom of choosing either the D-F or C-F relay strategy, we propose a unified relay framework, where both the D-F and C-F strategies can be employed simultaneously in the network. This framework employs nested blocks combined with backward decoding to allow for the full incorporation of the best known D-F and C-F relay strategies. The achievable rates under our unified relay framework are found to combine both the best known D-F and C-F achievable rates and include them as special cases. It is also demonstrated through a Gaussian network example that our achievable rates are generally better than the rates obtained with existing unified schemes and with D-F or C-F alone.

Smart phones and tablet computers have greatly boosted the demand for services via wireless access points, keeping constant pressure on the network providers to deliver vast amounts of data over the wireless infrastructure. To enlarge coverage and enhance throughput, relaying has been adopted in the new generation of wireless communication systems, such as in the Long-Term Evolution Advanced standard,  and will continue to play an important role in the next generation wireless infrastructure. Depending on functionality, relaying can be characterizing into three main categories: amplify-and-forward (AF), compression-and-forward (CF), and decode-and-forward (DF).  In this thesis, we investigate different cooperative strategies in wireless networks when relaying is in use. We first investigate  the capacity outer and inner bounds for a wireless multicast relay network where two sources, connected by error-free backhaul, multicast to two destinations with the help of a full-duplex relay node.  For high-rate backhaul scenarios, we find the exact cut-set bound of the capacity region by extending the proof of the converse for the Gaussian relay channel. For low-rate backhaul scenarios, we present two genie-aided outer bounds by extending the previous proof and introducing two lemmas on conditional (co-)variance. Our inner bounds are derived from various cooperative strategies by combining DF/CF/AF relaying with network coding schemes. We also extend the noisy network coding scheme and the short-message noisy network coding approach to correlated sources. For low-rate backhaul, we propose a new coding scheme, partial-decode-and-forward based linear network coding. We derive the achievable rate regions  for these schemes and measure the performance in term of achievable rates over Gaussian channels. By numerical investigation we observe significant gains over benchmark schemes and demonstrate that the gap between upper and lower bounds is in general not large. We also show that for high-rate backhaul, the cut-set bound can be achieved  when the signal-to-noise ratios lie in the sphere defined by the source-relay and relay-destination channel gains. For wireless networks with independent noise, we propose a simple framework to get capacity outer and inner bounds based on the ``one-shot'' bounding models. We first extend the models for two-user broadcast channels to many-user scenarios and then establish the gap between upper and lower bounding models. For networks with coupled links, we propose  a channel decoupling method which can decompose the network into overlapping multiple-access channels and broadcast channels.  We then apply the one-shot models and create an upper bounding network with only  bit-pipe connections. When developing the lower bounding network, we propose a  two-step update of these models for each coupled broadcast and multiple-access channels. We demonstrate by some examples that the resulting upper bound is in general very good and the gap between the upper and lower bounds is usually not large. For relay-aided downlink scenarios, we propose a cooperation scheme by cancelling interference at the transmitter. It is indeed a symbol-by-symbol approach to one-dimension dirty paper coding (DPC). For finite-alphabet signaling and interference, we derive the optimal (in terms of maximum mutual information) modulator under a given power constraint. A sub-optimal modulator is also proposed by formulating an optimization problem that maximizes the minimum distance of the signal constellation, and this non-convex optimization problem is approximately solved by semi-definite relaxation.  Bit-level simulation shows that the optimal and sub-optimal modulators can achieve significant gains over the Tomlinson-Harashima precoder (THP) benchmark and over non-DPC reference schemes, especially when the power of the interference is larger than the power of the noise. / <p>QC 20121015</p>

Despite much research on the throughput of relaying networks under idealized interference models, many practical wireless networks rely on physical-layer protocols that preclude the concurrent reception of multiple transmissions. In this work, we develop analytical frameworks for the uplink of a multi-source single-channel relay-aided wireless system where transmissions are scheduled to avoid collisions. We study amplify-and-forward and decode-and-forward strategies, in both time-sharing and network-coded variants, and provide mathematical models to investigate their achievable rate regions. Both general and optimal power allocations are considered. We also find the cut-set outer bounds for the rate regions. Moreover, we present a comparison between these methods with the simple time sharing scheme. Our numerical results reveal that optimizing power allocation favors the time sharing scheme significantly more than it does the relaying schemes. The proposed analysis provides means to quantitatively evaluate the efficacy of relaying under the collision model, leading to pragmatic design guidelines.

Despite much research on the throughput of relaying networks under idealized interference models, many practical wireless networks rely on physical-layer protocols that preclude the concurrent reception of multiple transmissions. In this work, we develop analytical frameworks for the uplink of a multi-source single-channel relay-aided wireless system where transmissions are scheduled to avoid collisions. We study amplify-and-forward and decode-and-forward strategies, in both time-sharing and network-coded variants, and provide mathematical models to investigate their achievable rate regions. Both general and optimal power allocations are considered. We also find the cut-set outer bounds for the rate regions. Moreover, we present a comparison between these methods with the simple time sharing scheme. Our numerical results reveal that optimizing power allocation favors the time sharing scheme significantly more than it does the relaying schemes. The proposed analysis provides means to quantitatively evaluate the efficacy of relaying under the collision model, leading to pragmatic design guidelines.

This work is motivated from error-correcting codes in the brain. To counteract the effect of representation noise, a large number of neurons participate in encoding even low-dimensional variables. In many brain areas, the mean firing rates of neurons as a function of represented variable, called the tuning curve, have unimodal shape centered at different values, defining a unary code. This dissertation focuses on a new type of neural code where neurons have periodic tuning curves, with a diversity of periods. Neurons that exhibit this tuning are grid cells of the entorhinal cortex, which represent self-location in two-dimensional space. First, we investigate mutual information between such multi-scale codes and the coded variable as a function of tuning curve width. For decoding, we consider maximum likelihood (ML) and plausible neural network (NN) based models. For unary neural codes, Fisher information increases with narrower tuning, regardless of the decoding method. By contrast, for the multi-scale neural code, the optimal tuning curve width depends on the decoding method. While narrow tuning is optimal for ML decoding, a finite width, matched to statistics of the noise, is optimal with a NN decoder. This finding may explain why actual neural tuning curves have relatively wide tuning. Next, motivated by the observation that multi-scale codes involve non-trivial decoding, we examine a decoding algorithm based on belief propagation (BP) because BP promises certain gains in decoding efficiency. The decoding problem is first formulated as a subset selection problem on a graph and then approximately solved by BP. Even though the graph has many cycles, BP converges to a fixed point after few iterations. The mean square error of BP approaches to that of ML at high signal-to-noise ratios. Finally, using the multi-scale code, we propose a joint source-channel coding scheme that allows separate senders to transmit complementary information over additive Gaussian noise channels without cooperation. The receiver decodes one sender's codeword using the other as side information and achieves a lower distortion using the same number of transmissions. The proposed scheme offers a new framework to design distributed joint source-channel codes for continuous variables. / text

Cooperative communication is seen as a key concept to achieve ultra-reliable communication in upcoming fifth-generation mobile networks (5G). A promising cooperative communication concept is multiterminal source-channel coding, which attracted recent attention in the research community. This thesis lays theoretical foundations for understanding the performance of multiterminal source-channel codes in a vast variety of cooperative communication networks. To this end, we decouple the multiterminal source-channel code into a multiterminal source code and multiple point-to-point channel codes. This way, we are able to adjust the multiterminal source code to any cooperative communication network without modification of the channel codes. We analyse the performance in terms of the outage probability in two steps: at first, we evaluate the instantaneous performance of the multiterminal source-channel codes for fixed channel realizations; and secondly, we average the instantaneous performance over the fading process. Based on the performance analysis, we evaluate the performance of multiterminal source-channel codes in three cooperative communication networks, namely relay, wireless sensor, and multi-connectivity networks. For all three networks, we identify the corresponding multiterminal source code and analyse its performance by the rate region for binary memoryless sources. Based on the rate region, we derive the outage probability for additive white Gaussian noise channels with quasi-static Rayleigh fading. We find results for the exact outage probability in integral form and closed-form solutions for the asymptotic outage probability at high signal-to-noise ratio. The importance of our results is fourfold: (i) we give the ultimate performance limits of the cooperative communication networks under investigation; (ii) the optimality of practical schemes can be evaluated with respect to our results, (iii) our results are suitable for link-level abstraction which reduces complexity in network-level simulation; and (iv) our results demonstrate that all three cooperative communication networks are key technologies to enable 5G applications, such as device to device and machine to machine communications, internet of things, and internet of vehicles. In addition, we evaluate the performance improvement of multiterminal source-channel codes over other (non-)cooperative communications concepts in terms of the transmit power reduction given a certain outage probability level. Moreover, we compare our theoretical results to simulated frame-error-rates of practical coding schemes. Our results manifest the superiority of multiterminal source-channel codes over other (non-)cooperative communications concepts.

A converse method is developed for the source broadcast problem. Specifically, it is shown that the separation architecture is optimal for a variant of the source broadcast problem and the associated source-channel separation theorem can be leveraged, via a reduction argument, to establish a necessary condition for the original problem, which uni es several existing results in the literature. Somewhat surprisingly, this method, albeit based on the source-channel separation theorem, can be used to prove the optimality of non-separation based schemes and determine the performance limits in certain scenarios where the separation architecture is suboptimal. / Thesis / Master of Applied Science (MASc)

A large number of complex systems can be modelled as networks of interacting units. From a mathematical point of view the topology of such systems can be represented as graphs of which the nodes represent individual elements of the system and the edges interactions or relations between them. In recent years networks have become a principal tool for analyzing complex systems in many different fields. This thesis introduces an information theoretic approach for finding characteristic connectivity patterns of networks, also called network motifs. Network motifs are sometimes also referred to as basic building blocks of complex networks. Many real world networks contain a statistically surprising number of certain subgraph patterns called network motifs. In biological and technological networks motifs are thought to contribute to the overall function of the network by performing modular tasks such as information processing. Therefore, methods for identifying network motifs are of great scientific interest. In the prevalent approach to motif analysis network motifs are defined to be subgraphs that occur significantly more often in a network when compared to a null model that preserves certain features of the network. However, defining appropriate null models and sampling these has proven to be challenging. This thesis introduces an alternative approach to motif analysis which looks at motifs as regularities of a network that can be exploited to obtain a more efficient representation of the network. The approach is based on finding a subgraph cover that represents the network using minimal total information. Here, a subgraph cover is a set of subgraphs such that every edge of the graph is contained in at least one subgraph in the cover while the total information of a subgraph cover is the information required to specify the connectivity patterns occurring in the cover together with their position in the graph. The thesis also studies the connection between motif analysis and random graph models for networks. Developing random graph models that incorporate high densities of triangles and other motifs has long been a goal of network research. In recent years, two such model have been proposed . However, their applications have remained limited because of the lack of a method for fitting such models to networks. In this thesis, we address this problem by showing that these models can be formulated as ensembles of subgraph covers and that the total information optimal subgraph covers can be used to match networks with such models. Moreover, these models can be solved analytically for many of their properties allowing for more accurate modelling of networks in general. Finally, the thesis also analyzes the problem of finding a total information optimal subgraph cover with respect to its computational complexity. The problem turns out to be NP-hard hence, we propose a greedy heuristic for it. Empirical results for several real world networks from different fields are presented. In order to test the presented algorithm we also consider some synthetic networks with predetermined motif structure.

A wireless relay network consists of multiple source nodes, multiple destination nodes, and possibly many relay nodes in between to facilitate its transmission. It is clear that the performance of such networks highly depends on information for- warding strategies adopted at the relay nodes. This dissertation studies a particular information forwarding strategy called compute-and-forward. Compute-and-forward is a novel paradigm that tries to incorporate the idea of network coding within the physical layer and hence is often referred to as physical layer network coding. The main idea is to exploit the superposition nature of the wireless medium to directly compute or decode functions of transmitted signals at intermediate relays in a net- work. Thus, the coding performed at the physical layer serves the purpose of error correction as well as permits recovery of functions of transmitted signals. For the bidirectional relaying problem with Gaussian channels, it has been shown by Wilson et al. and Nam et al. that the compute-and-forward paradigm is asymptotically optimal and achieves the capacity region to within 1 bit; however, similar results beyond the memoryless case are still lacking. This is mainly because channels with memory would destroy the lattice structure that is most crucial for the compute-and-forward paradigm. Hence, how to extend compute-and-forward to such channels has been a challenging issue. This motivates this study of the extension of compute-and-forward to channels with memory, such as inter-symbol interference. The bidirectional relaying problem with parallel Gaussian channels is also studied, which is a relevant model for the Gaussian bidirectional channel with inter-symbol interference and that with multiple-input multiple-output channels. Motivated by the recent success of linear finite-field deterministic model, we first investigate the corresponding deterministic parallel bidirectional relay channel and fully characterize its capacity region. Two compute-and-forward schemes are then proposed for the Gaussian model and the capacity region is approximately characterized to within a constant gap. The design of coding schemes for the compute-and-forward paradigm with low decoding complexity is then considered. Based on the separation-based framework proposed previously by Tunali et al., this study proposes a family of constellations that are suitable for the compute-and-forward paradigm. Moreover, by using Chinese remainder theorem, it is shown that the proposed constellations are isomorphic to product fields and therefore can be put into a multilevel coding framework. This study then proposes multilevel coding for the proposed constellations and uses multistage decoding to further reduce decoding complexity.

A large number of complex systems can be modelled as networks of interacting units. From a mathematical point of view the topology of such systems can be represented as graphs of which the nodes represent individual elements of the system and the edges interactions or relations between them. In recent years networks have become a principal tool for analyzing complex systems in many different fields. This thesis introduces an information theoretic approach for finding characteristic connectivity patterns of networks, also called network motifs. Network motifs are sometimes also referred to as basic building blocks of complex networks. Many real world networks contain a statistically surprising number of certain subgraph patterns called network motifs. In biological and technological networks motifs are thought to contribute to the overall function of the network by performing modular tasks such as information processing. Therefore, methods for identifying network motifs are of great scientific interest. In the prevalent approach to motif analysis network motifs are defined to be subgraphs that occur significantly more often in a network when compared to a null model that preserves certain features of the network. However, defining appropriate null models and sampling these has proven to be challenging. This thesis introduces an alternative approach to motif analysis which looks at motifs as regularities of a network that can be exploited to obtain a more efficient representation of the network. The approach is based on finding a subgraph cover that represents the network using minimal total information. Here, a subgraph cover is a set of subgraphs such that every edge of the graph is contained in at least one subgraph in the cover while the total information of a subgraph cover is the information required to specify the connectivity patterns occurring in the cover together with their position in the graph. The thesis also studies the connection between motif analysis and random graph models for networks. Developing random graph models that incorporate high densities of triangles and other motifs has long been a goal of network research. In recent years, two such model have been proposed . However, their applications have remained limited because of the lack of a method for fitting such models to networks. In this thesis, we address this problem by showing that these models can be formulated as ensembles of subgraph covers and that the total information optimal subgraph covers can be used to match networks with such models. Moreover, these models can be solved analytically for many of their properties allowing for more accurate modelling of networks in general. Finally, the thesis also analyzes the problem of finding a total information optimal subgraph cover with respect to its computational complexity. The problem turns out to be NP-hard hence, we propose a greedy heuristic for it. Empirical results for several real world networks from different fields are presented. In order to test the presented algorithm we also consider some synthetic networks with predetermined motif structure.