abstract: Fundamental limits of fixed-to-variable (F-V) and variable-to-fixed (V-F) length universal source coding at short blocklengths is characterized. For F-V length coding, the Type Size (TS) code has previously been shown to be optimal up to the third-order rate for universal compression of all memoryless sources over finite alphabets. The TS code assigns sequences ordered based on their type class sizes to binary strings ordered lexicographically. Universal F-V coding problem for the class of first-order stationary, irreducible and aperiodic Markov sources is first considered. Third-order coding rate of the TS code for the Markov class is derived. A converse on the third-order coding rate for the general class of F-V codes is presented which shows the optimality of the TS code for such Markov sources. This type class approach is then generalized for compression of the parametric sources. A natural scheme is to define two sequences to be in the same type class if and only if they are equiprobable under any model in the parametric class. This natural approach, however, is shown to be suboptimal. A variation of the Type Size code is introduced, where type classes are defined based on neighborhoods of minimal sufficient statistics. Asymptotics of the overflow rate of this variation is derived and a converse result establishes its optimality up to the third-order term. These results are derived for parametric families of i.i.d. sources as well as Markov sources. Finally, universal V-F length coding of the class of parametric sources is considered in the short blocklengths regime. The proposed dictionary which is used to parse the source output stream, consists of sequences in the boundaries of transition from low to high quantized type complexity, hence the name Type Complexity (TC) code. For large enough dictionary, the $\epsilon$-coding rate of the TC code is derived and a converse result is derived showing its optimality up to the third-order term. / Dissertation/Thesis / Doctoral Dissertation Electrical Engineering 2018

In practice, a communication system works with finite blocklength codes because of the delay constraints and the information-theoretic bounds which are proposed for finite blocklength systems can be exploited to determine the performance of a designed system. In this thesis, achievable rates for given average error probabilities are considered for finite blocklength systems. Although classical bounds can be used to upper bound the error probability, these bounds require the optimization of auxiliary variables. In this work, a bound which is called the dependence testing (DT) bound that is free of any auxiliary variables is exploited. The DT bound is evaluated by introducing a normal approximation to the information density. Simulations carried out both for the Gaussian and discrete input alphabets show the proposed approximation enables very good prediction of the achievable rates. The proposed approximation is also used to calculate the average error probability for block fading channels. Simulations performed for Rayleigh block fading channels demonstrate that the total blocklength of the system in addition to the number of fading blocks should be accounted for especially when the number of fading blocks is large. A power allocation problem in block fading channels when the channel state information is available to the transmitting side is investigated in the final part of this work. The DT bound is optimized for a given channel state vector by allocating different power levels to each fading block by exploiting short-term power allocation. A simple power allocation algorithm is proposed which comes out with very similar results compared with the analytically computed values.

With the ever-growing demand for higher data rate in next generation communication systems, researchers are pushing the limits of the existing architecture. Due to the stochastic nature of communication channels, most systems use some form of adaptive methods to adjust the transmitting parameters and allocation of resources in order to overcome channel variations and achieve optimal throughput. We will study four cases of adaptive transmission and dynamic resource allocation in collaborative systems that are practically significant. Firstly, we study hybrid automatic repeat request (HARQ) techniques that are widely used to handle transmission failures. We propose HARQ policies that improve system throughput and are suitable for point-to-point, two-hop relay, and multi-user broadcast systems. Secondly, we study the effect of having bits of mixed SNR qualities in finite length codewords. We prove that by grouping bits according to their reliability so that each codeword contains homogeneous bit qualities, the finite blocklength capacity of the system is increased. Thirdly, we study the routing and resource allocation problem in multiple collaborative networks. We propose an algorithm that enables collaboration between networks which needs little to no side information shared across networks, but rather infers necessary information from the transmissions. The collaboration between networks provides a significant gain in overall throughput compared to selfish networks. Lastly, we present an algorithm that allocates disjoint transmission channels for our cognitive radio network in the DARPA Spectrum Collaboration Challenge (SC2). This algorithm uses the real-time spectrogram knowledge perceived by the radios and allocates channels adaptively in a crowded spectrum shared with other collaborative networks.