Measure of distance between two probability distributions plays a fundamental role in statistics and machine learning. Optimal Transport (OT) theory provides such distance. Recent advance in OT theory is a generalization of classical OT with entropy regularized, called entropic OT. Despite its convenience in computation, it still lacks theoretical support. In this thesis, we study the connection between entropic OT and Entropy Power Inequality (EPI). First, we prove an HWI-type inequality making use of the infinitesimal displacement convexity of OT map. Second, we derive two Talagrand-type inequalities using the saturation of EPI that corresponds to a numerical term in our expression. We evaluate for a wide variety of distributions this term whereas for Gaussian and i.i.d. Cauchy distributions this term is found in explicit form. We show that our results extend previous results of Gaussian Talagrand inequality for Sinkhorn distance to the strongly log-concave case. Furthermore, we observe a dimensional measure concentration phenomenon using the new Talagrand-type inequality. / Mått på avstånd mellan två sannolikhetsfördelningar spelar en grundläggande roll i statistik och maskininlärning. Optimal transport (OT) teori ger ett sådant avstånd. Nyligen framsteg inom OT-teorin är en generalisering av klassisk OT med entropi-reglerad, kallad entropisk OT. Trots dess bekvämlighet i beräkning saknar det fortfarande teoretiskt stöd. I denna avhandling studerar vi sambandet mellan entropisk OT och Entropy Power Inequality (EPI). Först bevisar vi en ojämlikhet av HWI-typ med användning av OT-kartans oändliga förskjutningskonvexitet. För det andra härleder vi två Talagrand-typkvaliteter med mättnaden av EPI som motsvarar ett numeriskt uttryck vårt uttryck. Vi utvärderar för ett brett utbud av distributioner den här termen för Gauss och i.i.d. Cauchy-distributioner denna term finns oförklarlig form. Vi visar att våra resultat utökar tidigare resultat av GaussianTalagrand-ojämlikhet för Sinkhorn-avstånd till det starkt log-konkava fallet. Dessutom observerar vi ett dimensionellt mått koncentrationsfenomen mot den nya Talagrand-typen ojämlikhet.

This dissertation studies several aspects of the interface between information theory and signal processing. Several new and existing results in information theory are researched from the perspective of signal processing. Similarly, some fundamental results in signal processing and statistics are studied from the information theoretic viewpoint. The first part of this dissertation focuses on illustrating the equivalence between Stein's identity and De Bruijn's identity, and providing two extensions of De Bruijn's identity. First, it is shown that Stein's identity is equivalent to De Bruijn's identity in additive noise channels with specific conditions. Second, for arbitrary but fixed input and noise distributions, and an additive noise channel model, the first derivative of the differential entropy is expressed as a function of the posterior mean, and the second derivative of the differential entropy is expressed in terms of a function of Fisher information. Several applications over a number of fields, such as statistical estimation theory, signal processing and information theory, are presented to support the usefulness of the results developed in Section 2. The second part of this dissertation focuses on three contributions. First, a connection between the result, proposed by Stoica and Babu, and the recent information theoretic results, the worst additive noise lemma and the isoperimetric inequality for entropies, is illustrated. Second, information theoretic and estimation theoretic justifications for the fact that the Gaussian assumption leads to the largest Cramer-Rao lower bound (CRLB) is presented. Third, a slight extension of this result to the more general framework of correlated observations is shown. The third part of this dissertation concentrates on deriving an alternative proof for an extremal entropy inequality (EEI), originally proposed by Liu and Viswanath. Compared with the proofs, presented by Liu and Viswanath, the proposed alternative proof is simpler, more direct, and more information-theoretic. An additional application for the extremal inequality is also provided. Moreover, this section illustrates not only the usefulness of the EEI but also a novel method to approach applications such as the capacity of the vector Gaussian broadcast channel, the lower bound of the achievable rate for distributed source coding with a single quadratic distortion constraint, and the secrecy capacity of the Gaussian wire-tap channel. Finally, a unifying variational and novel approach for proving fundamental information theoretic inequalities is proposed. Fundamental information theory results such as the maximization of differential entropy, minimization of Fisher information (Cramer-Rao inequality), worst additive noise lemma, entropy power inequality (EPI), and EEI are interpreted as functional problems and proved within the framework of calculus of variations. Several extensions and applications of the proposed results are briefly mentioned.