<div>Many important scientific, engineering, and societal challenges involve large systems of individual agents or components interacting in complex ways. For example, to understand the emergence of consciousness, we study the dendritic integration in neurons; to prevent disease and rumor outbreaks, we trace the dynamics of social networks; to perform complicated scientific experiments, we separate and control the independent variables. Collectively, the interactions between individual neurons/agents/variables are often non-linear, i.e., a subset of the agents jointly behave in a manner unlike the marginal behaviors of the individuals.</div><div><br></div><div>The goal of this thesis is to construct a theoretical framework for measuring, comparing, and representing complex interactions in stochastic systems. Specifically, tools from information theory, differential geometry, lattice theory, and linear algebra are used to identify and characterize higher-order interactions among random variables.</div><div><br></div><div>We first propose measures of unique, redundant, and synergistic interactions for small stochastic systems using information projections for the exponential family. Their magnitudes are endowed with information theoretical meanings naturally, since they are measured by the Kullback-Leibler divergence. We prove that these quantities satisfy various desired properties.</div><div><br></div><div>We next apply these measures to hypothesis testing and network communication. We interpret the unique information as the two types of error components in a hypothesis testing problem. We analytically show that there is a duality between the synergistic and redundant information in Gaussian Multiple Access Channels (MAC) and Broadcast Channels (BC). We establish a novel duality between the partial information decomposition components for MAC and BC in the general case.</div><div><br></div><div>We lastly propose a new concept of representing the partial information decomposition framework with random variables. We give necessary and sufficient conditions for the representation under the assumption of Gaussianity and develop a construction method.</div><div><br></div><div>This research has the potential to advance the fields of information theory, statistics, and machine learning by contributing novel ideas, implementing these ideas with innovative tools, and constructing new simulation methods.</div>

<p>Ispitivan&nbsp; je&nbsp; spregnut&nbsp; sistem&nbsp; jednačina&nbsp; za&nbsp; prognozu&nbsp; temperature&nbsp; na povr&scaron;ini&nbsp; i&nbsp; u&nbsp; dubljem sloju zemlji&scaron;ta.&nbsp; Računati&nbsp; su&nbsp; Ljapunovljevi eksponenti,&nbsp; bifurkacioni dijagram, atraktor i analiziran je domen re&scaron;enja. Uvedene su nove informacione mere&nbsp; bazirane na<br />Kolmogorovljevoj kompleksnosti,&nbsp; za kvantifikaciju&nbsp; stepena nasumičnosti u vremenskim serijama,.&nbsp; Nove mere su primenjene na razne serije dobijene merenjem fizičkih faktora životne sredine i pomoću klimatskih modela.</p> / <p>Coupled system of prognostic equations for&nbsp; the&nbsp; ground surface temperature and&nbsp; the deeper layer temperature was examind. Lyapunov exponents, bifurcation diagrams, attractor and the domain of solutions were analyzed.&nbsp; Novel information measures based on Kolmogorov complexity&nbsp; and used&nbsp; for the quantification of randomness in time series, were presented.Novel measures were tested on various time series obtained by measuring physical factors of the environment or as the climate model outputs.</p>

The thesis presents research on multiobjective optimization based on memetic computing and its applications in engineering. We have introduced a framework for adaptive multiobjective memetic optimization algorithms (AMMOA) with an information theoretic criterion for guiding the selection, clustering, and local refinements. A robust stopping criterion for AMMOA has also been introduced to solve non-linear and large-scale optimization problems. The framework has been implemented for different benchmark test problems with remarkable results. This thesis also presents two applications of these algorithms. First, an optimal image data hiding technique has been formulated as a multiobjective optimization problem with conflicting objectives. In particular, trade-off factors in designing an optimal image data hiding are investigated to maximize the quality of watermarked images and the robustness of watermark. With the fixed size of a logo watermark, there is a conflict between these two objectives, thus a multiobjective optimization problem is introduced. We propose to use a hybrid between general regression neural networks (GRNN) and the adaptive multiobjective memetic optimization algorithm (AMMOA) to solve this challenging problem. This novel image data hiding approach has been implemented for many different test natural images with remarkable robustness and transparency of the embedded logo watermark. We also introduce a perceptual measure based on the relative Rényi information spectrum to evaluate the quality of watermarked images. The second application is the problem of joint spectrum sensing and power control optimization for a multichannel, multiple-user cognitive radio network. We investigated trade-off factors in designing efficient spectrum sensing techniques to maximize the throughput and minimize the interference. To maximize the throughput of secondary users and minimize the interference to primary users, we propose a joint determination of the sensing and transmission parameters of the secondary users, such as sensing times, decision threshold vectors, and power allocation vectors. There is a conflict between these two objectives, thus a multiobjective optimization problem is used again in the form of AMMOA. This algorithm learns to find optimal spectrum sensing times, decision threshold vectors, and power allocation vectors to maximize the averaged opportunistic throughput and minimize the averaged interference to the cognitive radio network. / February 2016

In underwater tracking and surveillance, the active towed array sonar presents a way of discovering and tracking adversarial submerged targets that try to stay hidden. The configuration consist of listening and emitting hydrophones towed behind a ship. Moreover, it has inherent limitations, and the characteristics of sound in the ocean are complex. By varying the pulse form emitted and the trajectory of the ship the measurement accuracy may be improved. This type of optimization constitutes a sensor management problem. In this thesis, a model of the tracking scenario has been constructed derived from Cramér-Rao bound analyses. A model predictive control approach together with information measures have been used to optimize a filter's estimated state of the target. For the simulations, the MATLAB environment has been used. Different combinations of decision horizons, information measures and variations of the Kalman filter have been studied. It has been found that the accuracy of the Extended Kalman filter is too low to give consistent results given the studied information measures. However, the Unscented Kalman filter is sufficient for this purpose.

<p>Holographic duality as a rigorous approach to quantum gravity claims that a quantum gravitational system is exactly equal to a quantum theory without gravity in lower spacetime dimensions living on the boundary of the quantum gravitational system. The duality maps key questions about the emergence of spacetime to questions on the non-gravitational boundary system that are accessible to us theoretically and experimentally. Recently, various aspects of quantum information theory on the boundary theory have been found to be dual to the geometric aspects of the bulk theory. In this thesis, we study the exact and approximate quantum error corrections (QEC) in a general quantum system (von Neumann algebras) focused on QFT and gravity. Moreover, we study entanglement theory in the presence of conserved charges in QFT and the multiparameter multistate generalization of quantum relative entropy.</p>