Spelling suggestions: "subject:"renyi entropy"" "subject:"enyi entropy""
1 |
On curvature conditions using Wasserstein spacesKell, Martin 05 August 2014 (has links) (PDF)
This thesis is twofold. In the first part, a proof of the interpolation inequality along geodesics in p-Wasserstein spaces is given and a new curvature condition on abstract metric measure spaces is defined.
In the second part of the thesis a proof of the identification of the q-heat equation with the gradient flow of the Renyi (3-p)-Renyi entropy functional in the p-Wasserstein space is given. For that, a further study of the q-heat flow is presented including a condition for its mass preservation.
|
2 |
On Generalized Measures Of Information With Maximum And Minimum Entropy PrescriptionsDukkipati, Ambedkar 03 1900 (has links)
Kullback-Leibler relative-entropy or KL-entropy of P with respect to R defined as ∫xlnddPRdP , where P and R are probability measures on a measurable space (X, ), plays a basic role in the
definitions of classical information measures. It overcomes a shortcoming of Shannon entropy – discrete case definition of which cannot be extended to nondiscrete case naturally. Further, entropy and other classical information measures can be expressed in terms of KL-entropy and
hence properties of their measure-theoretic analogs will follow from those of measure-theoretic KL-entropy. An important theorem in this respect is the Gelfand-Yaglom-Perez (GYP) Theorem which equips KL-entropy with a fundamental definition and can be stated as: measure-theoretic KL-entropy equals the supremum of KL-entropies over all measurable partitions of X . In this thesis we provide the measure-theoretic formulations for ‘generalized’ information measures, and
state and prove the corresponding GYP-theorem – the ‘generalizations’ being in the sense of R ´enyi and nonextensive, both of which are explained below.
Kolmogorov-Nagumo average or quasilinear mean of a vector x = (x1, . . . , xn) with respect to a pmf p= (p1, . . . , pn)is defined ashxiψ=ψ−1nk=1pkψ(xk), whereψis an arbitrarycontinuous and strictly monotone function. Replacing linear averaging in Shannon entropy with Kolmogorov-Nagumo averages (KN-averages) and further imposing the additivity constraint – a characteristic property of underlying information associated with single event, which is logarithmic – leads to the definition of α-entropy or R ´enyi entropy. This is the first formal well-known generalization of Shannon entropy. Using this recipe of R´enyi’s generalization, one can prepare only two information measures: Shannon and R´enyi entropy. Indeed, using this formalism R´enyi
characterized these additive entropies in terms of axioms of KN-averages. On the other hand, if one generalizes the information of a single event in the definition of Shannon entropy, by replacing the logarithm with the so called q-logarithm, which is defined as lnqx =x1− 1 −1 −q , one gets
what is known as Tsallis entropy. Tsallis entropy is also a generalization of Shannon entropy but it does not satisfy the additivity property. Instead, it satisfies pseudo-additivity of the form x ⊕qy = x + y + (1 − q)xy, and hence it is also known as nonextensive entropy. One can apply
R´enyi’s recipe in the nonextensive case by replacing the linear averaging in Tsallis entropy with KN-averages and thereby imposing the constraint of pseudo-additivity. A natural question that
arises is what are the various pseudo-additive information measures that can be prepared with this recipe? We prove that Tsallis entropy is the only one. Here, we mention that one of the important characteristics of this generalized entropy is that while canonical distributions resulting from ‘maximization’ of Shannon entropy are exponential in nature, in the Tsallis case they result in power-law distributions.
The concept of maximum entropy (ME), originally from physics, has been promoted to a general principle of inference primarily by the works of Jaynes and (later on) Kullback. This connects information theory and statistical mechanics via the principle: the states of thermodynamic equi-
librium are states of maximum entropy, and further connects to statistical inference via select the probability distribution that maximizes the entropy. The two fundamental principles related to the concept of maximum entropy are Jaynes maximum entropy principle, which involves maximizing Shannon entropy and the Kullback minimum entropy principle that involves minimizing relative-entropy, with respect to appropriate moment constraints.
Though relative-entropy is not a metric, in cases involving distributions resulting from relative-entropy minimization, one can bring forth certain geometrical formulations. These are reminiscent of squared Euclidean distance and satisfy an analogue of the Pythagoras’ theorem. This property
is referred to as Pythagoras’ theorem of relative-entropy minimization or triangle equality and plays a fundamental role in geometrical approaches to statistical estimation theory like information geometry. In this thesis we state and prove the equivalent of Pythagoras’ theorem in the
nonextensive formalism. For this purpose we study relative-entropy minimization in detail and present some results.
Finally, we demonstrate the use of power-law distributions, resulting from ME-rescriptions
of Tsallis entropy, in evolutionary algorithms. This work is motivated by the recently proposed generalized simulated annealing algorithm based on Tsallis statistics.
To sum up, in light of their well-known axiomatic and operational justifications, this thesis establishes some results pertaining to the mathematical significance of generalized measures of information. We believe that these results represent an important contribution towards the ongoing
research on understanding the phenomina of information.
(For formulas pl see the original document)
ii
|
3 |
On curvature conditions using Wasserstein spacesKell, Martin 22 July 2014 (has links)
This thesis is twofold. In the first part, a proof of the interpolation inequality along geodesics in p-Wasserstein spaces is given and a new curvature condition on abstract metric measure spaces is defined.
In the second part of the thesis a proof of the identification of the q-heat equation with the gradient flow of the Renyi (3-p)-Renyi entropy functional in the p-Wasserstein space is given. For that, a further study of the q-heat flow is presented including a condition for its mass preservation.
|
4 |
Analysis of Internal Boundaries and Transition Regions in Geophysical Systems with Advanced Processing TechniquesKrützmann, Nikolai Christian January 2013 (has links)
This thesis examines the utility of the Rényi entropy (RE), a measure of the complexity of probability density functions, as a tool for finding physically meaningful patterns in geophysical data. Initially, the RE is applied to observational data of long-lived atmospheric tracers in order to analyse the dynamics of stratospheric transitions regions associated with barriers to horizontal mixing. Its wider applicability is investigated by testing the RE as a method for highlighting internal boundaries in snow and ice from ground penetrating radar (GPR) recordings. High-resolution 500 MHz GPR soundings of dry snow were acquired at several sites near Scott Base, Antarctica, in 2008 and 2009, with the aim of using the RE to facilitate the identification and tracking of subsurface layers to extrapolate point measurements of accumulation from snow pits and firn cores to larger areas.
The atmospheric analysis focuses on applying the RE to observational tracer data from the EOS-MLS satellite instrument. Nitrous oxide (N2O) is shown to exhibit subtropical RE maxima in both hemispheres. These peaks are a measure of the tracer gradients that mark the transition between the tropics and the mid-latitudes in the stratosphere, also referred to as the edges of the tropical pipe. The RE maxima are shown to be located closer to the equator in winter than in summer. This agrees well with the expected behaviour of the tropical pipe edges and is similar to results reported by other studies. Compared to other stratospheric mixing metrics, the RE has the advantage that it is easy to calculate as it does not, for example, require conversion to equivalent latitude and does not rely on dynamical information such as wind fields.
The RE analysis also reveals occasional sudden poleward shifts of the southern hemisphere tropical pipe edge during austral winter which are accompanied by increased mid-latitude N2O levels. These events are investigated in more detail by creating daily high-resolution N2O maps using a two-dimensional trajectory model and MERRA reanalysis winds to advect N2O observations forwards and backwards in time on isentropic surfaces. With the aid of this ‘domain filling’ technique it is illustrated that the increase in southern hemisphere mid-latitude N2O during austral winter is probably the result of the cumulative effect of several large-scale, episodic leaks of N2O-rich air from the tropical pipe. A comparison with the global distribution of potential vorticity strongly suggests that irreversible mixing related to planetary wave breaking is the cause of the leak events. Between 2004 and 2011 the large-scale leaks are shown to occur approximately every second year and a connection to the equatorial quasi-biennial oscillation is found to be likely, though this cannot be established conclusively due to the relatively short data set.
Identification and tracking of subsurface boundaries, such as ice layers in snow or the bedrock of a glacier, is the focus of the cryospheric part of this project. The utility of the RE for detecting amplitude gradients associated with reflections in GPR recordings is initially tested on a 25 MHz sounding of an Antarctic glacier. The results show distinct regions of increased RE values that allow identification of the glacial bedrock along large parts of the profile. Due to the low computational requirements, the RE is found to be an effective pseudo gain function for initial analysis of GPR data in the field. While other gain functions often have to be tuned to give a good contrast between reflections and background noise over the whole vertical range of a profile, the RE tends to assign all detectable amplitude gradients a similar (high) value, resulting in a clear contrast between reflections and background scattering. Additionally, theoretical considerations allow the definition of a ‘standard’ data window size with which the RE can be applied to recordings made by most pulsed GPR systems and centre frequencies. This is confirmed by tests with higher frequency recordings (50 and 500 MHz) acquired on the McMurdo Ice Shelf. However, these also reveal that the RE processing is less reliable for identifying more closely spaced reflections from internal layers in dry snow.
In order to complete the intended high-resolution analysis of accumulation patterns by tracking internal snow layers in the 500 MHz data from two test sites, a different processing approach is developed. Using an estimate of the emitted waveform from direct measurement, deterministic deconvolution via the Fourier domain is applied to the high-resolution GPR data. This reveals unambiguous reflection horizons which can be observed in repeat measurements made one year apart. Point measurements of average accumulation from snow pits and firn cores are extrapolated to larger areas by identifying and tracking a dateable dust layer horizon in the radargrams. Furthermore, it is shown that annual compaction rates of snow can be estimated by tracking several internal reflection horizons along the deconvolved radar profiles and calculating the average change in separation of horizon pairs from one year to the next. The technique is complementary to point measurements from other studies and the derived compaction rates agree well with published values and theoretical estimates.
|
5 |
Minimization Problems Based On A Parametric Family Of Relative EntropiesAshok Kumar, M 05 1900 (has links) (PDF)
We study minimization problems with respect to a one-parameter family of generalized relative entropies. These relative entropies, which we call relative -entropies (denoted I (P; Q)), arise as redundancies under mismatched compression when cumulants of compression lengths are considered instead of expected compression lengths. These parametric relative entropies are a generalization of the usual relative entropy (Kullback-Leibler divergence). Just like relative entropy, these relative -entropies behave like squared Euclidean distance and satisfy the Pythagorean property. We explore the geometry underlying various statistical models and its relevance to information theory and to robust statistics. The thesis consists of three parts.
In the first part, we study minimization of I (P; Q) as the first argument varies over a convex set E of probability distributions. We show the existence of a unique minimizer when the set E is closed in an appropriate topology. We then study minimization of I on a particular convex set, a linear family, which is one that arises from linear statistical constraints. This minimization problem generalizes the maximum Renyi or Tsallis entropy principle of statistical physics. The structure of the minimizing probability distribution naturally suggests a statistical model of power-law probability distributions, which we call an -power-law family. Such a family is analogous to the exponential family that arises when relative entropy is minimized subject to the same linear statistical constraints.
In the second part, we study minimization of I (P; Q) over the second argument. This minimization is generally on parametric families such as the exponential family or the - power-law family, and is of interest in robust statistics ( > 1) and in constrained compression settings ( < 1).
In the third part, we show an orthogonality relationship between the -power-law family and an associated linear family. As a consequence of this, the minimization of I (P; ), when the second argument comes from an -power-law family, can be shown to be equivalent to a minimization of I ( ; R), for a suitable R, where the first argument comes from a linear family. The latter turns out to be a simpler problem of minimization of a quasi convex objective function subject to linear constraints. Standard techniques are available to solve such problems, for example, via a sequence of convex feasibility problems, or via a sequence of such problems but on simpler single-constraint linear families.
|
6 |
Quantum Error Correction in Quantum Field Theory and GravityKeiichiro Furuya (16534464) 18 July 2023 (has links)
<p>Holographic duality as a rigorous approach to quantum gravity claims that a quantum gravitational system is exactly equal to a quantum theory without gravity in lower spacetime dimensions living on the boundary of the quantum gravitational system. The duality maps key questions about the emergence of spacetime to questions on the non-gravitational boundary system that are accessible to us theoretically and experimentally. Recently, various aspects of quantum information theory on the boundary theory have been found to be dual to the geometric aspects of the bulk theory. In this thesis, we study the exact and approximate quantum error corrections (QEC) in a general quantum system (von Neumann algebras) focused on QFT and gravity. Moreover, we study entanglement theory in the presence of conserved charges in QFT and the multiparameter multistate generalization of quantum relative entropy.</p>
|
7 |
Higher Spins, Entanglement Entropy And HolographyDatta, Shouvik 01 1900 (has links) (PDF)
The idea of holography [1, 2] finds a concrete realization in form of the AdS/CFT correspondence [3, 4]. This duality relates a field theory with conformal symmetries to quantum gravity living in one higher dimension. In this thesis we study aspects of black hole quasinormal modes, higher spin theories and entanglement entropy in the context of this duality. In almost all cases we have been able to subject the duality to some precision tests.
Quasinormal modes encode the spectrum of black holes and the time-scale of pertur-
bations therein [5]. From the dual CFT viewpoint they are the poles of retarded Green's function (or peaks in the spectral function) [6]. Quasinormal modes were previously studied for scalar, gauge field and fermion fluctuations [7]. We solve for these quasinormal modes of higher spin (s _ 2) fields in the background of the BTZ black hole [8, 9]. We obtain an exact solution for a field of arbitrary spin s (integer or half-integer) in the BTZ background. This implies that the BTZ is perhaps the only known black hole background where such an analysis can be done analytically for all bosonic and fermionic fields.
The quasinormal modes are shown to match precisely with the poles of the corresponding Green's function in the CFT living on the boundary. Furthermore, we show that one-loop determinants of higher spin fields can also be written as a product form [10] in terms of these quasinormal modes and this agrees with the same obtained by integrating the heat-kernel [11].
We then turn our attention to dualities relating higher-spin gravity to CFTs with W
algebra symmetries. Since higher spin gravity does go beyond diffeomorphism invariance, one needs re_ned notions of the usual concepts in differential geometry. For example, in general relativity black holes are defined by the presence of the horizon. However, higher spin gravity has an enlarged group of symmetries of which the diffeomorphisms form a subgroup. The appropriate way of thinking of solutions in higher spin gravity is via characterizations which are gauge invariant [12, 13]. We study classical solutions embedded in N = 2 higher spin supergravity. We obtain a general gauge-invariant condition { in terms of the odd roots of the superalgebra and the eigenvalues of the holonomy matrix of the background { for the existence of a Killing spinor such that these solutions are supersymmetric [14].
We also study black holes in higher spin supergravity and show that the partition function of these black holes match exactly with that obtained from a CFT with the same asymptotic symmetry algebra [15]. This involved studying the asymptotic symmetries of the black hole and thereby developing the holographic dictionary for the bulk charges and chemical potentials with the corresponding quantities of the CFT.
We finally investigate entanglement entropy in the AdS3/CFT2 context. Entanglement
entropy is an useful non-local probe in QFT and many-body physics [16]. We analytically evaluate the entanglement entropy of the free boson CFT on a circle at finite temperature (i.e. on a torus) [17]. This is one of the simplest and well-studied CFTs. The entanglement entropy is calculated via the replica trick using correlation functions of bosonic twist operators on the torus [18]. We have then set up a systematic high temperature expansion of the Renyi entropies and determined their finite size corrections. These _nite size corrections both for the free boson CFT and the free fermion CFT were then compared with the one-loop corrections obtained from bulk three dimensional handlebody spacetimes which have higher genus Riemann surfaces (replica geometry) as its boundary [19]. One-loop corrections in these geometries are entirely determined by the spectrum of the excitations present in the bulk. It is shown that the leading _nite size corrections obtained by evaluating the one-loop determinants on these handlebody geometries exactly match with those from the free fermion/boson CFTs. This provides a test for holographic methods to calculate one-loop corrections to entanglement entropy.
We also study conformal field theories in 1+1 dimensions with W-algebra symmetries at
_nite temperature and deformed by a chemical potential (_) for a higher spin current. Using OPEs and uniformization techniques, we show that the order _2 correction to the Renyi and entanglement entropies (EE) of a single interval in the deformed theory is universal [20]. This universal feature is also supported by explicit computations for the free fermion and free boson CFTs { for which the EE was calculated by using the replica trick in conformal perturbation theory by evaluating correlators of twist fields with higher spin operators [21]. Furthermore, this serves as a verification of the holographic EE proposal constructed from Wilson lines in higher spin gravity [22, 23].
We also examine relative entropy [24] in the context of higher-spin holography [25]. Relative entropy is a measure of distinguishability between two quantum states. We confirm the expected short-distance behaviour of relative entropy from holography. This is done by showing that the difference in the modular Hamiltonian between a high-temperature state and the vacuum matches with the difference in the entanglement entropy in the short-subsystem regime.
|
Page generated in 0.0709 seconds