- AutorIn
- Johannes Rauh
- Titel
- Finding the Maximizers of the Information Divergence from an Exponential Family
- Untertitel
- Finding the Maximizersof the Information Divergencefrom an Exponential Family
- Zitierfähige Url:
- https://nbn-resolving.org/urn:nbn:de:bsz:15-qucosa-77355
- Übersetzter Titel (DE)
- Das Auffinden der Maximierer der Informationsdivergenz von einer Exponentialfamilie
- Datum der Einreichung
- 03.11.2011
- Datum der Verteidigung
- 09.01.2011
- Abstract (EN)
- The subject of this thesis is the maximization of the information divergence from an exponential family on a finite set, a problem first formulated by Nihat Ay. A special case is the maximization of the mutual information or the multiinformation between different parts of a composite system. My thesis contributes mainly to the mathematical aspects of the optimization problem. A reformulation is found that relates the maximization of the information divergence with the maximization of an entropic quantity, defined on the normal space of the exponential family. This reformulation simplifies calculations in concrete cases and gives theoretical insight about the general problem. A second emphasis of the thesis is on examples that demonstrate how the theoretical results can be applied in particular cases. Third, my thesis contain first results on the characterization of exponential families with a small maximum value of the information divergence.
- Freie Schlagwörter (DE)
- Exponentialfamilie, Informationsdivergenz, Maximierung, Infomax-Prinzip
- Freie Schlagwörter (EN)
- Exponential family, information divergence, maximization, infomax principle
- Klassifikation (DDC)
- 519
- Normschlagwörter (GND)
- Informationstheorie
- Exponentialfamilie
- GutachterIn
- Prof. Dr. Jürgen Jost
- Prof. Dr. Andreas Knauf
- BetreuerIn
- Priv.-Dz. Nihat Ay
- Prof. Dr. Jürgen Jost
- Den akademischen Grad verleihende / prüfende Institution
- Universität Leipzig, Leipzig
- URN Qucosa
- urn:nbn:de:bsz:15-qucosa-77355
- Veröffentlichungsdatum Qucosa
- 19.10.2011
- Dokumenttyp
- Dissertation
- Sprache des Dokumentes
- Englisch
- Inhaltsverzeichnis
1. Introduction 2. Exponential families 2.1. Exponential families, the convex support and the moment map 2.2. The closure of an exponential family 2.3. Algebraic exponential families 2.4. Hierarchical models 3. Maximizing the information divergence from an exponential family 3.1. The directional derivatives of D(*|E ) 3.2. Projection points and kernel distributions 3.3. The function DE 3.4. The first order optimality conditions of DE 3.5. The relation between D(*|E) and DE 3.6. Computing the critical points 3.7. Computing the projection points 4. Examples 4.1. Low-dimensional exponential families 4.1.1. Zero-dimensional exponential families 4.1.2. One-dimensional exponential families 4.1.3. One-dimensional exponential families on four states 4.1.4. Other low-dimensional exponential families 4.2. Partition models 4.3. Exponential families with max D(*|E ) = log(2) 4.4. Binary i.i.d. models and binomial models 5. Applications and Outlook 5.1. Principles of learning, complexity measures and constraints 5.2. Optimally approximating exponential families 5.3. Asymptotic behaviour of the empirical information divergence A. Polytopes and oriented matroids A.1. Polytopes A.2. Oriented matroids Bibliography Index Glossary of notations