- Titel
- On the Optimization Properties of Optimal-Transport Based Generative Adversarial Linear Networks
- Zitierfähige Url:
- https://nbn-resolving.org/urn:nbn:de:bsz:15-qucosa2-1017929
- Datum der Einreichung
- 11.11.2024
- Datum der Verteidigung
- 18.12.2025
- Abstract (EN)
- This thesis is concerned with the analysis of the optimization problem in a class of generative models called Generative Adversarial Networks (GANs). GANs are successful generative models used to learn to mimic a given sampled distribution, and can be linked to Optimal Transport (OT) by the loss they rely on. OT is a reliable means to quantitatively compare two given probability distributions. The overall optimization problem is challenging as it involves a minimax optimization, in a generally highly non- convex-concave framework. In order to alleviate the analysis issues raised by this setting, several means can be employed. This thesis explores some of them in order to study a simplified OT-based generative model. After introducing relevant mathematical tools in Chapter 2, we present in Chapter 3 the general framework of GANs, as well as the different simplifications that can be employed in order to derive analytical guarantees on the optimization system. Two examples, one discrete and one continuous, are investigated in order to illustrate how the analytical framework can be used in very simplified settings. A particularly useful simplification is to reduce the minimax problem to a minimization problem only, by considering the maximization problem as being optimally solved at each time. In OT-based GAN, this typically boils down to analysing a type of distance between probability distributions, the Wasserstein distance. The minimization problem corresponds to finding a distribution that minimizes this distance to a given, target distribution. Chapters 4 and 5 leverage this simplification and study the landscape and convergence properties of a particular, linear OT-based GAN. In this case, the optimization problem can be cast as an overparametrized matrix factorization problem, combining a deep linear neural network with a covariance matrix parametrizations. The landscape of the resulting loss, based on the so-called Bures-Wasserstein distance between the covariance matrices, is investigated in Chapter 4. More precisely, the critical points of the loss are derived, under the appropriate parametrizations. Given a rank constraint on the model covariance matrix, we make explicit the global minimizer to the matrix approximation problem. We find that they are are identical to the minimizers when considering a Euclidean distance between the covariance matrices. The Hessian of the loss at the critical points is derived and numerically investigated, in order to understand what differences the Euclidean and the Bures-Wasserstein distances may introduce in the optimization. In Chapter 5, we analyze the gradient flow and gradient descent optimization schemes of this deep linear neural network under the Bures-Wasserstein loss. Convergence to minimizers of the loss is proven, under additional assumptions on the initialization of the scheme. The rank constraint on the approximation matrix appears here naturally as a bottleneck in the network representing the model covariance matrix. Complementary to the main discussion about GANs, the thesis also includes a chapter on the optimization landscape in a supervised learning setting (Chapter 6 on mode connectivity). Its conclusions may be linked to the GAN discussion in the simplified, limited setting of a minimization-only optimization scheme.
- Freie Schlagwörter (EN)
- Bures distance, Linear networks
- Klassifikation (DDC)
- 500
- Den akademischen Grad verleihende / prüfende Institution
- Universität Leipzig, Leipzig
- Version / Begutachtungsstatus
- angenommene Version / Postprint / Autorenversion
- URN Qucosa
- urn:nbn:de:bsz:15-qucosa2-1017929
- Veröffentlichungsdatum Qucosa
- 21.01.2026
- Dokumenttyp
- Dissertation
- Sprache des Dokumentes
- Englisch
- Lizenz / Rechtehinweis
CC BY-NC-SA 4.0