Normalizing flows represent a class of generative models that exhibit a number of attractive properties, but do not always achieve state-of-the-art performance when it comes to perceived naturalness of generated samples. To improve the quality of generated samples, this thesis examines methods to enhance the expressivity of discrete-time normalizing flow models and thus their ability to capture different aspects of the data. In the first part of the thesis, we propose an invertible neural network architecture as an alternative to popular architectures like Glow that require an individual neural network per flow step. Although our proposal greatly reduces the number of parameters, it has not been done before, as such architectures are believed to not be powerful enough. For this reason, we define two optional extensions that could greatly increase the expressivity of the architecture. We use augmentation to add Gaussian noise variables to the input to achieve arbitrary hidden-layer widths that are no longer dictated by the dimensionality of the data. Moreover, we implement Piecewise Affine Activation Functions that represent a generalization of Leaky ReLU activations and allow for more powerful transformations in every individual step. The resulting three models are evaluated on two simple synthetic datasets – the two moons dataset and one generated from a mixture of eight Gaussians. Our findings indicate that the proposed architectures cannot adequately model these simple datasets and thus do not represent alternatives to current stateof-the-art models. The Piecewise Affine Activation Function significantly improved the expressivity of the invertible neural network, but could not make use of its full potential due to inappropriate assumptions about the function’s input distribution. Further research is needed to ensure that the input to this function is always standard normal distributed. We conducted further experiments with augmentation using the Glow model and could show minor improvements on the synthetic datasets when only few flow steps (two, three or four) were used. However, in a more realistic scenario, the model would encompass many more flow steps. Lastly, we generalized the transformation in the coupling layers of modern flow architectures from an elementwise affine transformation to a matrixbased affine transformation and studied the effect this had on MoGlow, a flow-based model of motion. We could show that McMoGlow, our modified version of MoGlow, consistently achieved a better training likelihood than the original MoGlow on human locomotion data. However, a subjective user study found no statistically significant difference in the perceived naturalness of the samples generated. As a possible reason for this, we hypothesize that the improvements are subtle and more visible in samples that exhibit slower movements or edge cases which may have been underrepresented in the user study. / Normalizing flows representerar en klass av generativa modeller som besitter ett antal eftertraktade egenskaper, men som inte alltid uppnår toppmodern prestanda när det gäller upplevd naturlighet hos genererade data. För att förbättra kvaliteten på dessa modellers utdata, undersöker detta examensarbete metoder för att förbättra uttrycksfullheten hos Normalizing flows-modeller i diskret tid, och därmed deras förmåga att fånga olika aspekter av datamaterialet. I den första delen av uppsatsen föreslår vi en arkitektur uppbyggt av ett inverterbart neuralt nätverk. Vårt förslag är ett alternativ till populära arkitekturer som Glow, vilka kräver individuella neuronnät för varje flödessteg. Även om vårt förslag kraftigt minskar antalet parametrar har detta inte gjorts tidigare, då sådana arkitekturer inte ansetts vara tillräckligt kraftfulla. Av den anledningen definierar vi två oberoende utökningar till arkitekturen som skulle kunna öka dess uttrycksfullhet avsevärt. Vi använder så kallad augmentation, som konkatenerar Gaussiska brusvariabler till observationsvektorerna för att uppnå godtyckliga bredder i de dolda lagren, så att deras bredd inte längre begränsas av datadimensionaliteten. Dessutom implementerar vi Piecewise Affine Activation-funktioner (PAAF), vilka generaliserar Leaky ReLU-aktiveringar genom att möjliggöra mer kraftfulla transformationer i varje enskilt steg. De resulterande tre modellerna utvärderas med hjälp av två enkla syntetiska datamängder - ”the two moons dataset” och ett som genererats genom att blanda av åtta Gaussfördelningar. Våra resultat visar att de föreslagna arkitekturerna inte kan modellera de enkla datamängderna på ett tillfredsställande sätt, och därmed inte utgör kompetitiva alternativ till nuvarande moderna modeller. Den styckvisa aktiveringsfunktionen förbättrade det inverterbara neurala nätverkets uttrycksfullhet avsevärt, men kunde inte utnyttja sin fulla potential på grund av felaktiga antaganden om funktionens indatafördelning. Ytterligare forskning behövs för att hantera detta problem. Vi genomförde ytterligare experiment med augmentation av Glow-modellen och kunde påvisa vissa förbättringar på de syntetiska dataseten när endast ett fåtal flödessteg (två, tre eller fyra) användes. Däremot omfattar modeller i mer realistiska scenarion många fler flödessteg. Slutligen generaliserade vi transformationen i kopplingslagren hos moderna flödesarkitekturer från en elementvis affin transformation till en matrisbaserad affin transformation, samt studerade vilken effekt detta hade på MoGlow, en flödesbaserad modell av 3D-rörelser. Vi kunde visa att McMoGlow, vår modifierade version av MoGlow, konsekvent uppnådde bättre likelihood i träningen än den ursprungliga MoGlow gjorde på mänskliga rörelsedata. En subjektiv användarstudie på exempelrörelser genererade från MoGlow och McMoGlow visade dock ingen statistiskt signifikant skillnad i användarnas uppfattning av hur naturliga rörelserna upplevdes. Som en möjlig orsak till detta antar vi att förbättringarna är subtila och mer synliga i situationer som uppvisar långsammare rörelser eller i olika gränsfall som kan ha varit underrepresenterade i användarstudien.

Les modèles génératifs basés sur la vraisemblance sont des éléments fondamentaux pour la modélisation statistique des données structurées. Ils peuvent être utilisés pour synthétiser des échantillons de données réalistes, et la fonction de vraisemblance peut être utilisée pour comparer les modèles et déduire diverses quantités statistiques. Cependant, le défi réside dans le développement de modèles capables de saisir avec précision les schémas statistiques présentés dans la distribution des données. Les modèles existants rencontrent souvent des limitations en termes de flexibilité représentationnelle et d’évolutivité computationnelle en raison du choix de la paramétrisation, freinant ainsi la progression vers cet idéal. Cette thèse présente une exploration systématique des structures appropriées qui peuvent être exploitées pour concevoir des modèles génératifs basés sur la vraisemblance, allant des architectures spécialisées telles que les applications triangulaires et les applications de potentiel convexes aux systèmes dynamiques paramétriques tels que les équations différentielles neuronales qui présentent des contraintes minimales en termes de paramétrisation. Les modèles proposés sont fondés sur des motivations théoriques et sont analysés à travers le prisme du changement de variable associé au processus de génération de données. Cette perspective permet de considérer ces modèles comme des formes distinctes de probability flows, unifiant ainsi des classes apparemment non liées de modèles génératifs basés sur la vraisemblance. De plus, des conceptions algorithmiques pratiques sont introduites pour calculer, approximer ou estimer les quantités nécessaires pour l’apprentissage et l’évaluation. Il est prévu que cette thèse suscite l’intérêt des communautés de modélisation générative et d’apprentissage automatique Bayésien/probabiliste, et qu’elle serve de ressource précieuse et d’inspiration pour les chercheurs et les praticiens du domaine. / Likelihood-based generative models are fundamental building blocks for statistical modeling of structured data. They can be used to synthesize realistic data samples, and the likelihood function can be used for comparing models and inferring various statistical quantities. However, the challenge lies in developing models capable of accurately capturing the statistical patterns presented in the data distribution. Existing models often face limitations in representational flexibility and computational scalability due to the choice of parameterization, impeding progress towards this ideal. This thesis presents a systematic exploration of suitable structures that can be exploited to design likelihood-based generative models, spanning from specialized architectures like triangular maps and convex potential maps to parametric dynamical systems such as neural differential equations that bear minimal parameterization restrictions. The proposed models are rooted in theoretical foundations and analyzed through the lens of the associated change of variable in the data generation process. This perspective allows for viewing these models as distinct forms of probability flows, thereby unifying seemingly unrelated classes of likelihood-based generative models. Moreover, practical algorithmic designs are introduced to compute, approximate, or estimate necessary quantities for training and testing purposes. It is anticipated that this thesis would be of interest to the generative modeling and Bayesian/probabilistic machine learning communities, and will serve as a valuable resource and inspiration for both researchers and practitioners in the field.

Text in Afrikaans / I wanted to investigate the way people experienced confessional pastoral therapy by conducting a qualitative research study. The role of the pastor were examined as well as the effect of discourses of power and ethics in pastoral counseling. I interviewed clients who were counseled by pastors of the AFM Church (Apostolic Faith Mission). The research supposes that knowledge and power discourses have a major influence in pastoral counseling which is often not accounted for. I wanted to see how clients experienced the effects of these discourses of power and ethics. I furthermore wanted to see how these power discourses could result in clients feeling subordinate to the pastor and his knowledge and the effect that these feelings might have on them. The purpose of this study was not to generalize the experiences of the participants, but rather to set a contextual background of the experiences of power in confessional pastoral counseling. / Practical Theology / M. Th. (Pastorale Terapie)

Text in Afrikaans / I wanted to investigate the way people experienced confessional pastoral therapy by conducting a qualitative research study. The role of the pastor were examined as well as the effect of discourses of power and ethics in pastoral counseling. I interviewed clients who were counseled by pastors of the AFM Church (Apostolic Faith Mission). The research supposes that knowledge and power discourses have a major influence in pastoral counseling which is often not accounted for. I wanted to see how clients experienced the effects of these discourses of power and ethics. I furthermore wanted to see how these power discourses could result in clients feeling subordinate to the pastor and his knowledge and the effect that these feelings might have on them. The purpose of this study was not to generalize the experiences of the participants, but rather to set a contextual background of the experiences of power in confessional pastoral counseling. / Philosophy, Practical and Systematic Theology / M. Th. (Pastorale Terapie)

Cette thèse porte sur deux sujets intrinsèquement liés : le calcul de la constante de normalisation d’un champ de Markov et l’estimation de l’affinité de liaison d’un complexe de protéines. Premièrement, afin d’aborder ce problème de comptage #P complet, nous avons développé Z*, basé sur un élagage des quantités de potentiels négligeables. Il s’est montré plus performant que des méthodes de l’état de l’art sur des instances issues d’interaction protéine-protéine. Par la suite, nous avons développé #HBFS, un algorithme avec une garantie anytime, qui s’est révélé plus performant que son prédécesseur. Enfin, nous avons développé BTDZ, un algorithme exact basé sur une décomposition arborescente qui a fait ses preuves sur des instances issues d’interaction intermoléculaire appelées “superhélices”. Ces algorithmes s’appuient sur des méthodes issuse des modèles graphiques : cohérences locales, élimination de variable et décompositions arborescentes. A l’aide de méthodes d’optimisation existantes, de Z* et des fonctions d’énergie de Rosetta, nous avons développé un logiciel open source estimant la constante d’affinité d’un complexe protéine protéine sur une librairie de mutants. Nous avons analysé nos estimations sur un jeu de données de complexes de protéines et nous les avons confronté à deux approches de l’état de l’art. Il en est ressorti que notre outil était qualitativement meilleur que ces méthodes. / This thesis is focused on two intrinsically related subjects : the computation of the normalizing constant of a Markov random field and the estimation of the binding affinity of protein-protein interactions. First, to tackle this #P-complete counting problem, we developed Z*, based on the pruning of negligible potential quantities. It has been shown to be more efficient than various state-of-the-art methods on instances derived from protein-protein interaction models. Then, we developed #HBFS, an anytime guaranteed counting algorithm which proved to be even better than its predecessor. Finally, we developed BTDZ, an exact algorithm based on tree decomposition. BTDZ has already proven its efficiency on intances from coiled coil protein interactions. These algorithms all rely on methods stemming from graphical models : local consistencies, variable elimination and tree decomposition. With the help of existing optimization algorithms, Z* and Rosetta energy functions, we developed a package that estimates the binding affinity of a set of mutants in a protein-protein interaction. We statistically analyzed our esti- mation on a database of binding affinities and confronted it with state-of-the-art methods. It appears that our software is qualitatively better than these methods.

La géométrie moléculaire, également connue sous le nom de conformation, est la représentation la plus intrinsèque et la plus informative des molécules. Cependant, prédire des conformations stables à partir de graphes moléculaires reste un problème difficile et fondamental en chimie et en biologie computationnelles. Les méthodes expérimentales et computationelles traditionnelles sont généralement coûteuses et chronophages. Récemment, nous avons assisté à des progrès considérables dans l'utilisation de l'apprentissage automatique, en particulier des modèles génératifs, pour accélérer cette procédure. Cependant, les approches actuelles basées sur les données n'ont généralement pas la capacité de modéliser des distributions complexes et ne tiennent pas compte de caractéristiques géométriques importantes. Dans cette thèse, nous cherchons à construire des modèles génératifs basés sur des principes pour la génération de conformation moléculaire qui peuvent surmonter les problèmes ci-dessus. Plus précisément, nous avons proposé des modèles de diffusion basés sur les flux, sur l'énergie et de débruitage pour la génération de structures moléculaires. Cependant, il n'est pas trivial d'appliquer ces modèles à cette tâche où la vraisemblance des géométries devrait avoir la propriété importante d'invariance par rotation par de translation. Inspirés par les progrès récents de l'apprentissage des représentations géométriques, nous fournissons à la fois une justification théorique et une mise en œuvre pratique sur la manière d'imposer cette propriété aux modèles. Des expériences approfondies sur des jeux de données de référence démontrent l'efficacité de nos approches proposées par rapport aux méthodes de référence existantes. / Molecular geometry, also known as conformation, is the most intrinsic and informative representation of molecules. However, predicting stable conformations from molecular graphs remains a challenging and fundamental problem in computational chemistry and biology. Traditional experimental and computational methods are usually expensive and time-consuming. Recently, we have witnessed considerable progress in using machine learning, especially generative models, to accelerate this procedure. However, current data-driven approaches usually lack the capacity for modeling complex distributions and fail to take important geometric features into account. In this thesis, we seek to build principled generative models for molecular conformation generation that can overcome the above problems. Specifically, we proposed flow-based, energy-based, and denoising diffusion models for molecular structure generation. However, it's nontrivial to apply these models to this task where the likelihood of the geometries should have the important property of rotational and translation invariance. Inspired by the recent progress of geometric representation learning, we provide both theoretical justification and practical implementation about how to impose this property into the models. Extensive experiments on common benchmark datasets demonstrate the effectiveness of our proposed approaches over existing baseline methods.

This research journey investigated the ways in which (1) the lives of adolescents have been influenced by parental divorce and subsequent remarriage, (2) exploring the relationships participants have with biological, nonresidential fathers and (3) to collaboratively present ways of doing family in alternative. Four adolescent girls took part in group conversations where they could were empowered to have their voices heard in a society where they are usually marginalized and silenced. As a result of these conversations a family game, FunFam, was developed that aimed to assist families in expanding communication within the family. Normalizing prescriptive discourses about divorce and remarriage were deconstructed to offer participants the opportunity to re-author their stories about their families. The second part of the research journey explored the problem-saturated stories that these four participants had with their biological, nonresidential fathers. They deconstructed the discourses that influenced this relationship and redefined the relationship to suit their expectations and wishes. / Practical Theology / M.Th.

This research journey investigated the ways in which (1) the lives of adolescents have been influenced by parental divorce and subsequent remarriage, (2) exploring the relationships participants have with biological, nonresidential fathers and (3) to collaboratively present ways of doing family in alternative. Four adolescent girls took part in group conversations where they could were empowered to have their voices heard in a society where they are usually marginalized and silenced. As a result of these conversations a family game, FunFam, was developed that aimed to assist families in expanding communication within the family. Normalizing prescriptive discourses about divorce and remarriage were deconstructed to offer participants the opportunity to re-author their stories about their families. The second part of the research journey explored the problem-saturated stories that these four participants had with their biological, nonresidential fathers. They deconstructed the discourses that influenced this relationship and redefined the relationship to suit their expectations and wishes. / Philosophy, Practical and Systematic Theology / M.Th.