We investigate the coherent gradient hypothesis and show that the coherence measurements are different on real and random data regardless of the network's initialization. We introduce "diffs," an attempt at an element-wise approximation at coherence, and investigate their properties. We study how coherence is affected by increasing the width of simple fully-connected networks. We then prune those fully-connected networks and find that sparse networks outperform dense networks with the same number of nonzero parameters. In addition, we show that it is possible to increase the performance of a sparse network by scaling the size of the dense parent network it is derived from. Finally we apply our pruning methods to ResNet50 and ViT and find that diff-based pruning can be competitive with other methods.

Testing is recognized as a key and challenging factor that can either boost or halt the game development in the mobile game industry. On one hand, manual testing is expensive and time-consuming, especially the wide spectrum of device hardware and software, so called fragmentation, significantly increases the cost to test applications on devices manually. On the other hand, automated testing is also very difficult due to more inherent technical issues to test games as compared to other mobile applications, such as non-native widgets, nondeterminism , complex game strategies and so on. Current testing frameworks (e.g., Android Monkey, Record and Replay) are limited because they adopt no domain knowledge to test games. Learning-based tools (e.g., Wuji) require tremendous resources and manual efforts to train a model before testing any game. The high cost of manual testing and lack of efficient testing tools for mobile games motivated the work presented in this thesis which aims to explore easy and efficient approaches to test mobile games efficiently and effectively. A new Android mobile game testing tool, called LIT, has been developed. LIT is a lightweight approach to generalize playtest tactics from manual testing, and to adopt the tactics for automatic game testing. LIT has two phases: tactic generalization and tactic concretization. In Phase I, when a human tester plays an Android game G for awhile (e.g., eight minutes), LIT records the tester's inputs and related scenes. Based on the collected data, LIT infers a set of context-aware, abstract playtest tactics that describe under what circumstances, what actions can be taken. In Phase II,LIT tests G based on the generalized tactics. Namely, given a randomly generated game scene, LIT tentatively matches that scene with the abstract context of any inferred tactic; if the match succeeds, LIT customizes the tactic to generate an action for playtest. Our evaluation with nine games shows LIT to outperform two state-of-the-art tools and are reinforcement learning (RL)-based tool, by covering more code and triggering more errors. This implies that LIT complements existing tools and helps developers better test certain games (e.g., match3). / Master of Science / Testing is recognized as a key and challenging factor that can either boost or halt the game development in mobile game industry. On the one hand, manual testing is expensive and time-consuming, especially the wide spectrum of device hardware and software significantly increase cost to test applications on devices manually. On the other hand, automated testing is also very difficult due to more inherent technical issues to test games as compared to other mobile applications. The two factors motivated the work presented in this thesis. A new Android mobile game testing tool, called LIT, has been developed. LIT is a light weight approach to generalize playtest tactics from manual testing, and to adopt the tactics for automatic game testing. A playtest is the process in which testers play video games for software quality assurance. When a human tester plays an Android game G for awhile (e.g., eight minutes),LIT records the tester's inputs and related scenes. Based on the collected data, LIT infers a set of context-aware, abstract playtest tactics that describe under what circumstances, what actions can be taken. In Phase II, LIT tests G based on the generalized tactics. Namely, given a randomly generated game scene, LIT tentatively matches that scene with the abstract context of any inferred tactic; if the match succeeds, LIT customizes the tactic to generate an action for playtest. Our evaluation with nine games shows LIT to outperform two state-of-the-art tools and a reinforcement learning (RL)-based tool, by covering more code and triggering more errors. This implies that LIT complements existing tools and helps developers better test certain games (e.g., match3)

Generalization error describes how well a supervised machine learning algorithm predicts the labels of input data that it has not been trained with. This project aims to explore two different methods for bounding generalization error, f-CMI and ISMI, which explicitly use mutual information. Our experiments are based on the experiments in the papers in which the methods were proposed. The experiments implement and validate the accuracy of the mathematically derived bounds. Each methodology also has a different method for calculating mutual information. The ISMI bound experiment used a multivariate normal distribution dataset, whereas a dataset consisting of cats and dogs was used for the experiment using f-CMI. Our results show that both methods are capable of bounding the generalization error of a binary classification algorithm and provide bounds that closely follow the true generalization error. The results of the experiments agree with the original experiments, indicating that the proposed methods also work for similar applications with different datasets. / Generaliseringsfel beskriver hur väl en övervakad maskininlärnings algoritm förutspår etiketter av indata som den inte har blivit tränad med. Syftet med projektet är att utforska två olika metoder för att begränsa generaliseringsfelet, f-CMI och ISMI som explicit använder ömsesidig information. Vårt experiment är baserat på experimenten i artiklarna som tog fram metoderna. Experimenten implementerade och validerade noggrannheten av de matematiskt härleda gränserna. Varje metod har olika sätt att beräkna den ömsesidiga informationen. ISMI gräns experimentet använde en flerdimensionell normalfördelning som data set, medan en datauppsättning med katter och hundar användes för f-CMI gränsen. Våra resultat visar att båda metoder kan begränsa generaliseringsfelet av en binär klassificerings algoritm och förse gränser som nära följer det sanna generaliseringsfelet. Resultatet av experimenten instämmer med de ursprungliga författarnas experiment vilket indikerar att de föreslagna metoderna också fungerar for liknande tillämpningar med andra data set. / Kandidatexjobb i elektroteknik 2022, KTH, Stockholm

Le principe d’invariance par rapport à la causalité est au coeur d’approches notables telles que la minimisation du risque invariant (IRM) qui cherchent à résoudre les échecs de généralisation hors distribution (OOD). Malgré la théorie prometteuse, les approches basées sur le principe d’invariance échouent dans les tâches de classification courantes, où les caractéristiques invariantes (causales) capturent toutes les informations sur l’étiquette. Ces échecs sont-ils dus à l’incapacité des méthodes à capter l’invariance ? Ou le principe d’invariance lui-même est-il insuffisant ? Pour répondre à ces questions, nous réexaminons les hypothèses fondamentales dans les tâches de régression linéaire, où il a été démontré que les approches basées sur l’invariance généralisent de manière prouvée l’OOD. Contrairement aux tâches de régression linéaire, nous montrons que pour les tâches de classification linéaire, nous avons besoin de restrictions beaucoup plus fortes sur les changements de distribution, sinon la généralisation OOD est impossible. De plus, même avec des restrictions appropriées sur les changements de distribution en place, nous montrons que le principe d’invariance seul est insuffisant. Nous prouvons qu’une forme de contrainte de goulot d’étranglement d’information avec l’invariance aide à résoudre les échecs clés lorsque les caractéristiques invariantes capturent toutes les informations sur l’étiquette et conservent également le succès existant lorsqu’elles ne le font pas. Nous proposons une approche qui combine ces deux principes et démontre son efficacité sur des tests unitaires linéaires et sur divers jeux de données réelles de grande dimension. / The invariance principle from causality is at the heart of notable approaches such as invariant risk minimization (IRM) that seek to address out-of-distribution (OOD) generalization failures. Despite the promising theory, invariance principle-based approaches fail in common classification tasks, where invariant (causal) features capture all the information about the label. Are these failures due to the methods failing to capture the invariance? Or is the invariance principle itself insufficient? To answer these questions, we revisit the fundamental assumptions in linear regression tasks, where invariance-based approaches were shown to provably generalize OOD. In contrast to the linear regression tasks, we show that for linear classification tasks we need much stronger restrictions on the distribution shifts, or otherwise OOD generalization is impossible. Furthermore, even with appropriate restrictions on distribution shifts in place, we show that the invariance principle alone is insufficient. We prove that a form of the information bottleneck constraint along with invariance helps address the key failures when invariant features capture all the information about the label and also retains the existing success when they do not. We propose an approach that combines both these principles and demonstrate its effectiveness on linear unit tests and on various high-dimensional real datasets.

Aposematic species have evolved conspicuous warning signals, such as bright colors and striking patterns, to deter predators. Some edible and harmless species take advantage of this deterrent effect by mimicking their appearance. Mimicry is a great example of how natural selection produces remarkable adaptations. However, while some species evolve a very close similarity to their models to effectively avoid attacks, others are successful in doing so despite an incomplete similarity, i.e. imperfect mimicry. In some cases, it is surprising how such a crude disguise can fool predators. Why and how imperfect mimicry can persist has been much discussed and considered as a problem for the theory of natural selection. It is therefore of great interest to understand what makes it possible. Predator psychology is an important factor in the evolution of aposematism and mimicry. In the past decades it has been suggested that certain components of prey appearance are more important to predators than others during prey assessment. We developed this idea by incorporating concepts from associative learning, and presented a new approach to explain imperfect mimicry. Our general hypothesis is that prey traits have different salience to predators. Certain traits are perceived as highly salient and are thus used primarily in the discrimination and generalization of prey, while traits with low salience are overshadowed and not used in the assessment. The salience of a trait can depend on how conspicuous or discriminable it is in the particular context, and can vary due to for example previous predator experience. We tested our ideas with wild blue tits and domestic chickens as predators, and artificial and semi-natural prey stimuli. In paper I we found that the trait that was perceived as most salient (color) was the one used to discriminate and generalize between prey. Mimics of that specific trait were highly avoided, despite differences in the other traits. We also found that salience is relative and context dependent (paper II). In a context where two traits were perceived as similarly salient, mimicry of a single trait offered intermediate protection, while mimicry of both offered high protection. In another context, the traits were perceived differently salient, and mimicry of one trait was enough for high protection. In paper III we tested a proposed scenario for the initiation of mimicry evolution in the edible butterfly mimic Papilio polyxenes asterius to its noxious model Battus philenor. The results showed that a partial similarity with the model in the salient black wing color offered intermediate protection from attacks, despite a general dissimilarity. This thesis investigates the major questions of imperfect mimicry: the initial step of mimicry evolution, the persistence of imperfect mimicry, and variation in mimic-model similarity. We conclude that mimicry evolution can begin in a non-mimetic species that acquires similarity to a model species in a high-salience trait. When multiple traits have similar salience, multi-trait mimicry is needed for higher protection. Mimicry can remain imperfect if the differences are in traits with low salience, and therefore under low or no selection pressure to change. To complete the picture, we showed that predators can have a biased generalization toward a more pronounced version of a salient trait (paper IV). The evolution of aposematism could therefore be explained by gradual enhancement of salient traits. / <p>At the time of the doctoral defense, the following papers were unpublished and had a status as follows: Paper 3: Accepted. Paper 4: Manuscript.</p>

Abstract : The objective is to provide methods to improve the performance, or prediction accuracy of standard stacking approach, which is an ensemble method composed of simple, heterogeneous base models, through the integration of the diversity generation, combination and/or selection stages for regression problems. In Chapter 1, we propose to combine a set of level-1 learners into a level-2 learner, or ensemble. We also propose to inject a diversity generation mechanism into the initial cross-validation partition, from which new cross-validation partitions are generated, and sub-sequent ensembles are trained. Then, we propose an algorithm to select best partition, or corresponding ensemble. In Chapter 2, we formulate the partition selection as a Pareto-based multi-criteria optimization problem, as well as an algorithm to make the partition selection iterative with the aim to improve more the ensemble prediction accuracy. In Chapter 3, we propose to generate multiple populations or partitions by injecting a diversity mechanism to the original dataset. Then, an algorithm is proposed to select the best partition among all partitions generated by the multiple populations. All methods designed and implemented in this thesis get encouraging, and favorably results across different dataset against both state-of-the-art models, and ensembles for regression. / Résumé : L’objectif est de fournir des techniques permettant d’améliorer la performance de l’algorithme de stacking, une méthode ensembliste composée de modèles de base simples et hétérogènes, à travers l’intégration de la génération de la diversité, la sélection et combinaison des modèles. Dans le chapitre 1, nous proposons de combiner différents sous-ensembles de modèles de base obtenus au primer niveau. Nous proposons un mécanisme pour injecter de la diversité dans la partition croisée initiale, à partir de laquelle de nouvelles partitions de validation croisée sont générées, et les ensembles correspondant sont formés. Ensuite, nous proposons un algorithme pour sélectionner la meilleure partition. Dans le chapitre 2, nous formulons la sélection de la partition comme un problème d’optimisation multi-objectif fondé sur un principe de Pareto, ainsi que d’un algorithme pour faire une application itérative de la sélection avec l’objectif d’améliorer d’avantage la précision d’ensemble. Dans le chapitre 3, nous proposons de générer plusieurs populations en injectant un mécanisme de diversité à l’ensemble de données original. Ensuite, un algorithme est proposé pour sélectionner la meilleur partition entre toutes les partitions produite par les multiples populations. Nous avons obtenu des résultats encourageants avec ces algorithmes lors de comparaisons avec des modèles reconnus sur plusieurs bases de données.

The purpose of this study was to investigate the effects of explicit instruction combined with video prompting to teach text comprehension skills to students with autism spectrum disorder. Participants included 4 elementary school students with autism. A multiple probe across participants design was used to evaluate the intervention’s effectiveness. Results indicated that the intervention was successful for all participants. All participants mastered the comprehension skills; however, data were highly variable during the acquisition phase. Implications for researchers and practitioners are discussed.

Reliability Generalization (RG) is a meta-analytic method that examines the sources of measurement error variance for scores for multiple studies that use a certain instrument or group of instruments that measure the same construct (Vacha-Haase, Henson, & Caruso, 2002). Researchers have been conducting RG studies for over 10 years since it was first discussed by Vacha-Haase (1998). Henson and Thompson (2002) noted that, as RG is not a monolithic technique; researchers can conduct RG studies in a variety of ways and include diverse variables in their analyses. Differing recommendations exist in regards to how researchers should retrieve, code, and analyze information when conducting RG studies and these differences can affect the conclusions drawn from meta-analytic studies (Schmidt, Oh, & Hayes, 2009) like RG. The present study is the first comprehensive review of both current RG practices and RG recommendations. Based upon the prior research findings of other meta-analytic review papers (e.g., Dieckmann, Malle, & Bodner 2009), the overarching hypothesis was that there would be differences between current RG practices and best practice recommendations made for RG studies. Data consisted of 64 applied RG studies and recommendation papers, book chapters, and unpublished papers/conference papers. The characteristics that were examined included how RG researchers: (a) collected studies, (b) organized studies, (c) coded studies, (d) analyzed their data, and (e) reported their results. The results showed that although applied RG researchers followed some of the recommendations (e.g., RG researchers examined sample characteristics that influenced reliability estimates), there were some recommendations that RG researchers did not follow (e.g., the majority of researchers did not conduct an a priori power analysis). The results can draw RG researchers’ attentions to areas where there is a disconnect between practice and recommendations as well as provide a benchmark for assessing future improvement in RG implementation.

A major goal of ecology is to understand the factors that shape interactions among species. In this study, I explored the little-known associations between beetles and macrofungal fruiting bodies to characterize patterns of beetle-fungus association and to investigate sources of variation in the structure of these trophic interactions. First, I characterized the composition and diversity of beetle-sporocarp associations at two sites in the Appalachian Mountains and foothills, and evaluated the extent to which beetle community structure varied with fungal species, sporocarp age, and sporocarp dry mass. My results showed that beetle abundance and diversity differed among fungal species and were positively associated with sporocarp age and dry mass. I also found evidence of a nested structure in beetle-sporocarp interactions, wherein specialists on both sides of the association interact preferentially with more generalized species. Next, I performed a field study of beetle-sporocarp associations over two summers to evaluate the factors related to interaction breadth in trophic associations. I found evidence that interaction breadth varies with the palatability of the food organism (as indicated by sporocarp toughness and sporocarp age) and showed that beetle interaction breadth was negatively correlated with sporocarp persistence. I found strong intraseasonal variation in interaction breadth, but no evidence that this variation was structured by precipitation or differences in beetle community composition. In my third chapter, I conducted a field experiment to investigate (1) the importance of an individual food organism's physical properties in determining its relative importance in the beetle-sporocarp interaction network and (2) whether the structure of the beetle-sporocarp interaction network cycles predictably with the time of day. My results show that size and density of individual food organisms may be important factors in determining their relative importance in an interaction network, and offer the first evidence of diurnal cycling in the structure of interaction networks.

The purpose of this study was threefold: (a) to examine the separate effects of increased accuracy on multiple-choice/rank-order written tests and coaching on the teaching performance of participants; (b) to compare generalization across tasks produced by the workshop and coaching; and (c) to assess maintenance of teaching performance. Following baseline, two adults received a lecture on discrete trial teaching procedures. A written test measured verbal performance on workshop material periodically throughout this phase. During the next phase, each adult then experienced further training via in-situ coaching. A multiple baseline design across tasks was used during the coaching phase. Results of the workshop training package revealed an inverse relationship between the strongest verbal performance and strongest teaching performance skill areas. In addition, only with the introduction of the in-situ coaching package did teacher performance improve significantly across all behaviors. Child responding remained relatively constant throughout the study, regardless of teacher performance. Some generalization of teacher behavior was observed across tasks, but was extremely variable across both workshop and coaching conditions. After the cessation of coaching, teacher performance remained stable across maintenance phases and at a 6-week follow-up.