Domain-specific languages can significantly speed up the development of software applications. However, it usually takes a few iterations of the language design before it achieves such power. At the same time, many domains tend to evolve quite often today, which implies that domain-specific languages have to evolve accordingly. Thus, being able to evolve a language in a painless manner is crucial. Unfortunately, current state-of-the-art research does not provide enough answers on how to efficiently evolve domain-specific languages. We present an approach to evolving a language in order to reduce the amount of code cloning it introduces. The approach specifically targets those languages whose design causes users to create many duplicated code segments. We target domain-specific languages as they tend to be more challenging to evolve due to their specifics, but the approach may be applicable to general purpose programming languages as well. The approach was tested on a real-world domain-specific language that is used in a financial domain. We proposed three improvements and current users helped us evaluate them. We found that the proposed improvements would reduce code cloning, which provides evidence that the approach can be used in a real-world environment. Furthermore, this work provides a solid basis for further research in the area of application of code cloning detection results. In particular, code cloning detection results and the ideas we presented show potential to be extended and used to facilitate domain analysis.

Human language is both a cognitive and a cultural phenomenon. Any evolutionary account of language, then, must address both biological and cultural evolution. In this thesis, I give a mainly cultural evolutionary answer to two main questions: firstly, how do working systems of learned communication arise in populations in the absence of external or internal guidance? Secondly, how do those communication systems take on the fundamental structural properties found in human languages, i.e. systematicity at both a meaningless and meaningful level? A large, multi-disciplinary literature exists for each question, full of apparently conflicting results and analyses. My aim in this thesis is to survey this work, so as to find any commonalities and bring this together in order to provide a minimal account of the cultural evolution of language. The first chapter of this thesis takes a number of well-established models of the emergence of signalling systems. These are taken from several different fields: evolutionary linguistics, evolutionary game theory, philosophy, artificial life, and cognitive science. By using a common framework to directly compare these models, I show that three underlying commonalities determine the ability of any population of agents to reliably develop optimal signalling. The three requirements are that i) agents can create and transfer referential information, ii) there is a systemic bias against ambiguity, and iii) some mechanism leading to information loss exists. Following this, I extend the model to determine the effects of including referential uncertainty. I show that, for the group of models to which this applies, this places certain extra restrictions on the three requirements stated above. In the next chapter, I use an information-theoretic framework to construct a novel analysis of signalling games in general, and rephrase the three requirements in more formal terms. I then show that we can use these 3 criteria as a diagnostic for determining whether any given signalling game will lead to optimal signalling, without the requirement for repeated simulations. In the final, much longer, chapter, I address the topic of duality of patterning. This involves a lengthy review of the literature on duality of patterning, combinatoriality, and compositionality. I then argue that both levels of systematicity can be seen as a functional adaptation which maintains communicative accuracy in the face of noisy processes at different levels of analysis. I support this with results from a new, minimally-specified model, which also clarifies and informs a number of long-fought debates within the field.

The question of how language evolved has received an increasing amount of attention in recent years. Compared to seemingly more complex phenomena such as syntax, word meanings are usually seen as relatively easy to explain. Mainstream accounts in psycholinguistics and evolutionary linguistics assume that word meanings correspond to stable concepts which are prior to language and derive straightforwardly from human perception of structure in the world. Taking a cognitive linguistic approach based on psycholinguistic evidence, I argue instead that word meanings are conventions, grounded, learned and used in the context of communication. The meaning of a word is the sum of its contexts of use, with particular features of these contexts made more or less salient by mechanisms of attentional learning and communicative inference. Evolutionarily, word meanings arise as an emergent product of humans’ adapted tendency to infer each other’s intentions using contextual cues. They are then shaped over cultural evolution by the need to be learnable and useful for communication. This thesis presents a series of experiments that test the effect of these pressures on the origins and development of word meanings. Experiment 1 investigates the origins of strong tendencies for words to specify features on particular dimensions (such as the shape bias). The results show that these tendencies arise via attentional learning effects amplified by iterated learning. Dimensions which are less salient in contexts of learning and use drop out of word meanings as they are passed down a chain of learners. Experiments 2, 3 and 4 investigate the structure of word meanings produced during either paired communication games or individual labelling of images by similarity. While communication alone leads to word meanings that are unstructured and poorly aligned within pairs, communication plus iterated learning leads to word meanings that increase in structure and alignment over generations. Finally, Experiment 5 investigates the interaction of event structure and developing conventions in shaping word meanings. The structure of events in an artificial world is shown to influence lexicalisation patterns in the languages conventionalised by communicating pairs. Event features that are less predictable across communicative contexts tend to be more strongly associated with the conventions in the language. Overall, the experiments show that rather than straightforwardly reflecting pre-linguistic conceptualisation, word meanings are also dynamically shaped by learning and communication. In addition, these processes are constrained by the conventions that already exist within a language. This illuminates the mixture of convergence and diversity we see in word meanings in natural languages, and gives insight into their evolutionary origins.

Cultural artifacts, such as language, survive and replicate by passing from mind to mind. Cultural evolution always proceeds by an inductive process, where behaviors are never directly copied, but reverse engineered by the cognitive mechanisms involved in learning and production. I will refer to this type of evolutionary change as inductive evolution and explain how this represents a broader class of evolutionary processes that can include both neutral and selective evolution. This thesis takes a mechanistic approach to understanding the forces of evolution underlying change in culture over time, where the mechanisms of change are sought within human cognition. I define culture as anything that replicates by passing through a cognitive system and take language as a premier example of culture, because of the wealth of knowledge about linguistic behaviors (external language) and its cognitive processing mechanisms (internal language). Mainstream cultural evolution theories related to social learning and social transmission of information define culture ideationally, as the subset of socially-acquired information in cognition that affects behaviors. Their goal is to explain behaviors with culture and avoid circularity by defining behaviors as markedly not part of culture. I take a reductionistic approach and argue that all there is to culture is brain states and behaviors, and further, that a complete explanation of the forces of cultural change can not be explained by a subset of cognition related to social learning, but necessarily involves domain-general mechanisms, because cognition is an integrated system. Such an approach should decompose culture into its constituent parts and explore 1) how brains states effect behavior, 2) how behavior effects brain states, and 3) how brain states and behaviors change over time when they are linked up in a process of cultural transmission, where one person's behavior is the input to another. I conduct several psychological experiments on frequency learning with adult learners and describe the behavioral biases that alter the frequencies of linguistic variants over time. I also fit probabilistic models of cognition to participant data to understand the inductive biases at play during linguistic frequency learning. Using these inductive and behavioral biases, I infer a Markov model over my empirical data to extrapolate participants' behavior forward in cultural evolutionary time and determine equivalences (and divergences) between inductive evolution and standard models from population genetics. As a key divergence point, I introduce the concept of non-binomial cultural drift, argue that this is a rampant form of neutral evolution in culture, and empirically demonstrate that probability matching is one such inductive mechanism that results in non-binomial cultural drift. I argue further that all inductive problems involving representativeness are potential drivers of neutral evolution unique to cultural systems. I also explore deviations from probability matching and describe non-neutral evolution due to inductive regularization biases in a linguistic and non-linguistic domain. Here, I offer a new take on an old debate about the domain-specificity vs -generality of the cognitive mechanisms involved in language processing, and show that the evolution of regularity in language cannot be predicted in isolation from the general cognitive mechanisms involved in frequency learning. Using my empirical data on regularization vs probability matching, I demonstrate how the use of appropriate non-binomial null hypotheses offers us greater precision in determining the strength of selective forces in cultural evolution.

Nous apprenons très jeunes une quantité de règles nous permettant d'interagir avec d'autres personnes: des conventions sociales. Elles diffèrent des autres types d'apprentissage dans le sens où les premières personnes à les avoir utilisées n'ont fait qu'un choix arbitraire parmi plusieurs alternatives possibles: le côté de la route où conduire, la forme d'une prise électrique, ou inventer de nouveaux mots. À cause de celà, lorsqu'une nouvelle convention se crée au sein d'une population d'individus interagissant entre eux, de nombreuses alternatives peuvent apparaître et conduire à une situation complexe où plusieurs conventions équivalentes coexistent en compétition. Il peut devenir difficile de les retenir toutes, comment faisons-nous pour trouver un accord efficacement ? Nous exerçons communément un contrôle actif sur nos situations d'apprentissage, en par exemple sélectionnant des activités qui ne soient ni trop simples ni trop complexes. Il a été montré que ce type de comportement, dans des cas comme l'apprentissage sensori-moteur, aide à apprendre mieux, plus vite, et avec moins d'exemples. Est-ce que de tels mécanismes pourraient aussi influencer la négociation de conventions sociales? Le lexique est un exemple particulier de convention sociale: quels mots associer avec tel objet ou tel sens? Une classe de modèles computationels, les Language Games, montrent qu'il est possible pour une population d'individus de construire un langage commun via une série d'interactions par paires. En particulier, le modèle appelé Naming Game met l'accent sur la formation du lexique reliant mots et sens, et montre une typique explosion de la complexité avant de commencer à écarter les conventions synonymes ou homonymes et arriver à un consensus. Dans cette thèse, nous introduisons l'idée de l'apprentissage actif et du contrôle actif de la croissance de la complexité dans le Naming Game, sous la forme d'une politique de choix du sujet de conversation, applicable à chaque interaction. Différentes stratégies sont introduites, et ont des impacts différents sur à la fois le temps nécessaire pour converger vers un consensus et la quantité de mémoire nécessaire à chaque individu. Premièrement, nous limitons artificiellement la mémoire des agents pour éviter l'explosion de complexité locale. Quelques stratégies sont présentées, certaines ayant des propriétés similaires au cas standard en termes de temps de convergence. Dans un deuxième temps, nous formalisons ce que les agents doivent optimiser, en se basant sur une représentation de l'état moyen de la population. Deux stratégies inspirées de cette notion permettent de limiter les besoins en mémoire sans avoir à contraindre le système, et en prime permettent de converger plus rapidement. Nous montrons ensuite que la dynamique obtenue est proche d'un comportement théorique optimal, exprimé comme une borne inférieure au temps de convergence. Finalement, nous avons mis en place une expérience utilisateur en ligne sous forme de jeu pour collecter des données sur le comportement d'utilisateurs réels placés dans le cadre du modèle. Les résultats suggèrent qu'ils ont effectivement une politique active de choix de sujet de conversation, en comparaison avec un choix aléatoire.Les contributions de ce travail de thèse incluent aussi une classification des modèles de Naming Games existants, et un cadriciel open-source pour les simuler. / Social conventions are learned mostly at a young age, but are quite different from other domains, like for example sensorimotor skills. The first people to define conventions just picked an arbitrary alternative between several options: a side of the road to drive on, the design of an electric plug, or inventing a new word. Because of this, while setting a new convention in a population of interacting individuals, many competing options can arise, and lead to a situation of growing complexity if many parallel inventions happen. How do we deal with this issue?Humans often exhert an active control on their learning situation, by for example selecting activities that are neither too complex nor too simple. This behavior, in cases like sensorimotor learning, has been shown to help learn faster, better, and with fewer examples. Could such mechanisms also have an impact on the negotiation of social conventions ? A particular example of social convention is the lexicon: which words we associated with given meanings. Computational models of language emergence, called the Language Games, showed that it is possible for a population of agents to build a common language through only pairwise interactions. In particular, the Naming Game model focuses on the formation of the lexicon mapping words and meanings, and shows a typical burst of complexity before starting to discard options and find a final consensus. In this thesis, we introduce the idea of active learning and active control of complexity growth in the Naming Game, in the form of a topic choice policy: agents can choose the meaning they want to talk about in each interaction. Several strategies were introduced, and have a different impact on both the time needed to converge to a consensus and the amount of memory needed by individual agents. Firstly, we artificially constrain the memory of agents to avoid the local complexity burst. A few strategies are presented, some of which can have similar convergence speed as in the standard case. Secondly, we formalize what agents need to optimize, based on a representation of the average state of the population. A couple of strategies inspired by this notion help keep the memory usage low without having constraints, but also result in a faster convergence process. We then show that the obtained dynamics are close to an optimal behavior, expressed analytically as a lower bound to convergence time. Eventually, we designed an online user experiment to collect data on how humans would behave in the same model, which shows that they do have an active topic choice policy, and do not choose randomly. Contributions from this thesis also include a classification of the existing Naming Game models and an open-source framework to simulate them.

Human languages are not static entities. Linguistic conventions, whose social and communicative meaning are understood by all members of a speech community, are gradually altered or replaced, whether by changing their forms, meanings, or by the loss of or introduction of altogether new distinctions. How do large speech communities go about re-negotiating arbitrary associations in the absence of centralised coordination? This thesis first provides an overview of the plethora of explanations that have been given for language change. Approaching language change in a quantitative and evolutionary framework, mathematical and computational modelling is put forward as a tool to investigate and compare these different accounts and their purported underlying mechanisms in a rigorous fashion. The central part of the thesis investigates a relatively recent addition to the pool of mechanisms that have been proposed to influence language change: I will compare previous accounts with a momentum-based selection account of language change, a replicator-neutral model where the popularity of a variant is modulated by its momentum, i.e. its change in frequency of use in the recent past. I will discuss results from a multi-agent model which show that the dynamics of a trend-amplifying mechanism like this are characteristic of language change, in particular by exhibiting spontaneously generated s-shaped transitions. I will also discuss several empirical predictions made by a momentum-based selection account which contrast with those that can be derived from other accounts of language change. Going beyond theoretical arguments for the role of trends in language change, I will go on to present fieldwork data of speakers’ awareness of ongoing syntactic changes in the Shetland dialect of Scots. Data collected using a novel questionnaire methodology show that individuals possess explicit knowledge about the direction as well as current progression of ongoing changes, even for grammatical structures which are very low in frequency. These results complement previous experimental evidence which showed that individuals both possess and make use of implicit knowledge about age-dependent usage differences during ongoing sound changes. Echoing the literature on evolutionary approaches to language change, the final part of the thesis stresses the importance of explicitly situating different pressures either in the domain of the innovation of new or else the selection of existing variants. Based on a modification of the Wright-Fisher model from population genetics, I will argue that trend-amplification selection mechanisms provide predictions that neatly match empirical facts, both in terms of the diachronic dynamics of language change, as well as in terms of the synchronic distribution of linguistic traits that we find in the world.

In 1935 the linguist George Kingsley Zipf made a now classic observation about the relationship between a word's length and its frequency: the more frequent a word is, the shorter it tends to be. He claimed that this 'Law of Abbreviation' is a universal structural property of language. The Law of Abbreviation has since been documented in a wide range of human languages, and extended to animal communication systems and even computer programming languages. Zipf hypothesised that this universal design feature arises as a result of individuals optimising form-meaning mappings under competing pressures to communicate accurately but also efficiently - his famous Principle of Least Effort. In this thesis, I present a novel set of studies which provide direct experimental evidence for this explanatory hypothesis. Using a miniature artificial language learning paradigm, I show in Chapter 2 that language users optimise form-meaning mappings in line with the Law of Abbreviation only when pressures for accuracy and efficiency both operate during a communicative task. These results are robust across different methods of data collection: one version of the experiment was run in the lab, and another was run online, using a novel method I developed which allows participants to partake in dyadic interaction through a web-based interface. In Chapter 3, I address the growing body of work suggesting that a word's predictability in context may be an even stronger determiner of its length than its frequency alone. For instance, Piantadosi et al. (2011) show that shorter words have a lower average surprisal (i.e., tend to appear in more predictive contexts) than longer words, in synchronic corpora across many languages. We hypothesise that the same communicative pressures posited by the Principle of Least Effort, when acting on speakers in situations where context manipulates the information content of words, can give rise to these lexical distributions. Adapting the methodology developed in Chapter 2, I show that participants use shorter words in more predictive contexts only when subject to the competing pressures for accurate and efficient communication. In a second experiment, I show that participants are more likely to use shorter words for meanings with a lower average surprisal. These results suggest that communicative pressures acting on individuals during language use can lead to the re-mapping of a lexicon to align with 'Uniform Information Density', the principle that information content ought to be evenly spread across an utterance, such that shorter linguistic units carry less information than longer ones. Over generations, linguistic behaviour such as that observed in the experiments reported here may bring entire lexicons into alignment with the Law of Abbreviation and Uniform Information Density. For this to happen, a diachronic process which leads to permanent lexical change is necessary. However, crucial evidence for this process - decreasing word length as a result of increasing frequency over time - has never before been systematically documented in natural language. In Chapter 4, I conduct the first large-scale diachronic corpus study investigating the relationship between word length and frequency over time, using the Google Books Ngrams corpus and three different word lists covering both English and French. Focusing on words which have both long and short variants (e.g., info/information), I show that the frequency of a word lemma may influence the rate at which the shorter variant gains in popularity. This suggests that the lexicon as a whole may indeed be gradually evolving towards greater efficiency. Taken together, the behavioural and corpus-based evidence presented in this thesis supports the hypothesis that communicative pressures acting on language-users are at least partially responsible for the frequency-length and surprisal-length relationships found universally across lexicons. More generally, the approach taken in this thesis promotes a view of language as, among other things, an evolving, efficient code for information transfer.

Languages are culturally transmitted through a repeated cycle of learning and communicative interaction. These two aspects of cultural transmission impose (at least) three interacting pressures that can shape the evolution of linguistic structure: a pressure for learnability, a pressure for expressivity, and a pressure for coordination amongst users in a linguistic community. This thesis considers how these sometimes competing pressures impact linguistic complexity across cultural time. Using artificial language and iterated learning experimental paradigms, I investigate the conditions under which complexity in morphological and syntactic systems emerges, spreads, and reduces. These experiments illustrate the interaction of transmission, learning and use in hitherto understudied domains - morphosyntax and word order. In a first study (Chapter 2), I report the first iterated learning experiments to investigate the evolution of complexity in compositional structure at the word and sentence level. I demonstrate that a complex meaning space paired with pressures for learnability and communication can result in compositional hierarchical constituent structure, including fixed combinatorial rules of word formation and word order. This structure grants a productive and productively interpretable language and only requires learners to acquire a finite lexicon and a finite set of combinatorial rules (i.e., a grammar). In Chapter 3, I address the unique effect of communicative interaction on linguistic complexity, by removing language learning completely. Speakers use their native language to express novel meanings either in isolation or during communicative interaction. I demonstrate that even in this case, communicative interaction leads to more efficient and overall simpler linguistic systems. These first two studies provide support for the claim that morphological and syntactic complexity are shaped by an overarching drive towards simplicity (or learnability) in language learning and communication. Chapter 4 reports a series of experiments assessing the possibility that the simplicity bias found in the first two studies operates at a different strength depending on the linguistic level. Studies in natural language learning and in pidgin/creole genesis suggest that while morphological variation seems to be highly susceptible to regularisation, variation in other syntactic features, like word order, appears more likely to be reproduced. I test this experimentally by comparing regularisation of unconditioned variation across morphology and word order in the context of artificial language learning. I show that language users in fact regularise unconditioned variation in a similar way across linguistic levels, suggesting that the simplicity bias may be driven by a single, non-level-specific mechanism. Taken together, the experimental evidence presented in this thesis supports the hypothesis that the cultural and cognitive pressures acting on language users during learning and communicative interaction - for learnability, expressivity and coordination - are at least partially responsible for the evolution of linguistic complexity. Specifically, they are responsible for the emergence of linguistic complexity which maximises learnability and communicative efficiency, and for the reduction of complexity which does not. More generally, the approach taken in this thesis promotes a view of complexity in linguistic systems as an evolving variable determined by the biases of language learners and users as languages are culturally transmitted.

No description available.

No description available.