An interactive narrative is a form of digital entertainment in which players can create or influence a dramatic storyline through actions, typically by assuming the role of a character in a fictional virtual world. The interactive narrative systems usually employ a drama manager (DM), an omniscient background agent that monitors the fictional world and determines what will happen next in the players' story experience. Prevailing approaches to drama management choose successive story plot points based on a set of criteria given by the game designers. In other words, the DM is a surrogate for the game designers. In this dissertation, I create a data-driven personalized drama manager that takes into consideration players' preferences. The personalized drama manager is capable of (1) modeling the players' preference over successive plot points from the players' feedback; (2) guiding the players towards selected plot points without sacrificing players' agency; (3) choosing target successive plot points that simultaneously increase the player's story preference ratings and the probability of the players selecting the plot points. To address the first problem, I develop a collaborative filtering algorithm that takes into account the specific sequence (or history) of experienced plot points when modeling players' preferences for future plot points. Unlike the traditional collaborative filtering algorithms that make one-shot recommendations of complete story artifacts (e.g., books, movies), the collaborative filtering algorithm I develop is a sequential recommendation algorithm that makes every successive recommendation based on all previous recommendations. To address the second problem, I create a multi-option branching story graph that allows multiple options to point to each plot point. The personalized DM working in the multi-option branching story graph can influence the players to make choices that coincide with the trajectories selected by the DM, while gives the players the full agency to make any selection that leads to any plot point in their own judgement. To address the third problem, the personalized DM models the probability that the players transitioning to each full-length stories and selects target stories that achieve the highest expected preference ratings at every branching point in the story space. The personalized DM is implemented in an interactive narrative system built with choose-your-own-adventure stories. Human study results show that the personalized DM can achieve significantly higher preference ratings than non-personalized DMs or DMs with pre-defined player types, while preserve the players' sense of agency.

In this thesis we present two approaches to improve automatic playtesting using player modeling. By modeling various cohorts of players we are able to train Convolutional Neural Network based agents that simulate human gameplay using different strategies directly learnt from real player data. The goal is to use the developed agents to predict useful metrics of newly created game content. We validated our approaches using the game Candy Crush Saga, a non-deterministic match-three puzzle game with a huge search space and more than three thousand levels available. To the best of our knowledge this is the first time that player modeling is applied in a match-three puzzle game. Nevertheless, the presented approaches are general and can be extended to other games as well. The proposed methods are compared to a baseline approach that simulates gameplay using a single strategy learnt from random gameplay data. Results show that by simulating different strategies, our approaches can more accurately predict the level difficulty, measured as the players’ success rate, on new levels. Both the approaches improved the mean absolute error by 13% and the mean squared error by approximately 23% when predicting with linear regression models. Furthermore, the proposed approaches can provide useful insights to better understand the players and the game. / I denna uppsats presenterar vi två tillvägagångssätt för att förbättra automatisk speltestning genom modellering av spelare. Genom att modellera olika grupper av spelare kunde vi träna Convolutional Neural Network-baserade agenter för att simulera mänskligt spelande med hjälp av olika strategier som är lärda direkt från mänsklig spelardata. Målet är att använda de utvecklade agenterna för att förutsäga användbar metrik av nyskapat spelinnehåll. Vi validerade vårt tillvägagångssätt genom Candy Crush Saga, ett icke-deterministiskt 3-matchnings pusselspel med mer än tre tusen nivåer. Detta är första gången som spelarmodellering appliceras på ett 3-matchnings pusselspel. De presenterade tillvägagångssätten är mer generella och kan utökas till andra spel. De föreslagna tillvägagångssätten är jämförda med ett tillvägagångssätt som simulerar spelande genom en strategi som är lärd direkt från slumpmässig mänsklig spelardata. Resultatet visar att vårt tillvägagångssätt, genom simulering av olika strategier är, mer exakt för att förutsäga spelarens svårighet, mätt genom spelarens framgång, på nya nivåer. Båda tillvägagångssätten förbättrade mean absolute error med 13% och mean squared error med ungefär 23%. Dessutom kan de föreslagna tillvägagångssätten ge en användbar insikt för att bättre förstå spelarna och spelet.

Humans procrastinate and avoid performing activities that they deem dull, repetitive, and out of their comfort zone. Gamification was conceived to reverse the situation by turning those activities into fun and entertaining actions exploiting game-like elements. In practice, however, many challenges arise. Gameful environments cannot satisfy every player's preference and motivational need with a one-fits-all strategy. However, meeting players' motivational affordances can provide intrinsic rewards rather than extrinsic (e.g., points and badges). Producing intrinsic rewards is desirable as they are more likely to foster long-term retention than the extrinsic counterpart. Therefore, gamified systems should be designed to learn and understand players' preferences and motivational drivers to generate specific adaptation strategies for each player. Those adaptation strategies govern the procedural generation of personalized game elements - examples are task difficulty, social-play versus solo-play, or aesthetic tools. However, an appropriate personalization requires intelligent and effective player profiling mechanisms. Player profiles can be retrieved through the analysis of telemetry data, and thus in-game behaviors. In this project, we studied players' individual and social behaviors to understand their personalities and identities within the game. Specifically, we analyzed data from an open-world, persuasive, gamified system: Play&Go. Play&Go implements game-like mechanics to instill more ecological transportation habits among its users. The gamified app offers various ways for players to interact with the game and among one another. Despite Play&Go being one of the few examples of gamification implementing more diverse game mechanics than solely points and leaderboards, it still does not reach the complexity of AAA entertainment games. Thus, it limits the applicability of an in-depth analysis of players' behaviors, constrained by the type of available features. Yet, we argue that gameful systems still provide enough information to allow content adaptation. In this work, we study players' in-game activity from different perspectives to explore gamification's potential. Towards this, we analyzed telemetry data to (1) learn from players' activity, (2) extract their profiles, and (3) understand social dynamics in force within the game. Our results show how players' experience in gamified systems is closer to games than expected, especially in social environments. Hence, telemetry data is a precious source of knowledge also in gamification and can help retain information on players' churn, preferences, and social influence. Finally, we propose a modular theoretical framework for adaptive gamification to generate personalized content designed to learn players' preferences iteratively.

Um dos desafios da Inteligência Artificial aplicada em jogos é o aprendizado de comportamento, em que o objetivo é utilizar estatísticas obtidas da interação entre jogador e jogo de modo a reconhecer características particulares de um jogador ou monitorar a evolução de seu comportamento no decorrer do tempo. A maior parte dos trabalhos na área emprega modelos previamente aprendidos, por meio da utilização de algoritmos de Aprendizado de Máquina. Entretanto, são poucos os trabalhos que consideram que o comportamento de um jogador pode evoluir no tempo e que, portanto, reconhecer quando essas mudanças ocorrem é o primeiro passo para produzir jogos que se adaptam automaticamente às capacidades do jogador. Para detectar variações comportamentais em um jogador, são necessários algoritmos que processem dados de modo incremental. Esse pré-requisito motiva o estudo de algoritmos para detecção de mudanças da área de Mineração em Fluxos Contínuos de Dados. Entretanto, algumas das características dos algoritmos disponíveis na literatura inviabilizam sua aplicação direta ao problema de detecção de mudança em jogos. Visando contornar essas dificuldades, esta tese propõe duas novas abordagens para detecção de mudanças de comportamento. A primeira abordagem é baseada em um algoritmo incremental de agrupamento e detecção de novidades que é independente do número e formato dos grupos presentes nos dados e que utiliza um mecanismo de janela deslizante para detecção de mudanças de comportamento. A segunda abordagem, por outro lado, é baseada na comparação de janelas de tempo consecutivas utilizando espectrogramas gerados a partir dos dados contidos em cada janela. Os resultados experimentais utilizando simulações e dados de jogos comerciais indicam a aplicabilidade dos algoritmos propostos na tarefa de detecção de mudanças de comportamento de um jogador, assim como mostram sua vantagem em relação a outros algoritmos para detecção de mudança disponíveis na literatura / One of the challenges of Artificial Intelligence applied to games is behavior learning, where the objective is to use statistics derived from the interaction between the player and the game environment in order to recognize particular player characteristics or to monitor the evolution of a players behavior along time. The majority of work developed in this area applies models that were previously learned through the use of Machine Learning techniques. However, only a few pieces of work consider that the players behavior can evolve over time and, therefore, recognizing when behavior changes happen is the first step towards the production of games that adapt to the players needs. In order to detect changes in the behavior of a player, incremental algorithms are necessary, what motivates the study of change detection algorithms from the area of Data Stream Mining. However, some of the characteristics of the algorithms available in the literature make their application to the task of change detection in games unfeasible. To overcome these difficulties, this work proposes two new approaches for change detection. The first approach is based on an incremental clustering and novelty detection algorithm which is independent of the number and format of clusters and uses a mechanism for change detection based on sliding windows. The second approach, on the other hand, is based on the comparison of consecutive time windows using spectrograms created from the data inside each window. Experimental results using simulations and data from commercial games indicate the applicability of the proposed algorithms in the task of detecting a players changing behavior, as well as present their advantage when compared to other change detection algorithms available in the literature

Um dos desafios da Inteligência Artificial aplicada em jogos é o aprendizado de comportamento, em que o objetivo é utilizar estatísticas obtidas da interação entre jogador e jogo de modo a reconhecer características particulares de um jogador ou monitorar a evolução de seu comportamento no decorrer do tempo. A maior parte dos trabalhos na área emprega modelos previamente aprendidos, por meio da utilização de algoritmos de Aprendizado de Máquina. Entretanto, são poucos os trabalhos que consideram que o comportamento de um jogador pode evoluir no tempo e que, portanto, reconhecer quando essas mudanças ocorrem é o primeiro passo para produzir jogos que se adaptam automaticamente às capacidades do jogador. Para detectar variações comportamentais em um jogador, são necessários algoritmos que processem dados de modo incremental. Esse pré-requisito motiva o estudo de algoritmos para detecção de mudanças da área de Mineração em Fluxos Contínuos de Dados. Entretanto, algumas das características dos algoritmos disponíveis na literatura inviabilizam sua aplicação direta ao problema de detecção de mudança em jogos. Visando contornar essas dificuldades, esta tese propõe duas novas abordagens para detecção de mudanças de comportamento. A primeira abordagem é baseada em um algoritmo incremental de agrupamento e detecção de novidades que é independente do número e formato dos grupos presentes nos dados e que utiliza um mecanismo de janela deslizante para detecção de mudanças de comportamento. A segunda abordagem, por outro lado, é baseada na comparação de janelas de tempo consecutivas utilizando espectrogramas gerados a partir dos dados contidos em cada janela. Os resultados experimentais utilizando simulações e dados de jogos comerciais indicam a aplicabilidade dos algoritmos propostos na tarefa de detecção de mudanças de comportamento de um jogador, assim como mostram sua vantagem em relação a outros algoritmos para detecção de mudança disponíveis na literatura / One of the challenges of Artificial Intelligence applied to games is behavior learning, where the objective is to use statistics derived from the interaction between the player and the game environment in order to recognize particular player characteristics or to monitor the evolution of a players behavior along time. The majority of work developed in this area applies models that were previously learned through the use of Machine Learning techniques. However, only a few pieces of work consider that the players behavior can evolve over time and, therefore, recognizing when behavior changes happen is the first step towards the production of games that adapt to the players needs. In order to detect changes in the behavior of a player, incremental algorithms are necessary, what motivates the study of change detection algorithms from the area of Data Stream Mining. However, some of the characteristics of the algorithms available in the literature make their application to the task of change detection in games unfeasible. To overcome these difficulties, this work proposes two new approaches for change detection. The first approach is based on an incremental clustering and novelty detection algorithm which is independent of the number and format of clusters and uses a mechanism for change detection based on sliding windows. The second approach, on the other hand, is based on the comparison of consecutive time windows using spectrograms created from the data inside each window. Experimental results using simulations and data from commercial games indicate the applicability of the proposed algorithms in the task of detecting a players changing behavior, as well as present their advantage when compared to other change detection algorithms available in the literature