[pt] A tarefa de Processamento de Linguagem Natural consiste em analisar linguagens naturais de forma computacional, facilitando o desenvolvimento de programas capazes de utilizar dados falados ou escritos. Uma das tarefas mais importantes deste campo é a Análise de Dependência. Tal tarefa consiste em analisar a estrutura gramatical de frases visando extrair aprender dados sobre suas relações de dependência. Em uma sentença, essas relações se apresentam em formato de árvore, onde todas as palavras são interdependentes. Devido ao seu uso em uma grande variedade de aplicações como Tradução Automática e Identificação de Papéis Semânticos, diversas pesquisas com diferentes abordagens são feitas nessa área visando melhorar a acurácia das árvores previstas. Uma das abordagens em questão consiste em encarar o problema como uma tarefa de classificação de tokens e dividi-la em três classificadores diferentes, um para cada sub-tarefa, e depois juntar seus resultados de forma incremental. As sub-tarefas consistem em classificar, para cada par de palavras que possuam relação paidependente, a classe gramatical do pai, a posição relativa entre os dois e a distância relativa entre as palavras. Porém, observando pesquisas anteriores nessa abordagem, notamos que o gargalo está na terceira sub-tarefa, a predição da distância entre os tokens. Redes Neurais Recorrentes são modelos que nos permitem trabalhar utilizando sequências de vetores, tornando viáveis problemas de classificação onde tanto a entrada quanto a saída do problema são sequenciais, fazendo delas uma escolha natural para o problema. Esse trabalho utiliza-se de Redes Neurais Recorrentes, em específico Long Short-Term Memory, para realizar a tarefa de predição da distância entre palavras que possuam relações de dependência como um problema de classificação sequence-to-sequence. Para sua avaliação empírica, este trabalho segue a linha de pesquisas anteriores e utiliza os dados do corpus em português disponibilizado pela Conference on Computational Natural Language Learning 2006 Shared Task. O modelo resultante alcança 95.27 por cento de precisão, resultado que é melhor do que o obtido por pesquisas feitas anteriormente para o modelo incremental. / [en] The Natural Language Processing task consists of analyzing the grammatical structure of a sentence written in natural language aiming to learn, identify and extract information related to its dependency structure. This data can be structured like a tree, since every word in a sentence has a head-dependent relation to another word from the same sentence. Since Dependency Parsing is used in many applications like Machine Translation, Semantic Role Labeling and Part-Of-Speech Tagging, researchers aiming to improve the accuracy on their models are approaching this task in many different ways. One of the approaches consists in looking at this task as a token classification problem, using different classifiers for each sub-task and joining them in an incremental way. These sub-tasks consist in classifying, for each head-dependent pair, the Part-Of-Speech tag of the head, the relative position between the two words and the distance between them. However, previous researches using this approach show that the bottleneck lies in the distance classifier. Recurrent Neural Networks are a kind of Neural Network that allows us to work using sequences of vectors, allowing for classification problems where both our input and output are sequences, making them a great choice for the problem at hand. This work studies the use of Recurrent Neural Networks, in specific Long Short-Term Memory networks, for the head-dependent distance classifier sub-task as a sequence-to-sequence classification problem. To evaluate its efficiency, this work follows the line of previous researches and makes use of the Portuguese corpus of the Conference on Computational Natural Language Learning 2006 Shared Task. The resulting model attains 95.27 percent precision, which is better than the previous results obtained using incremental models.

The purposes of the present study were to investigate the relationship between short-term memory and reading in learning disabled and average learners, and to determine whether this relationship is different between ages 8 to 10 and ages 11 to 13 in these two populations. Studies have shown that children with learning disabilities tend to perform poorer on short-term memory tasks compared to children with no disabilities. The present study was conducted because the short-term memory component in the Stanford-Binet Intelligence Scale is new and it was felt that information regarding this test's usefulness with learning disabled students would be beneficial for individuals in the field of educational assessment. A total of 80 children, 39 average and 41 learning disabled were selected from the five public elementary schools that have learning disabilities classes in the Langley School District. For each group of learning disabled children selected from the learning disabilities class, an equal number of average learners was chosen from the same school. The children were divided into two age groups: 8- to 10-year-olds and 11- to 13-year-olds and then further divided into their two learning categories. Four short-term memory subtests of the Stanford-Binet Intelligence Scale: Fourth Edition: Bead Memory, Memory for Sentences, Memory for Digits and Memory for Objects and three reading comprehension subtests, from B.C. QUick Individual Educational Test, Peabody Individual Achievement Test and Test of Reading Comprehension respectively, were administered to all groups to measure short-term memory and reading. The Multivariate Analysis of Variance and the Pearson Product-Moment Correlation were used to analyse the data. Results showed that the average learners scored significantly higher than the learning disabled group in both short-term memory and reading. There was no interaction effect of learning group and age on reading or short-term memory. Significant relationships were found between short-term mmeory and reading for the average learning group but none was found for the learning disabled group. / Education, Faculty of / Graduate

[pt] Prever emprego e desemprego é de grande importância para praticamente todos os agentes de uma economia. Emprego é uma das principais variáveis analisadas como indicador econômico, e desemprego serve para os policy makers como uma orientação às suas decisões. Neste trabalho, eu estudo quais características das duas séries podemos usar para auxiliar no tratamento dos dados e métodos empregados para auxiliar no poder preditivo das mesmas. Eu comparo modelos de machine (Random Forest e Lasso Adaptativo) e Deep (Long short Term memory) learning, procurando capturar as não linearidades e dinâmicas de ambas séries. Os resultados encontrados sugerem que o modelo AR com Random Forest aplicado nos resíduos, como uma maneira de separar parte linear e não linear, é o melhor modelo para previsão de emprego, enquanto Random Forest e AdaLasso com Random Forest aplicado nos resíduos são os melhores para o desemprego. / [en] Forecasting employment and unemployment is of great importance for virtually all agents in the economy. Employment is one of the main variables analyzed as an economic indicator, and unemployment serves to policy makers as a guide to their actions. In this essay, I study what features of both series we can use on data treatment and methods used to add to the forecasting predictive power. Using an AR model as a benchmark, I compare machine (Random Forest and Adaptive Lasso) and deep (Long Short Term Memory) learning methods, seeking to capture non-linearities of both series dynamics. The results suggests that an AR model with a Random Forest on residuals (as a way to separate linear and non-linear part) is the best model for employment forecast, while Random Forest and AdaLasso with Random Forest on residuals were the best for unemployment forecast.

Human activity data is typically represented as time series data, and RNNs, often with LSTM cells, are commonly used for recognition in this field. However, RNNs and LSTM-RNNs are often too resource-intensive for real-time applications on resource constrained devices, making them unsuitable. This thesis project is carried out at Wrlds AB, Stockholm. At Wrlds, all machine learning is run in the cloud, but they have been attempting to run their AI algorithms on their embedded devices. The main task of this project was to investigate alternative network structures to minimize the size of the networks to be used on human activity data. This thesis investigates the use of Fast GRNN, a deep learning algorithm developed by Microsoft researchers, to classify human activity on resource-constrained devices. The FastGRNN algorithm was compared to state-of-the-art RNNs, LSTM, GRU, and Simple RNN in terms of accuracy, classification time, memory usage, and energy consumption. This research is limited to implementing the FastRNN algorithm on Nordic SoCs using their SDK and TensorFlow Lite Micro. The result of this thesis shows that the proposed network has similar performance as LSTM networks in terms of accuracy while being both considerably smaller and faster, making it a promising solution for human activity recognition on embedded devices with limited computational resources and merits further investigation. / Rörelse igenkännings analys är oftast representerat av tidsseriedata där ett RNN modell meden LSTM arkitektur är oftast den självklara vägen att ta. Dock så är denna arkitektur väldigt resurskrävande för applikationer i realtid och gör att det uppstår problem med resursbegränsad hårdvara. Detta examensarbete är utfört i samarbete med Wrlds Technologies AB. På Wrlds så körs deras maskin inlärningsmodeller på molnet och lokalt på mobiltelefoner. Wrlds har nu påbörjat en resa för att kunna köra modeller direkt på små inbyggda system. Examensarbete kommer att utvärdera en FastGRNN som är en NN-arkitektur utvecklad av Microsoft i syfte att användas på resurs begränsad hårdvara. FastGRNN algoritmen jämfördes med andra högkvalitativa arkitekturer som RNNs, LSTM, GRU och en simpel RNN. Träffsäkerhet, klassifikationstid, minnesanvändning samt energikonsumtion användes för att jämföra dom olika varianterna. Detta arbete kommer bara att utvärdera en FastGRNN algoritm på en Nordic SoCs och kommer att användas deras SDK samt Tensorflow Lite Micro. Resultatet från detta examensarbete visar att det utvärderade nätverket har liknande prestanda som ett LSTM nätverk men också att nätverket är betydligt mindre i storlek och därmed snabbare. Detta betyder att ett FastGRNN visar lovande resultat för användningen av rörelseigenkänning på inbyggda system med begränsad prestanda kapacitet.

This study aims to investigate the implementation of an AI model that predicts customer purchases, in the telecom industry. The thesis also outlines how such an AI model can assist decision-making in marketing strategies. It is concluded that designing the AI model by following a Recurrent Neural Network (RNN) architecture with a Long Short-Term Memory (LSTM) layer, allow for a successful implementation with satisfactory model performances. Stepwise instructions to construct such model is presented in the methodology section of the study. The RNN-LSTM model further serves as an assisting tool for marketers to assess how a consumer’s website behavior affect their purchase behavior over time, in a quantitative way - by observing what the authors refer to as the Customer Purchase Propensity Journey (CPPJ). The firm empirical basis of CPPJ, can help organizations improve their allocation of marketing resources, as well as benefit the organization’s online presence by allowing for personalization of the customer experience. / Denna studie undersöker implementeringen av en AI-modell som förutspår kunders köp, inom telekombranschen. Studien syftar även till att påvisa hur en sådan AI-modell kan understödja beslutsfattande i marknadsföringsstrategier. Genom att designa AI-modellen med en Recurrent Neural Network (RNN) arkitektur med ett Long Short-Term Memory (LSTM) lager, drar studien slutsatsen att en sådan design möjliggör en framgångsrik implementering med tillfredsställande modellprestation. Instruktioner erhålls stegvis för att konstruera modellen i studiens metodikavsnitt. RNN-LSTM-modellen kan med fördel användas som ett hjälpande verktyg till marknadsförare för att bedöma hur en kunds beteendemönster på en hemsida påverkar deras köpbeteende över tiden, på ett kvantitativt sätt - genom att observera det ramverk som författarna kallar för Kundköpbenägenhetsresan, på engelska Customer Purchase Propensity Journey (CPPJ). Den empiriska grunden av CPPJ kan hjälpa organisationer att förbättra allokeringen av marknadsföringsresurser, samt gynna deras digitala närvaro genom att möjliggöra mer relevant personalisering i kundupplevelsen.

This thesis investigates the effectiveness of employing pretraining and a discrete "control signal" bottleneck layer in a neural network trained in aircraft navigation through deep reinforcement learning. The study defines two distinct tasks to assess the efficacy of this approach. The first task is utilized for pretraining specific parts of the network, while the second task evaluates the potential benefits of this technique. The experimental findings indicate that the network successfully learned three main macro actions during pretraining. flying straight ahead, turning left, and turning right, and achieved high rewards on the task. However, utilizing the pretrained network on the transfer task yielded poor performance, possibly due to the limited effective action space or deficiencies in the training process. The study discusses several potential solutions, such as incorporating multiple pretraining tasks and alterations of the training process as avenues for future research. Overall, this study highlights the challanges and opportunities associated with combining pretraining with a discrete bottleneck layer in the context of simulated aircraft navigation using reinforcement learning. / Denna studie undersöker effektiviteten av att använda förträning och en diskret "styrsignal" som fungerar som flaskhals i ett neuralt nätverk tränat i flygnavigering med hjälp av djup förstärkande inlärning. Studien definierar två olika uppgifter för att bedöma effektiviteten hos denna metod. Den första uppgiften används för att förträna specifika delar at nätverket, medan den andra uppgiften utvärderar de potentiella fördelarna med denna teknik. De experimentella resultaten indikerar att nätverket framgångsrikt lärde sig tre huvudsakliga makrohandlingar under förträningen: att flyga rakt fram, att svänga vänster och att svänga höger, och uppnådde höga belöningar för uppgiften. Men att använda det förtränade nätverket för den uppföljande uppgiften gav dålig prestation, möjligen på grund av det begränsade effektiva handlingsutrymmet eller begränsningar i träningsprocessen. Studien diskuterar flera potentiella lösningar, såsom att inkorporera flera förträningsuppgifter och ändringar i träningsprocessen, som möjliga framtida forskningsvägar. Sammantaget belyser denna studie de utmaningar och möjligheter som är förknippade med att kombinera förträning med ett diskret flaskhalslager inom kontexten av simulerad flygnavigering och förstärkningsinlärning.

Have you ever heard a song that hasn’t played in years, and immediately recognize it? What cognitive processes determine this, and why does it seemingly happen to everyone? Using the Expectancy Theory of Music (Meyer, 1965) a working explanation for the possibility of why such strange phenomena exists is proposed. Based on expectancy, words and melody are processed together, and sung words are treated as part of the expected whole. In three experiments, memory was tested using same- different task. Each experiments investigates a different level of memory. Taking into account systematic uncertainty and the violation of expectancy when an unexpected appears, these experiments were able to be analyzed and studied in regards to their effects on memory. College students from the Claremont Colleges are to be randomly selected for this experiment. Findings should show a consistent interaction between melody and vocal sequences throughout each experiment.

It is increasingly recognised that, in adulthood, attentional control plays an important role in optimising the ability to encode and maintain items in visual short-term memory (VSTM). Memory capacity limits increase dramatically over childhood, but the mechanisms through which children guide attention to maximise VSTM remain poorly understood. Through a number of experiments manipulating different parameters, the current thesis aimed to explore the developmental trajectories of the neurocognitive mechanisms underlying selective attention within VSTM and to examine whether variations in attentional control are accompanied by individual differences in VSTM capacity. Chapters 2 and 3 investigated the development of attentional orienting in preparation for encoding and during maintenance. Younger children emerged as less able than older children and adults to orient attention to internally held representations. Therefore, Chapter 4 tested whether younger children’s attentional orienting is differentially affected by memory load. While attentional orienting prior to encoding was more beneficial when required to remember a greater number of items, cueing benefits during maintenance were similar across load conditions. Chapter 5 investigated whether temporal parameters influence younger children’s variable ability to orient attention during maintenance. Attentional orienting operated more efficiently on transient iconic traces rather than on VSTM representations due to passive decay of the memory traces as a function of time. Chapter 6 assessed whether the characteristics of the memoranda constrain the efficiency of attentional orienting within VSTM. Attentional orienting supported differentially the maintenance of familiar and meaningless items and pinpointed the quantitative improvement of mnemonic strategies over development. Finally, Chapter 7 examined the temporal dynamics of prospective and retrospective orienting of attention in VSTM. Children deployed neural pathways underpinning attentional orienting less efficiently than adults and differentially across the two orienting conditions suggesting their neural dissociation. Overall, findings from the current thesis define how children develop the ability to deploy attentional control in service of VSTM.

It is now widely accepted that the experience of pain is subject to cognitive influences that may determine the severity of subjectively perceived pain. Many of these top-down factors rely on memory-based processes, which in turn are related to prior experience, learned beliefs and behaviours about pain. As such, memory for pain heavily contributes to the physical pain experience. We posit that pain memory is bidirectional in that following each painful event a trace is stored and that these traces in turn may modify future pain perception prospectively. The following body of work explores aspects of what we have termed a memory template for pain. The results of these chapters taken together, examine these bidirectional aspects of short-term memory for pain employing a recall pain task. Specifically, we explore how, after an acute pain event, a short-term mental representation of the initial event persists. We show that during this time, sensory re-experiencing of the painful event is possible. Furthermore, we investigate aspects of recalled pain, namely intensity and vividness. Data suggests that the intensity and the vividness of this mental representation are determined by the intensity of the initial stimulus, as well as the time-to-test delay. We identify regions that characterise short-term memory for pain. Following on from studies in motor and visual imagery, we explore how pain imagery in the form of recall may affect subsequent pain perception. Our results demonstrate that the inclusion of pain-related imagery preceding physical pain events reduces affective qualities of pain. Testing healthy, naïve subjects, we replicate the effect observed in studies using attention management and imagery strategies, which normally require extensive training. Finally, in a cohort of neuropathic pain patients we show significant reductions in white matter connectivity between areas responsible for working and prospective memory. Collectively, these studies emphasise and elucidate the role of short-term memory of pain in physical pain perception. Acting both retrospectively and prospectively, cognitive reinforcement can increase or decrease the subjective feeling of pain, and therefore manipulating how pain is recalled may have therapeutic potential.

Thesis (M.A (Social and Psychological Research))--University of the Witwatersrand, Faculty of Humanities, School of Human & Community Development, 2016 / Due to linguistic diversity within South Africa, multilingualism is becoming increasingly prominent. Since South Africa is host to 11 official languages, it is the norm rather than the exception that South Africans are exposed to more than one language. This has social, educational and cognitive implications. Specifically, research indicates that the acquisition of additional languages to an individual’s mother tongue has a positive effect on working memory – the short-term storage and manipulation of information during the performance of cognitive tasks – which may confer a ‘bi(multi)lingual advantage’ and could improve academic performance. Consequently, the aim of this study was to determine whether working memory ability differs significantly between students who are monolingual or multilingual, while statistically controlling for intellectual ability and socio-economic status between these groups. Participants were 78 undergraduate students, comprising English first- (monolingual, Mage = 20.06 years, SD = .88) and second- or additional-language (multilingual, Mage = 20.03 years, SD = 1.03) speakers, matched for age, gender and socio-economic status. Language groups were compared on the Automated Working Memory Assessment (Alloway, 2007) and subtests of the Wechsler Adult Intelligence Scale – Third Edition (Wechsler, 1997). One-way between-group ANCOVAs showed that (a) the multilingual group outperformed the monolingual group across five of six non-verbal subtests, namely Mazes Memory and Block Recall (non-verbal simple span), and Odd One Out, Mister X and Spatial Recall (non-verbal complex span), (b) the multilingual group outperformed the monolingual group on two verbal subtests, namely Digit Recall (verbal simple span) and Listening Recall (verbal complex span), (c) the language groups performed equivalently on verbal simple and complex tasks of Word Recall, Non-word Recall, Counting Recall and Backwards Digit Recall. The findings contribute to the extant literature confirming a ‘bi(multi)lingual advantage’ in executive functioning. Theoretical and practical implications are discussed in light of academic performance. Keywords: working memory, monolingualism, multilingualism, bi(multi)lingual advantage, South Africa