Many persons with aphasia (PWA), who have trouble communicating after a stroke, have difficulty naming objects, frequently producing speech errors. Picture (confrontation) naming tasks are commonly used to assess the presence and/or severity of naming difficulties, but these tests do not adequately capture the underlying cause of impairment. This project addresses the limitations of the standard picture naming paradigm by incorporating the measurement of eye movements, thereby providing a precise estimate of participants’ visual attention during the task. While prior studies have measured eye movements to distractor pictures when a spoken word is presented, to our knowledge no eye tracking studies have examined picture naming with written distractors in aphasia. Using a novel approach, we measured PWA’s and healthy controls’ eye movements as they selected the correct written word corresponding to the picture over other related words (semantically and sound-based distractors). The results of this project seek to: (1) indicate the feasibility of a novel eye tracking paradigm to study both intact and impaired lexical retrieval; (2) provide detailed information about the nature and time course of impaired naming; and (3) yield insight into the relative preservation of semantic and phonological representations in aphasia.

Longitudinal growth patterns are routinely seen in medical studies where developments of individuals on one or more outcome variables are followed over a period of time. Many current methods for modeling growth presuppose a parametric relationship between the outcome and time (e.g., linear, quadratic); however, these relationships may not accurately capture growth over time. Functional mixed effects (FME) models provide flexibility in handling longitudinal data with nonparametric temporal trends because they allow the data to determine the shape of the curve. Although FME methods are well-developed for continuous, normally distributed outcome measures, nonparametric methods for handling categorical outcomes are limited. In this thesis, we propose a Bayesian hierarchical FME model to account for growth curves with non-Gaussian outcomes. In particular, we extend traditional FME models which assume normally distributed outcomes by modeling the probabilities associated with the binomially distributed outcomes and adding an additional level to the hierarchical model to correctly specify the outcomes as binomially distributed. We then extend the proposed binomial FME model to the multinomial setting where the outcomes consist of more than two nominal categories. Current modeling approaches include modeling each category of a multinomial outcome separately via linear and nonlinear mixed effects models; yet, these approaches ignore the inherent correlation among the categories of the outcome. Our model captures this correlation through a sequence of conditional binomial FME models which results in one model simultaneously estimating probabilities in all categories. Lastly, we extend our binomial FME model to address a common medical situation where multiple outcomes are measured on subjects over time and investigators are interested in simultaneously assessing the impact of all outcomes. We account for the relationship between outcomes by altering the correlation structure in the hierarchical model and simultaneously estimating the outcome curves. Our methods are assessed via simulation studies and real data analyses where we investigate the ability of the models to accurately predict the underlying growth trajectory of individuals and populations. Our applications include analyses of speech development data in adults and children with cochlear implants and analyses on eye-tracking data used to assess word processing in cochlear implant patients.

Three eye-tracking experiments were conducted to explore the effects of phonological, semantic and root activation in spoken word recognition (SWR) in Saudi Arabian Arabic. Arabic roots involve both phonological and semantic information, therefore, a series of three studies were conducted to isolate the effect of the root independently from phonological and semantic effects. Each experiment consisted of a series of trials. On each trial, participants were presented with a display with four images: a target, a competitor, and two unrelated images. Participants were asked to click on the target image. Participants' proportional fixations to the four areas of interest and their reaction times (RT) were automatically recorded and analyzed. The assumption is that eye movements to the different types of images and RTs reflect degrees of lexical activation. Experiment 1 served as a foundation study to explore the nature of phonological, semantic and root activation. Experiment 2A and 2B aimed to explore the effect of the Arabic root as a function of semantic transparency and phonological onset similarity. Growth Curve Analyses (Mirman, 2014, GCA;) were used to analyze differences in target and competitor fixations across conditions. Results of these experiments highlight the importance of phonological, semantic and root effects in SWR in Arabic. Fixations to competitors were graded and corresponded to the different amounts of phonological, semantic and morphological overlap between targets and competitors. The results of this work highlight the importance of the Arabic consonantal root as an independent processing unit in lexical access in SWR in Arabic that is separable from phonological and semantic units of processing. Finally, the results of this work provided support to models of SWR that feature both whole-word processing as well as morphological decomposition (e.g. Baayen, Dijkstra, & Schreuder, 1997; Giraudo & Grainger, 2000; Schreuder & Baayen, 1997). They also provide support to the morpheme-based theory of Arabic morphology (McCarthy, 1979, 1981).

Most models of auditory word recognition describe the activation of lexical items in a continuous and graded manner. Much evidence in favor of these models comes from the visual-world paradigm, using either eye fixations or computer cursor trajectories as dependent measures. In particular, Spivey, Grosjean and Knoblich (2005) relied on their observation of unimodality in the distribution of cursor trajectories to argue in favor of a single cognitive process consistent with a continuous model of lexical activation. The present study addresses two questions: (1) whether the logic of inferring the number of cognitive processes from distributional analyses can be extended to a different dependent variable – reaction times, and (2) how robust the distribution of cursor trajectories is to changes in cursor speed (mouse gain). In Experiment 1, eye movements and reaction times were recorded in a visual-world paradigm and reaction times were modeled using ex-Gaussian curve-fitting. Participants responded slower to trials with a phonological competitor presented alongside the target than to trials with a control image presented alongside the target. Crucially, this difference was manifested as a shifting of the distribution rather than as a skewing of the distribution and lends additional support for a continuous model of lexical activation. Experiment 2 measured eye and mouse movements concurrently in a similar visual-world task to investigate the relationship between these two dependent measures at the level of the individual trial. In addition, Experiment 2 manipulated the speed of the cursor (mouse gain) between subjects. The low mouse gain served to reduce the effect of phonological competition. Moreover, the shape of the distribution of cursor trajectories across phonological competitor and control conditions was indistinct with low mouse gain, while the shape of the distributions across the two conditions differed with high mouse gain. This effect of mouse gain shows that the distribution of cursor trajectories is not robust to changes in mouse gain. Moreover, it raises questions about the strength of the linking hypothesis necessary to interpret the distribution of cursor trajectories.

本文主要探討中文聲調在口語字彙觸接過程中所扮演的角色。實驗一藉由眼動實驗中的Visual World Paradigm作業，觀察中文聲調影響口語字彙辨識的時序歷程。受試者在聽到指導語和目標字之後，用滑鼠在螢幕上點選聽到的目標字，例如，螢幕上出現的字包含一個目標字：「摸」、一個競爭字(與目標字只有聲調相同：「挖」或是與目標字只有音段相同：「抹」)，以及兩個聲調與音段和目標字完全不同的無關字：「怒」、「菊」。為了觀察目標字、競爭字及無關字在口語字彙處理時的競爭，我們會計算各個字彙的凝視比例。實驗一中由於聲調與目標字相同的競爭字與目標字的第一個音段就開始產生差異，因此未觀察到聲調早期介入的影響。實驗二透過與實驗一相同的實驗程序及方法，操弄目標字和競爭字中聲調和前兩個音段(Cohort)的異同以探測更早期的聲調影響。螢幕呈現包含一個目標字「湯」、一個競爭字(前兩個音段和聲調皆與目標字相同：「胎」，或是只有前兩個音段相同但聲調與目標字不同：「泰」)，以及兩個聲調與音段和目標字完全不同的無關字「剖」、「痕」。結果顯示，聲調在語音訊息前兩個音段時就會產生影響，也就是聲調的影響在語音結束前即有作用。再者，本文發現聲調無法單獨且獨立地對於語音辨識產生影響，此看法與聲調表徵需以“toneme” node獨立地存在於the modified TRACE model的看法不盡相同 (Malins & Joanisse, 2010; Ye & Connine, 1999; Zhao, Guo, Zhou, & Shu, 2011)。 / The present study aims to examine the role of tonal information during Mandarin Chinese spoken character recognition. Two eye-tracking experiments were conducted with the visual world paradigm, which participants heard a Chinese monosyllabic character and used a mouse to click on the corresponding character in a visual array of 4 characters on the screen. Experiment 1 manipulated the relationship between the spoken target characters and written characters on the screen, including a target (e.g., /mɔ1/‘touch’), a tonal competitor (the tone was the same as target except segment: e.g., /wa1/‘dig’) or a segmental competitor (the segmental structure was the same with the target except tone: e.g., /mɔ3/ ‘wipe’), and two unrelated distractors (the segments and tone were different from target: e.g., /nu4/ ‘anger’, and /tɕy2/ ‘chrysanthemum’). The fixation proportions on target, competitors and the unrelated distractors were computed during the unfolding of the auditory target stimuli. The results showed tonal difference was detected before the end of auditory stream. However, no early involvement of tonal information was found, which may due to the tonal competitor and target shared no segment from the first phoneme. In order to examine the earlier tonal processing, Experiment 2 manipulated two types of cohort competitors sharing the initial two segments with the target (e.g., /tʰɑŋ1/ “soup”), a cohort-tone competitor, e.g., /tʰaj1/ “fetus” (both tone and initial two segments are the same with target) and a cohort-only competitor e.g., /tʰaj4/ “peaceful” (initial two segments is the same with the target but with different tone). Result showed that tone affected spoken character recognition while processing the two initial segments. In addition, tone could not affect spoken character processing independently, which might be inconsistent with the assumption that tone is a separate level of representation, called “toneme” node, in the modified TRACE model (Malins & Joanisse, 2010; Ye & Connine, 1999; Zhao et al., 2011).

Chez les enfants souffrant de troubles spécifiques du langage oral (TSLO), des déficits demémoire à court terme verbale et d’acquisition lexicale sont fréquemment décrits. Ces capacités sont évaluées par des tâches de répétition et plus généralement de production verbale, mettant en jeu demultiples composants susceptibles d’être altérés dans les TSLO. Dans ce contexte, l’objectif généralde cette thèse était l’examen de mécanismes de mémorisation verbale à l’aide du paradigme dumonde visuel. L’oculométrie consiste à analyser les mouvements oculaires des participants sur desimages, à la suite de stimulations verbales. La reconnaissance et la mémorisation à court terme demots parlés familiers ont été examinées à travers quatre études, auxquelles les participants avec etsans TSLO âgés de 5 et 6 ans, ont obtenu des résultats comparables. Seules des différences enfonction de l’âge ont été révélées, reflétant les changements développementaux du traitement lexical.Dans un second temps, le paradigme d’appariement rapide pour l’acquisition de nouveaux mots(« fast mapping »), nécessitant peu de présentations, a été proposé à trois groupes d’enfants. Leurscapacités d’acquisition lexicale ont à nouveau été évaluées via le paradigme du monde visuel à unetâche de reconnaissance de mots parlés. Les groupes TSLO présentant des déficits en répétitionverbale, ont néanmoins reconnu et apparié les nouvelles étiquettes lexicales aux images référentes,comme leurs pairs au développement typique. Les études menées ont permis de mettre en évidenceque le maintien à court terme de l'information verbale ainsi que l’encodage de nouvelles informationssont efficients chez les enfants avec TSLO, lorsque la production verbale n’est pas sollicitée. / In children suffering from specific language impairment (SLI), verbal short term memory disorders and lexical learning inability are frequently described. These skills are assessed through repetition tasks and more generally, with verbal output modality which variety of components can be impaired in SLI. In this context, the general aim of this research work was the examination of verbal memorization processes through visual world paradigm. This eye-tracking tasks consist in analyzing eye movements of participants on pictures, consecutively to verbal stimulations. Recognition and short term retention of familiar spoken words were investigated through four studies, in which participants aged 5 and 6, with and without SLI, shown similar patterns of result. Only age differences were revealed, suggesting developmental changing of lexical processing. In a second phase, the fast mapping word learning paradigm requiring few occurences of new words, was proposed to three groups of children. Their lexical learning abilities were again assessed within the visual world paradigm and a spoken word recognition task. SLI groups with poor performances in verbal repetition tasks, have nevertheless associated the new lexical label with referent picture, as their peers with typical development. These studies suggest that verbal information retention in short term memory and encoding of new information are efficient in SLI children, when the tasks do not require any verbal output.

No description available.

語言系統中，形音之間的映照關係為一持續受到探討的議題。本篇論文旨在利用眼動閱讀實驗以及口語理解─視覺典範深入探究閱聽中文形聲字時，音形映照之存在性及其時序歷程。 本文進行兩個眼動實驗。實驗一旨在探究視覺的閱讀歷程中，中文的音形映照於詞彙層次和次詞彙層次之歷程及影響。結果顯示，同音字密度效果未達顯著，但音形對應一致性效果於晚期眼動指標中顯著呈現。顯示出音形映照於視覺詞彙辨識的晚期產生影響，並證實語音表徵和字形表徵於視覺處理中是互為雙向影響之觀點，實驗一結果並支持音至形的反向連結映照於視覺模式中具有其重要性。實驗二主要使用口語理解─視覺典範，進一步探究並檢驗音形映照於口語詞彙辨識過程中之時序及歷程。結果顯示，音形對應一致性效果於早期眼動指標中顯著呈現，隨後同音字密度效果於晚期眼動指標中顯著呈現。此研究結果支持雙向交互激發模型(BIAM)之假設，顯示詞彙辨識機制中，存在字音和字形表徵之間的動態交互連結；並證實字形表徵於口語詞彙辨識過程中時序性的影響。 整體而論，本研究整合形音映照之概念所衍生出的形音一致性效果及同音字密度效果，就視、聽兩種模式之眼動實驗，檢視音形映照歷程，並進一步提供音形映照的實驗證據，探究其於中文語言系統的文字辨識歷程中之影響力與重要性。 / The present study aims to examine the states of phonological to orthographic (P-O) mappings when reading and listening Chinese phonograms. Two eye tracking experiments regarding to visual and auditory modalities were conducted to explore the intrinsic nature and the temporal dynamics of P-O mappings in Chinese word recognition. Experiment 1 manipulated homophone density and P-O consistency to investigate the involvement of P-O mappings at lexical and sublexical levels during the reading process. The result of Experiment 1 revealed that the P-O consistency effect was evident in second-pass eye movement indices of total viewing time (TVT) and rereading rate (RRR), demonstrating a late occurrence of P-O consistency effect at a the verification stage in reading. The occurrence of P-O consistency is in accordance with the view that the phonological information and orthographic representation are activated in a bidirectional flow, which implies that the mappings from phonology to orthography were guaranteed during the reading process. Experiment 2 utilizes visual world paradigm to explore the P-O mappings in auditory modality and further inspect the temporal dynamic in listening Chinese spoken characters. The result demonstrated that the P-O consistency effect emerged approximately 300 ms earlier than the homophone density effect, reflecting the early P-O consistency effect and the relatively late HD effect during the temporal stage of spoken word recognition. The result also supports the bidirectional activation of orthographic and phonological codes during word recognition, demonstrating the notion that the dynamic influences of orthography representation on spoken word recognition are ubiquitous at both sub-lexical and lexical levels.

過去二十多年來，心理語言學研究關注詞彙歧義解困 (lexical ambiguity resolution)歷程發生時，語義脈絡與多義詞的語義頻率之間的交互作用。許多研究發現，當語境支持非均勢同形異義詞 (unbalanced homograph) 的次要語義時，同形異義詞的凝視時間長於與其有相同字形頻率的單義詞 (unambiguous control)，此為次要語義偏向效應 (subordinate bias effect)。根據再排序觸接模型 (reordered-access model)，次要語義偏向效應來自於主要語義與次要語義的競爭；相對地，選擇觸接模型 (selective access model)則認為只有與語境相關的語義被激發，因此，次要語義偏向效應是因為提取到一個使用頻率較低的語義。本論文進行兩個眼動實驗。實驗一檢視中文多義詞的次要語義偏向效應以區辨兩種詞彙歧義解困模型分別提出的解釋。本實驗的材料使用了低頻同形異義詞、低頻單義詞、以及高頻單義詞。結果顯示，當使用的單義詞與多義詞字形頻率相同時，在目標詞及後目標詞上(目標詞後一個詞)皆發生了次要語義偏向效應。實驗二利用口語理解─視覺典範中透過受試者理解語音訊息時同步記錄眼動的作業方式來探究次要語義偏向效應是否來自於主要語義的激發。當口語句子中的目標詞被唸出後，會計算出隨著時間增加眼睛落在四個雙字詞的凝視比例。結果發現次要語義因為語境的選擇在聽到目標詞後大約500毫秒時就可被激發，主要語義則在一聽完多義詞後被激發。因此，多義詞的兩個語義在聽到目標詞後大約900至1300毫秒時(相當於在後目標詞時)發生競爭。整體而言，本研究顯示即使語境支持多義詞的次要語義，主要語義依然會被激發。因此，次要語義偏向效應是由兩個語義競爭後所造成的結果，符合再排序觸接模型的解釋。 / Research in psycholinguistics throughout the last two decades has focused on the interaction between linguistic context and meaning dominance during lexical ambiguity resolution. Many studies demonstrated the subordinate bias effect when the preceding context biased for the subordinate meaning (i.e. infrequent meaning) of an unbalanced homograph. According to the reordered access model, the SBE is due to competition between the dominant and subordinate meanings. On the contrary, the selective access model assumes only the context-relevant meaning is activated and the SBE is a result of access to a low frequent meaning. Two eye tracking experiments of sentence reading and sentence listening were conducted. Experiment 1 examined the SBE of Chinese homographs to differentiate the two accounts. We utilized low frequency homographs along with their matched low and high-frequency unambiguous words. The results showed the SBE emerging in fixation durations of the target region and post-target region (i.e. next two words of the target), when unambiguous controls were matched to the word-form frequency of ambiguous words. Experiment 2 used visual world paradigm to explore temporal dynamics of dominant meaning activation responsible for the SBE in an instructional eyetracking-during-listening task. Fixation probabilities on four disyllabic printed words were analyzed during a time period after a target word was uttered in a spoken sentence. The results supported the reordered access model. The subordinate meaning was activated by contextual information at about 500 ms after the onset of acoustic homograph at the time when context penetrated to make its favored meaning available. Soon after the offset of homograph, the dominant meaning became active. Both meanings associated with the homograph were activated during the time windows of 901 ms to 1300 ms, which approximately corresponding to the acoustic onset of post target. In sum, our studies demonstrate that the dominant meaning is activated even when the contextual information biases to the subordinate meaning of a homograph. The subordinate bias effect is the result of competition from two meanings, conforming to the reordered access model.