The simultaneity judgment (SJ) and temporal order judgment (TOJ) tasks are the two widely used methods for measuring the window of multisensory integration; however, there are some indications that these two tasks involve different cognitive processes and therefore produce unrelated results. The present study measured observers’ visuotactile window of integration using these two tasks in order to examine whether or not SJs and TOJs produce consistent results for this particular pairing of modalities. Experiment 1 revealed no significant correlations between the SJ and TOJ tasks, indicating that they appear to measure distinct processes in visuotactile integration, and in addition showed that both sensory and decisional factors contribute to this difference. These findings were replicated in Experiment 2, which, along with Experiment 3, also showed that the reliability of the SJ and TOJ tasks may in part be responsible for the lack of agreement between these two tasks. A secondary result concerned the point of subjective simultaneity (PSS), which were tactile-leading across all three experiments. This contradicts some of the previous literature in visuotactile integration. Manipulating the spatial distance between the visual and tactile stimulus (Experiment 2) and the certainty of stimulus location (Experiment 3) did not lead to significant changes of the location of the PSS. / Thesis / Master of Science (MSc) / Perception often involves the use of more than one sensory modality at the same time; for example, touching an object usually produces sensory signals in the visual and tactile modalities. Since the amount of time needed to transmit and process sensory signals is different among the modalities, the brain allows for a certain time difference between signals of various pairs of modalities that it will consider as coming from one event. Two tasks commonly used to measure these allowable time differences are the simultaneity judgment (SJ) and temporal order judgment (TOJ) tasks. Although they are usually used interchangeably, the present data show that the results from these tasks in the visuotactile pairing of modalities are unrelated, and a major contributing reason appears to be that these tasks are not the most reliable.

Attended objects are perceived differently than unattended objects. Spatial attention is consistently associated with an enhancement in spatial resolution. However, the relation between spatial attention and temporal resolution is not as straightforward. Some studies have shown that attention enhances temporal resolution, and others have shown that attention degrades temporal resolution. The motivation of the present work was to investigate the source of this discrepancy. In particular, the research herein examines the hypothesis that attention degrades temporal resolution when the target stimuli are easily integrated (i.e., according to the Gestalt principles of grouping), and that attention enhances temporal resolution when the targets are easy to perceptually segregate. Temporal resolution was assessed by the mean just noticeable difference (the minimum temporal interval in milliseconds required by observers to perform the task at 75% accuracy) in a visual temporal order judgment (TOJ) task. Trials involved the presentation of two targets, at randomly varying stimulus onset asynchronies, and observers reported which one they perceived first. The primary research questions concerned the effect of perceptual grouping on temporal resolution, and the influence of attention on that relation. Grouping processes were manipulated using a variety of Gestalt principles and attention was investigated under conditions of automatically-and voluntarily-driven orienting. Three main findings emerged: temporal resolution is worse for grouped than ungrouped targets; attention modulates the effect of grouping on temporal resolution on a continuum - strong grouping effects produce large impairments on temporal resolution, and weaker grouping effects produce smaller impairments; and automatic and voluntary spatial orienting affect the relation between grouping and temporal resolution differently - automatic orienting augments the relation, while voluntary orienting does not. I conclude that the discrepant findings in the previous research are due to object-based factors pertaining to the target stimuli and propose an object-based theory of temporal perception. / Thesis / Doctor of Philosophy (PhD)

Prior entry refers to the notion that attended stimuli are perceived sooner than unattended stimuli due to a speed up in sensory processing. The century long debate regarding the prior entry phenomenon’s existence has always been grounded in the degree to which the methods applied to the problem allow for cognitive response bias. This thesis continues that trend by applying the synchrony comparison judgment method to the problem of audiovisual prior entry. Experiment 1 put this method into context with two other common psychophysical methods – the temporal order judgment and the synchrony judgment – that have been applied to the prior entry problem. The results of this experiment indicated that the temporal order judgment method was out of step with the other two methods in terms of the parameter estimates typically used to evaluate prior entry. Experiment 2 evaluated and confirmed that a specific response bias helps explain the difference in parameter estimates between the temporal order judgment method and the other two. Experiment 3 evaluated the precision of the synchrony comparison judgment method. The results indicated that the method was precise enough to detect potentially small prior entry effect sizes, and that it afforded the ability to detect those participants with points of subjective synchrony that deviate substantially from zero. Finally, Experiment 4 applied the synchrony comparison judgment method to a prior entry scenario. A prior entry effect was not realized. Overall, this thesis highlights the drawbacks of all previous methods used to evaluate audiovisual perception, including prior entry, and validates the use of the synchrony comparison judgment. Further, due to the resistance of this method to response bias, this result now stands as the most convincing evidence yet against the prior entry phenomenon.

Numbers convey different meanings when used in different contexts (Wiese, 2003). In a cardinal context, a number will tell us how many entities are in a set and convey quantity meaning. In an ordinal context, a number will refer to the relative position (or rank) of one element within a sequence; non-numerical ordered series (e.g. the letters of the alphabet) can also be used to provide meaningful order information. Because quantity and order are linked up with each other in the cognitive number domain (the larger the quantity a number refers to, the later it is located in the conventional number sequence), the question of whether they rely on some common or distinct underlying mechanism(s) is theoretically relevant and was addressed in the present thesis. Experimental studies showed evidence of both similarities (similar distance and SNARC effects, recruitment of parietal and frontal regions, and conjoint impairment or preservation after brain damage) and dissociations (different developmental course, dissociation after cerebral lesion, and specific behavioural markers) between quantity and order neuro-functional processes. The aim of the present thesis was to clarify the relationship between numerical quantity and order processing and to test the hypothesis that they rely on (at least partially) dissociated mechanisms. We tested this hypothesis in a single case study, an electrophysiological study and in two behavioural experiments. In the neuropsychological study, we reported the case of patient CO, who showed Gerstmann syndrome after bilateral parietal damage and became unable to process sequence order relations (e.g. he couldn't recite the number sequence backwards, nor decide whether a number, letter, day or month comes before or after a given target in the corresponding sequence, and he was unable to verify the order of items in a pair). Nonetheless, the patient had largely preserved quantity processing abilities (he could compare numbers and dot patterns to find the smaller or larger, and showed a standard distance effect, he could produce a number smaller or larger than a given target, and match dot patterns with Arabic numerals). Overall, CO's pattern of performance was interpreted as reflecting the involvement of different mechanisms when processing quantity or sequence order relations. Our electrophysiological study corroborated this finding since different spatio-temporal patterns of the distance effect were observed when subjects had to process numbers in a quantity comparison task or in an order judgment task. Quantity processing elicited an early distance effect over the P2p component on left parietal sites, whereas the distance effect was slightly delayed and bilaterally distributed in the numerical order judgment task; and this latter task additionally recruited prefrontal regions on a later (P3-counterpart) component. Finally, our behavioural study further emphasized the involvement of different mechanisms underlying the processing of quantity and numerical order and provided some evidence about the nature of these specific mechanisms. In the number comparison (quantity) task, the standard distance effect was proposed to reflect the involvement of a magnitude comparison mechanism; whereas the reverse distance effect observed in the numerical order verification task was taken as evidence for the recruitment of a serial search (recitation) process. Besides, the pair-order effect was also found to specifically affect order but not quantity judgments. Taken together, the data collected in the present thesis lend further support to the hypothesis that quantity and numerical order rely on distinct processing mechanisms that can be damaged selectively after cerebral lesions, that recruit similar brain areas but with a different spatio-temporal course and that show specific behavioural markers.

Numbers convey different meanings when used in different contexts (Wiese, 2003). In a cardinal context, a number will tell us how many entities are in a set and convey quantity meaning. In an ordinal context, a number will refer to the relative position (or rank) of one element within a sequence; non-numerical ordered series (e.g. the letters of the alphabet) can also be used to provide meaningful order information. Because quantity and order are linked up with each other in the cognitive number domain (the larger the quantity a number refers to, the later it is located in the conventional number sequence), the question of whether they rely on some common or distinct underlying mechanism(s) is theoretically relevant and was addressed in the present thesis. Experimental studies showed evidence of both similarities (similar distance and SNARC effects, recruitment of parietal and frontal regions, and conjoint impairment or preservation after brain damage) and dissociations (different developmental course, dissociation after cerebral lesion, and specific behavioural markers) between quantity and order neuro-functional processes. The aim of the present thesis was to clarify the relationship between numerical quantity and order processing and to test the hypothesis that they rely on (at least partially) dissociated mechanisms. We tested this hypothesis in a single case study, an electrophysiological study and in two behavioural experiments. In the neuropsychological study, we reported the case of patient CO, who showed Gerstmann syndrome after bilateral parietal damage and became unable to process sequence order relations (e.g. he couldn't recite the number sequence backwards, nor decide whether a number, letter, day or month comes before or after a given target in the corresponding sequence, and he was unable to verify the order of items in a pair). Nonetheless, the patient had largely preserved quantity processing abilities (he could compare numbers and dot patterns to find the smaller or larger, and showed a standard distance effect, he could produce a number smaller or larger than a given target, and match dot patterns with Arabic numerals). Overall, CO's pattern of performance was interpreted as reflecting the involvement of different mechanisms when processing quantity or sequence order relations. Our electrophysiological study corroborated this finding since different spatio-temporal patterns of the distance effect were observed when subjects had to process numbers in a quantity comparison task or in an order judgment task. Quantity processing elicited an early distance effect over the P2p component on left parietal sites, whereas the distance effect was slightly delayed and bilaterally distributed in the numerical order judgment task; and this latter task additionally recruited prefrontal regions on a later (P3-counterpart) component. Finally, our behavioural study further emphasized the involvement of different mechanisms underlying the processing of quantity and numerical order and provided some evidence about the nature of these specific mechanisms. In the number comparison (quantity) task, the standard distance effect was proposed to reflect the involvement of a magnitude comparison mechanism; whereas the reverse distance effect observed in the numerical order verification task was taken as evidence for the recruitment of a serial search (recitation) process. Besides, the pair-order effect was also found to specifically affect order but not quantity judgments. Taken together, the data collected in the present thesis lend further support to the hypothesis that quantity and numerical order rely on distinct processing mechanisms that can be damaged selectively after cerebral lesions, that recruit similar brain areas but with a different spatio-temporal course and that show specific behavioural markers.

Prior entry refers to the notion that attended stimuli are perceived sooner than unattended stimuli due to a speed up in sensory processing. The century long debate regarding the prior entry phenomenon’s existence has always been grounded in the degree to which the methods applied to the problem allow for cognitive response bias. This thesis continues that trend by applying the synchrony comparison judgment method to the problem of audiovisual prior entry. Experiment 1 put this method into context with two other common psychophysical methods – the temporal order judgment and the synchrony judgment – that have been applied to the prior entry problem. The results of this experiment indicated that the temporal order judgment method was out of step with the other two methods in terms of the parameter estimates typically used to evaluate prior entry. Experiment 2 evaluated and confirmed that a specific response bias helps explain the difference in parameter estimates between the temporal order judgment method and the other two. Experiment 3 evaluated the precision of the synchrony comparison judgment method. The results indicated that the method was precise enough to detect potentially small prior entry effect sizes, and that it afforded the ability to detect those participants with points of subjective synchrony that deviate substantially from zero. Finally, Experiment 4 applied the synchrony comparison judgment method to a prior entry scenario. A prior entry effect was not realized. Overall, this thesis highlights the drawbacks of all previous methods used to evaluate audiovisual perception, including prior entry, and validates the use of the synchrony comparison judgment. Further, due to the resistance of this method to response bias, this result now stands as the most convincing evidence yet against the prior entry phenomenon.

Encoding time is one of the most important features of the mammalian brain. The visual system, comprising almost half of the brain is of no exception. Time processing enables us to make goal-directed behavior in the optimum “time window” and launch a ballistic eye movement, reach/grasp an object or direct our processing resources (attention) from one point of interest to another. In addition, encoding time is critical for higher cognitive functions, enabling us to make causal inferences. The limitations of temporal individuation in the visual stream seem to vary across the visual field: the resolution gradually drops as objects become farther away from the center of gaze, where little differences were found in terms of resolution for objects in the upper versus lower visual field. This resolution of temporal individuation is vastly different from the resolution ascribed to spatial individuation. If individuation is mediated through attention, as some researchers have proposed, the general term ”attention” seems to possess different properties, at least regarding temporal and spatial processing. Next we looked at another aspect of encoding time: Temporal Order Judgments (TOJ), where animals had to judge the relative timing onset of two visual events. After training two monkeys on the task, we recorded from neurons in the lateral intraparietal area (LIP), while the animals reported the perceived order of two visual stimuli. We found that LIP neurons show differential activity based on the animal’s perceptual choice: when the animal reports the stimulus inside the receptive field of the neuron as first, the cells show an increased level of activity compared to when the animal reports he same stimulus as second. This differential activity was most reliable in the tonic period of the response \((\sim100 ms\) after stimulus onset). However, no difference in visual response latencies was observed between the different perceptual choices. The parietal cortex has previously been implicated in temporal processing based on patient studies as well as neuroimaging investigations. Physiological studies have also suggested the involvement of parietal area in encoding elapsed time. However, our study is the first to demonstrate parietal neurons encoding relative timing. / Psychology

Prior entry refers to the notion that attended stimuli are perceived sooner than unattended stimuli due to a speed up in sensory processing. The century long debate regarding the prior entry phenomenon’s existence has always been grounded in the degree to which the methods applied to the problem allow for cognitive response bias. This thesis continues that trend by applying the synchrony comparison judgment method to the problem of audiovisual prior entry. Experiment 1 put this method into context with two other common psychophysical methods – the temporal order judgment and the synchrony judgment – that have been applied to the prior entry problem. The results of this experiment indicated that the temporal order judgment method was out of step with the other two methods in terms of the parameter estimates typically used to evaluate prior entry. Experiment 2 evaluated and confirmed that a specific response bias helps explain the difference in parameter estimates between the temporal order judgment method and the other two. Experiment 3 evaluated the precision of the synchrony comparison judgment method. The results indicated that the method was precise enough to detect potentially small prior entry effect sizes, and that it afforded the ability to detect those participants with points of subjective synchrony that deviate substantially from zero. Finally, Experiment 4 applied the synchrony comparison judgment method to a prior entry scenario. A prior entry effect was not realized. Overall, this thesis highlights the drawbacks of all previous methods used to evaluate audiovisual perception, including prior entry, and validates the use of the synchrony comparison judgment. Further, due to the resistance of this method to response bias, this result now stands as the most convincing evidence yet against the prior entry phenomenon.

After exposure to a short, constant delay between voluntary movement and sensory stimuli, temporal recalibration (TR) arises to realign asynchronous stimuli. The objective of this study was to determine if awareness of the temporal lag between a motor response (i.e., a keypress) and a sensory event (i.e., a visual flash) is necessary for TR to occur. We further investigated whether manipulating the motor and judgment tasks required modifies the influence of awareness on TR due to the cognitive processes engaged. Participants (n = 22) were randomly divided between two groups (Group 1: Aware and Group 2: Unaware). The Aware group was told of the temporal lag between their keypress and visual flash at the beginning of the experiment, whereas the Unaware group was not. All participants completed 8 blocks of trials, in which the motor tasks (e.g., a single or repetitive tap), judgment tasks (e.g., judging the order of the keypress in relation to the visual flash or judging whether the two stimuli were simultaneous or not), and temporal lag between keypress and visual flash (e.g., a 0 ms or 100 ms lag) varied. TR was determined by comparing judgments between corresponding blocks of trials in which the temporal lag was 0 ms to 100 ms. Results revealed that both the Aware and Unaware groups of participants demonstrated TR across both motor and judgment tasks, and that the magnitude of TR did not vary across Aware and Unaware participants or tasks. Thus, results of the present study revealed that awareness of a temporal lag does not influence the magnitude of motor-sensory TR achieved.

The experiments reported within this thesis use psychophysical techniques to examine the factors which determine perceived multisensory timing in humans. Chapters 1 and 2 describe anatomical and psychophysical features of temporal processing, respectively, whilst Chapter 3 introduces the reader to psychophysical methods. Chapter 4 examines the relationship between two measures of sensory latency, reaction time (RT) and crossmodal temporal order judgment (TOJ). Despite task and attentional manipulations the two measures do not correlate, suggesting that they measure some fundamentally different aspect(s) of temporal perception. Chapter 5 examines the effects of adaptation to asynchronous stimulus pairs on perceived audiovisual (AV), audiotactile (AT) and visuotactile (VT) temporal order. Significant temporal shifts are recorded in all three conditions. Evidence is also presented showing that crossmodal TOJs are intransitive. Chapter 6 shows that concurrent adaptation to two sets of asynchronous AV stimulus pairs causes perceived AV temporal order to recalibrate at two locations simultaneously, and that AV asynchrony adaptation effects are significantly affected by observers' attention during adaptation. Finally, Chapter 7 shows that when observers are accustomed to a physical delay between motor actions and sensory events, an event presented at a reduced delay appears to precede the causative motor action. The data are well-described by a simple model based on a strong prior assumption of physical synchrony between motor actions and their sensory consequences.