Mémoire numérisé par la Direction des bibliothèques de l'Université de Montréal.

Perceptual multistability is a phenomenon that is mostly studied in all modalities separately. The phenomenon reveals fundamental principles of the perceptual system in the formation of an emerging cognitive representation in the consciousness. The momentary perceptual organizations evoked during the stimulation with ambiguous stimuli switches between several perceptual organizations or percepts: The auditory streaming stimulus in audition and the moving plaids stimulus in vision, elicit different at least two percepts that dominate awareness exclusively for a random phase or dominance duration before an inevitable switch to another percept occurs. The similarity in the perceptual experience has led to propose a global mechanism contributing to the perceptual multistability phenomena crossmodally. Contrary, the difference in the perceptual experience has led to propose a distributed mechanism that is modality-specific. The development of a hybrid model has synergized both approaches. We accumulate empirical evidence for the contribution of a global mechanism, albeit distributed mechanisms play an indispensable role in this cross-modal interplay. The overt report of the perceptual experience in our experiments is accompanied by the recording of objective, cognitive markers of the consciousness: Reflexive movements of the eyes, namely the dilation of the pupil and the optokinetic nystagmus, correlate with the unobservable perceptual switches and perceptual states respectively and have their neuronal rooting in the brainstem. We complement earlier findings on the sensitivity of the pupil to visual multistability: It was shown in two independent experiments that the pupil dilates at the time of reported perceptual switches in auditory multistability. A control condition on confounding effects from the reporting process confines the results. Endogenous, evoked internally by the unchanged stimulus ambiguity, and exogenous, evoked externally by the changes in the physical properties of the stimulus, perceptual switches could be discriminated based on the maximal amplitude of the dilation. The effect of exogenous perceptual has on the pupil were captured in a report and no-report task to detect confounding perceptual effects. In two additional studies, the moment-by-moment coupling and coupling properties of percepts between concurrent multistable processes in audition, evoked by auditory streaming, and in vision, evoked by moving plaids, were found crossmodally. In the last study, the externally induced percept in the visual multistable process was not relayed to the simultaneous auditory multistable process: Still, the observed general coupling is fragile but existent. The requirement for the investigation of a moment-by-moment coupling of the multistable perceptual processes was the application of a no-report paradigm in vision: The visual stimulus evokes an optokinetic nystagmus that has machine learnable different properties when following either of the two percepts. In combination with the manually reported auditory percept, attentional bottlenecks due to a parallel report were circumvented. The two main findings, the dilation of the pupil along reported auditory perceptual switches and the crossmodal coupling of percepts in bimodal audiovisual multistability, speak in favor of a partly global mechanism being involved in control of perceptual multistability; the global mechanism is incarcerated by the, partly independent, distributed competition of percepts on modality level. Potentially, supramodal attention-related modulations consolidate the outcome of locally distributed perceptual competition in all modalities.:COVER 1 BIBLIOGRAPHISCHE BESCHREIBUNG 2 ACKNOWLEDGEMENTS 3 CONTENTS 4 CHAPTER 1: Introduction 6 C1.1: Stability and uncertainty in perception 6 C1.2: Auditory, visual and audio-visual multistability 14 C1.3: Capturing the subjective perceptual experience 25 C1.4: Limitations of preceding studies, objectives, and outline of the Thesis 33 CHAPTER 2: Study 1 “Pupillometry in auditory multistability” 36 C2.1.1 Experiment 1: Introduction 36 C2.1.2 Experiment 1: Material and Methods 38 C2.1.3 Experiment 1: Data analysis 44 C2.1.4 Experiment 1: Results 48 C2.1.5 Experiment 1: Discussion 52 C2.2.1 Experiment 2: Introduction 54 C2.2.2 Experiment 2: Material and Methods 54 C2.2.3 Experiment 2: Data analysis 56 C2.2.4 Experiment 2: Results 57 C2.3 Experiment 1 & 2: Discussion 61 C2.4 Supplement Study 1 65 CHAPTER 3: Study 2 “Multimodal moment-by-moment coupling in perceptual bistability” 71 C3.1.1 Experiment 1: Introduction 71 C3.1.2 Experiment 1: Results 74 C3.1.3 Experiment 1: Discussion 80 C3.1.4 Experiment 1: Material and Methods 84 C3.1.5 Experiment 1: Data analysis 87 C3.2 Supplement Study 2 92 CHAPTER 4: Study 3 “Boundaries of bimodal coupling in perceptual bistability” 93 C4.1.1 Experiment 1: Introduction 93 C4.1.2 Experiment 1: Material and Methods 98 C4.1.3 Experiment 1: Data analysis 102 C4.1.4 Experiment 1: Results 108 C4.1.5 Experiment 1: Discussion 114 C4.2.1 Experiment 2: Introduction 116 C4.2.2 Experiment 2: Material and Methods 119 C4.2.3 Experiment 2: Data analysis 125 C4.2.4 Experiment 2: Results 133 C4.3 Experiment 1 & 2: Discussion 144 C4.4 Supplement Study 3 151 CHAPTER 5: General Discussion 154 C5.1 Significance for models of multistability and implications for the perceptual architecture 162 C5.2 Recommendations for future research 166 C5.3 Conclusion 168 REFERENCES 170 APPENDIX 186 A1: List of Figures 186 A2: List of Tables 188 A3: List of Abbreviations and Symbols 189

Sensory inputs can be ambiguous. A physically constant stimulus that induces several perceptual alternatives is called multistable. Many factors can influence perception. In this thesis I investigate factors that affect visual multistability. All presented studies use a pattern-component rivalry stimulus consisting of two gratings drifting in opposite directions (called the plaid stimulus). This induces an “integrated” perception of a moving plaid (the pattern) or a “segregated” perception of overlaid gratings (the components). One study (chapter 2) investigates parameter dependence of a plaid stimulus on perception, with particular emphasis on the first percept. Specifically, it addresses how the enclosed angle (opening angle) affects the perception at stimulus onset and during prolonged viewing. The effects that are shown persist even if the stimulus is rotated. On a more abstract level it is shown that percepts can influence each other over time (chapter 3) which emphasizes the importance of instructions and report mode. In particular, it relates to the decision which percepts are instructed to be reported at all as well as which percepts can be reported as separate entities and which are pooled into the same response option. A further abstract level (predictability of a stimulus change, chapter 5) shows that transferring effects from one modality to another modality (specifically from audition to vision) requires careful choice of stimulus parameters. In this context, we give considerations to the proposal for a wider usage of sequential stopping rules (SSR, chapter 4), especially in studies where effect sizes are hard to estimate a priori. This thesis contributes to the field of visual multistability by providing novel experimental insights into pattern-component rivalry and by linking these findings to data on sequential dependencies, to the optimization of experimental designs, and to models and results from another sensory modality.:Bibliographische Beschreibung 3 Acknowledgments 4 CONTENTS 5 Collaborations 7 List of Figures 8 List of Tables 8 1. Introduction 9 1.1. Tristability 10 1.2. Two or more interpretations? 11 1.3. Multistability in different modalities 12 1.3.1. Auditory multistability 12 1.3.2. Haptic multistability 13 1.3.3. Olfactory multistability 13 1.4. multistability with several interpretations 13 1.5. Measuring multistability 14 1.5.1. The optokinetic nystagmus 14 1.5.2. Pupillometry 15 1.5.3. Measuring auditory multistability 15 1.5.4. Crossmodal multistability 16 1.6. Factors governing multistability 16 1.6.1. Manipulations that do not involve the stimulus 16 1.6.2. Manipulation of the stimulus 17 1.6.2.1. Factors affecting the plaid stimulus 17 1.6.2.2. Factors affecting the auditory streaming stimulus 18 1.7. Goals of this thesis 18 1.7.1. Overview of the thesis 18 2. Parameter dependence in visual pattern-component rivalry at onset and during prolonged viewing 21 2.1. Introduction 21 2.2. Methods 24 2.2.1. Participants 24 2.2.2. Setup 24 2.2.3. Stimuli 25 2.2.4. Procedure 26 2.2.5. Analysis 27 2.2.6. (Generalized) linear mixed-effects models 30 2.3. Results 30 2.3.1. Experiment 1 30 2.3.1.1. Relative number of integrated percepts 31 2.3.1.2. Generalized linear mixed-effects model 32 2.3.1.3. Dominance durations 33 2.3.1.4. Linear mixed-effects models 33 2.3.1.5. Control: Disambiguated trials 33 2.3.1.6. Time course of percept reports at onset 34 2.3.1.7. Eye movements 35 2.3.2. Experiment 2 36 2.3.2.1. Relative number of percepts 36 2.3.2.2. Generalized linear mixed-effects model 37 2.3.2.3. Dominance durations 38 2.3.2.4. Linear mixed-effects model 38 2.3.2.5. Control: Disambiguated trials 40 2.3.2.6. Time course of percept reports at onset 42 2.3.2.7. Eye movements 44 2.4. Discussion 45 2.5. Appendix 49 2.5.1. Appendix A 49 3. Perceptual history 51 3.1. Markov chains 52 3.1.1. Markov chains of order 1 and 2 52 3.2. Testing for Markov chains 55 3.2.1. The method of Naber and colleagues (2010) 56 3.2.1.1. The method 56 3.2.1.2. Advantages and disadvantages of the method 56 3.2.2. Further methods for testing Markov chains 57 3.3. Summary and discussion 58 4. Sequential stopping rules 60 4.1. The COAST rule 61 4.2. The CLAST rule 61 4.3. The variable criteria sequential stopping rule 61 4.4. Discussion 62 4.5. Using the vcSSR when transferring an effect from audition to vision 64 5. Predictability in visual multistability 66 5.1. Pretests 66 5.2. Predictability effects in visual pattern-component rivalry 69 5.2.1. Introduction 69 5.2.2. Methods 71 5.2.2.1. Participants 71 5.2.2.2. Setup 72 5.2.2.3. Stimuli 73 5.2.2.4. Conditions 73 5.2.2.5. Design and procedure 73 5.2.2.6. Analysis 74 5.2.3. Results 75 5.2.3.1. Valid reports 75 5.2.3.2. Verification of reports by eye movements 76 5.2.3.3. Onset latency 76 5.2.3.4. Dominance durations 78 5.2.3.5. Relative dominance of the segregated percept 78 5.2.4. Discussion 78 6. General discussion 83 6.1. Reporting percepts 83 6.1.1. Providing two versus three response options 83 6.1.2. Stimuli with more than three percepts 84 6.1.3. When to pool percepts together and when not 84 6.1.4. Leaving out percepts 87 6.1.5. Measuring (unreported) percepts 88 6.2. Comparing influencing factors on different levels 88 6.3. The use of the vcSSR 90 6.4. Valid reports 90 6.5. Conclusion 93 References 94