This thesis explores the use of multimodality to communicate retrieved results of Internet search engines. The investigation aimed to investigate suitable multimodal metaphors which would increase the level of usability of Internet search engine interfaces and enhance users` experience in the search activity. The study consisted of three experiments based on questionnaires and Internet search activities with a set of text-based and multimodal interfaces. These interfaces were implemented in two browsing platforms, named AVBRO and AVBRO II. In the first experiment, the efficiency of specific multimodal metaphors to communicate additional information of retrieved results was investigated. The experiment also sought to obtain users` views of these metaphors with a questionnaire. An experimental multimodal interface of the AVBRO platform, which communicated additional information with a combination of three 2D graphs and musical stimuli, was used as a basis for the experiment, together with the Google search engine. The results obtained led to the planning of a second experiment. The aim of this experiment was to obtain and compare the level of usability of four different experimental multimodal interfaces and one traditional text-based interface, all implemented in the AVBRO II platform. Effectiveness, efficiency and users` satisfaction were used as criteria to evaluate the usability of these interfaces. In the third and final experiment the usability analysis of a traditional text-based interface and the two most suitable experimental multimodal interfaces of the AVBRO II platform was further investigated. Learnability, errors rate, efficiency, memorability and users` satisfaction were used as criteria to evaluate the usability of these interfaces. The analysis of the results obtained from these experiments provided the basis for a set of design guidelines for the development of usable interfaces based on a multimodal approach.

Föreliggande arbete inriktar sig på lärares sätt att kommunicera i undervisning av ensemble där de bemöter flera elever samtidigt. Med hjälp av videoobservationer av fyra lärare som bedriver undervisning av mindre ensembler på gymnasiet har jag analyserat fram vilka kommunikationssätt som används samt hur de används. Resultatet visar att lärarna använder sig av så väl verbal som icke-verbal kommunikation i sin undervisning. Via flertalet multimodala och semiotiska resurser kommunicerar musiklärarna med sina elever under lektionens gång. De kommunikationsspråk som förekommer är ett socialt, humoristiskt, auktoritärt, gestaltande, kunnigt och målande språk. I diskussionen tar jag bland annat upp hur lärarna i undersökningen valt att designa sin undervisning utifrån situationerna om spel och i spel, samt vad detta ger för förutsättningar för dem själva och deras elever. / The study focuses on how teachers communicate when teaching music ensembles in which they meet several students simultaneously. With the help of recorded video lessons of four teachers while teaching small ensembles at the high school level, I have analysed the means of communication and how these are applied. The results show that teachers use verbal as well as nonverbal communication in their teaching. They communicate with their students during the lesson via several multimodal semiotic resources. The language of communication that results is a social, humorous, authoritative, creative, knowledgeable and vivid language. In the chapter on discussion, I present among other things, how the teachers in the study chose to design their education based on the situations regarding performance and through performance, and the conditions thus created for themselves and their students.

Entrés dans l’Union européenne en 2004, la République tchèque et la République slovaque restent encore aujourd’hui à cause ou plutôt grâce à leur présence relativement discrète sur la scène internationale mal connus de leurs voisins les plus proches. Cela est aussi dû au déficit d’information lié d’une part à leur jeune histoire et, d’autre part, à la longue période d’isolement suite à la Guerre froide qui avait laissé apparaître et s’enraciner de part et d’autre du « Mur » des images fausses sur la culture et l’identité des peuples.Cette thèse de doctorat est consacrée à la Première République tchécoslovaque (1918-1938), une époque où les peuples tchèque et slovaque expérimentaient ensemble leurs premiers pas sur la scène internationale en tant qu’État indépendant. Elle se place dans la perspective de l’histoire économique contemporaine et s’attache à analyser plus spécialement la mise en place de réseaux de transport qui constituaient l’armature de la nouvelle économie. Leur rôle était éminent dans la mesure où la survie économique de la nouvelle entité politique, bâtie sur les décombres de l’empire austro-hongrois, en dépendait. Aussi, le pays s’était-il lancé dans de vastes programmes d’investissement dans les infrastructures de transport tout en mettant en place des compagnies publiques ou semi-publiques au service de la république. La période étudiée est relativement courte et homogène, avec une vie politique et une activité économique intenses. Ce fut une période de transition. De ce fait, elle présente un intérêt particulier pour l’historien qui examine les spécificités de la politique des transports et les différents concepts et solutions expérimentés (le monopole de transport public, la gestion « commerciale » des entreprises publiques…), adoptés ou rejetés.La thèse tente d’apporter au public francophone quelques éléments de compréhension de cette partie de l’Europe difficile à cerner par l’historiographie occidentale du fait de ses contours géopolitiques qui avaient connu plusieurs modifications majeures au cours du 20e siècle et de la barrière linguistique. Son auteur étant polyglotte, il était possible de mettre à contribution des sources originales et bibliographiques en langues tchèque, slovaque et allemande tout en assurant une rédaction en français conforme aux standards attendus par une thèse de doctorat soutenue en France. / Even though they have been in the European Union since 2004, The Czech Republic and Slovakia remain relatively unknown, partly because of their limited presence on the international stage. Other factors are ignorance of their recent history and a long period of isolation during the Cold War, which allowed to appear and take root false ideas about the culture and identity of the people on both sides of the “Wall”.This doctorate thesis is dedicated to the First Czechoslovak republic (1918-1938), a time when the Czech and Slovak people made their first steps as an independent State on the international scene. It is paced under the perspective of contemporary economic history and analysis specifically the transport networks which formed the backbone of the new economy. Their role was essential for the economic survival of the new state, built on the ruins of the Austro-Hungarian Empire. In fact, the state launched a vast program of investment in transport infrastructures and, in parallel, founded a series of public or semi-public transport companies at the service of the republic. The period studied is relatively short and homogeneous, with intense political and economic activity. It was a period of transition. For these reasons it is of particular interest to the historian examining the specificities of public transport policy and the different concepts and solutions (a public transport monopoly, the commercial management of public companies…) adopted or rejectedThe thesis tries to bring to the French speaking world some tools for the comprehension of this part of Europe, difficult to pin down by Western historians because its geopolitical boundaries have seen several major upheavals during the 20th century, and because of the linguistic barrier. Its author is polyglot and has the ability to use original and library sources in Czech, Slovak and German, while writing and editing in French to the standards expected of a French doctorate.

A segmentação automática de vídeo em cenas é um problema atual e relevante dado sua aplicação em diversos serviços ligado à área de multimídia. Dentre as diferentes técnicas reportadas pela literatura, as multimodais são consideradas mais promissoras, dado a capacidade de extrair informações de diferentes mídias de maneira potencialmente complementar, possibilitando obter segmentações mais significativas. Ao usar informações de diferentes naturezas, tais técnicas enfrentam dificuldades para modelar e obter uma representação combinada das informações ou com elevado custo ao processar cada fonte de informação individualmente. Encontrar uma combinação adequada de informação que aumente a eficácia da segmentação a um custo computacional relativamente baixo torna-se um desafio. Paralelamente, abordagens baseadas em Aprendizagem Profunda mostraram-se eficazes em uma ampla gama de tarefas, incluindo classificação de imagens e vídeo. Técnicas baseadas em Aprendizagem Profunda, como as Redes Neurais Convolucionais (CNNs), têm alcançado resultados impressionantes em tarefas relacionadas por conseguirem extrair padrões significativos dos dados, incluindo multimodais. Contudo, CNNs não podem aprender adequadamente os relacionamentos entre dados que estão temporalmente distribuídos entre as tomadas de uma mesma cena. Isto pode tornar a rede incapaz de segmentar corretamente cenas cujas características mudam entre tomadas. Por outro lado, Redes Neurais Recorrentes (RNNs) têm sido empregadas com sucesso em processamento textual, pois foram projetadas para analisar sequências de dados de tamanho variável e podem melhor explorar as relações temporais entre as características de tomadas relacionadas, potencialmente aumentando a eficácia da segmentação em cenas. Há uma carência de métodos de segmentação multimodais que explorem Aprendizagem Profunda. Assim, este trabalho de doutorado propõe um método automático de segmentação de vídeo em cenas que modela o problema de segmentação como um problema de classificação. O método conta com um modelo que combina o potencial de extração de padrões das CNNs com o processamento de sequencias das RNNs. O modelo proposto elimina a dificuldade de modelar representações multimodais das diferentes informações de entrada além de permitir instanciar diferentes abordagens para fusão multimodal (antecipada ou tardia). Tal método foi avaliado na tarefa de segmentação em cenas utilizando uma base de vídeos pública, comparando os resultados obtidos com os resultados de técnicas em estado-da-arte usando diferentes abordagens. Os resultados mostram um avanço significativo na eficácia obtida. / Automatic video scene segmentation is a current and relevant problem given its application in various services related to multimedia. Among the different techniques reported in the literature, the multimodal ones are considered more promising, given the ability to extract information from different media in a potentially complementary way, allowing for more significant segmentations. By processing information of different natures, such techniques faces difficulties on modeling and obtaining a combined representation of information and cost problems when processing each source of information individually. Finding a suitable combination of information that increases the effectiveness of segmentation at a relatively low computational cost becomes a challenge. At the same time, approaches based on Deep Learning have proven effective on a wide range of tasks, including classification of images and video. Techniques based on Deep Learning, such as Convolutional Neural Networks (CNNs), have achieved impressive results in related tasks by being able to extract significant patterns from data, including multimodal data. However, CNNs can not properly learn the relationships between data temporarily distributed among the shots of the same scene. This can lead the network to become unable to properly segment scenes whose characteristics change among shots. On the other hand, Recurrent Neural Networks (RNNs) have been successfully employed in textual processing since they are designed to analyze variable-length data sequences and can be developed to better explore the temporal relationships between low-level characteristics of related shots, potentially increasing the effectiveness of scene segmentation. There is a lack of multimodal segmentation methods exploring Deep Learning. Thus, this thesis proposes an automatic method for video scene segmentation that models the problem of segmentation as a classification problem. The method relies on a model developed to combine the potential for extracting patterns from CNNs with the potential for sequence processing of the RNNs. The proposed model, different from related works, eliminates the difficulty of modeling multimodal representations of the different input information, besides allowing to instantiate different approaches for multimodal (early or late) fusion. This method was evaluated in the scene segmentation task using a public video database, comparing the results obtained with the results of state-of-the-art techniques using different approaches. The results show a significant advance in the efficiency obtained.

Este trabalho trata do problema de otimização do resultado para uma empresa (maximização da receita) que presta o serviço de transporte de carga conteinerizada por via marítima num cenário onde a demanda dos clientes supera a oferta de capacidade disponível. No trabalho são descritas as características do problema da escolha do mix de carga em um horizonte de planejamento típico de várias semanas (tipicamente de 6 a 8) com detalhamento diário, ou seja, caracterizando um problema de múltiplos períodos. Também são consideradas as restrições de peso e de volume dos navios utilizados e disponibilidade de contêineres. A proposta é maximizar o retorno gerado pelo transporte através da escolha do conjunto de clientes que apresentem a melhor rentabilidade total dentre um universo definido em um processo mensal de levantamento de demanda. Contribuem para o aumento da complexidade da modelagem do problema a necessidade de utilização de variáveis inteiras em função da escolha de cada carga de cada cliente, considerando também vários navios em múltiplas rotas e programação do reposicionamento de contêineres vazios em conjunto com os cheios. Para a resolução deste problema foi desenvolvida uma heurística para a solução do modelo matemático que analisa a rentabilidade relativa de cada carga segundo critérios de ocupação (volume e peso) dos navios. A partir daí, as cargas mais rentáveis são alocadas e são verificadas as restrições de capacidade do navio utilizado e a disponibilidade de contêineres vazios em um processo interativo, onde são analisadas as opções de reposicionamento dos contêineres vazios até a obtenção da solução ou recusa da carga. A heurística proposta permite considerar diferentes critérios de rentabilidade dos clientes e cargas, tendo sido aplicada a um problema real para a sua validação. Os resultados obtidos mostram uma oportunidade de melhoria no processo atual tanto no aspecto de aumento da lucratividade do negócio, objetivo principal do trabalho, como em outras questões como a programação antecipada do reposicionamento dos contêineres vazios e a flexibilização do espaço alocado para esta movimentação. / This work presents a problem of profit optimization (revenue maximization) in container shipment company by maritime modal in a scenario where demand surplus the available capacity. In this work is described the characteristics of cargo mix problem in a multiple planning period of several weeks (typically from 6 to 8) with diary scheduling, which means that it turns to a multiple period problem. It is also considered the weight and volume restrictions of the boats and availability of containers. The proposal is to maximize the return generated by the transportation service by choosing the set of clients that generates the bigger profitability in a major group defined in a monthly process of demand evaluation. The complexity of the problem is enhanced by the utilization of integer variables, because of the need of choosing each client and its cargo, also considering that it may be more than one ship in multiple routes and the empty containers transportation along with the cargo. For the solution of the problem, it was developed a heuristic for the solution of the mathematical model that analyses the profitability of each client according to occupation (volume and weight) of the ships. From this moment on, the most profitable cargos are allocated and the restriction of capacity of the ships and availability of empty containers are in a interactive process until the solution is found or the client discharged. This method was tested in a real problem of a maritime transportation company, comparing the various profitability criteria of the clients and cargos. The obtained results show an opportunity of actual process improvement concerning the profitability of the business, main objective of this work, as in the anticipation of the empty containers repositioning and flexibilization of the space allocated to this operation.

Zusammenhänge der visuell-vestibulären Interaktion im Kortex des Menschen sollten auf grafisch individuell rekonstruierten und entfalteten Kortexoberflächen dargestellt und in Bezug zu den bisher vor allem aus tierexperimentellen Studien gewonnenen Daten (Guldin & Grüsser, 1998) diskutiert werden. Durch die Experimente sollte ein Beitrag zu einem grundlegenden Verständnis räumlicher Orientierungsprozesse beim Menschen geleistet werden. Mittels funktioneller Kernspintomografie (fMRI) wurden fünf Probanden untersucht, welche während zweier Experimente mit jeweils einem vestibulärem sowie einem visuellen Reiz stimuliert wurden (vestibuläre kalorische Stimualtion, visuelle Bewegungsstimulation). Bei den gemessenen Probanden kam es jeweils bei mindestens einem der zwei vestibulären Datensätze (li./re. kalorische Spülung) in der Conjunction Analysis zu Überlappungen mit den dargestellten Arealen der visuellen Stimulation. Es konnte ein weitverzweigtes System von visuell-vestibulären kortikalen Arealen dargestellt werden, mit Aktivierungen des präcentralen Kortex, Teilen des Sulcus centralis, der hinteren Insel, des hinteren Sulcus Cinguli, des Sulcus intraparietalis, sowie des lateralen Okzipitallappens. Interindividuelle Unterschiede manifestierten sich in der Größe der Überlappungen und deren genauer Lokalisation im jeweiligen Sulcus. Die Ergebnisse bestätigten einen Teil der aus tierexperimentellen Studien bekannten Beobachtungen. Erstmals konnte beim Menschen eine vestibuläre Erregung des MT/MST-Komplexes, wie es im Macaque-Affen beschrieben wurde (Bremmer et al., 1999) belegt werden. Ebenso wurde die Multimodalität von MT/MST und des Arelas PIVC im humanen Kortex erstmals beschrieben. / We aimed to identify convergence between visual and vestibular afferents in the human cortex on individually unfolded cortical hemispheres by functional magnetic resonance imaging (fMRI). Furthermore analogies with studies in different non-human primates (Guldin & Gruesser, 1998) were discussed. Five normal subjects were examined with fMRI with two separate experimental paradigms. We implemented caloric vestibular stimulation as first experiment, additionally visual motion stimulation was executed. In at least one out of two vestibular datasets (right, left caloric stimulation) overlaps in the conjunction analysis with the visual motion paradigm occurred. A wide-spread system of cortical visual-vestibular convergence was shown. The activation map comprised foci in the precentral cortex, parts of the central sulcus, the posterior insula, the posterior cingular sulcus, the intraparietal sulcus and the lateral occipital sulcus. Individual differences occurred in respect to the size of convergence and specific localization within the sulci. The results partly confirm observations made in non-human primates. For the first time vestibular input to the MT/MST-complex was shown, as describe in the macaque (Bremmer et al., 1999). Additional multimodal input was seen in area PIVC.

Based on cross-modal learning and multiple resources theory, human performance can be improved by receiving and processing additional streams of information from the environment. In alarm situations, alarm meanings need to be distinguishable from each other and learnable for users. In audible alarms, by manipulating the temporal characteristics of sounds different audible signals can be generated. However, in some cases such as in using discrete medical alarms, when there are too many audible signals to manage, changes in temporal characteristics may not generate discriminable signals that would be easy for listeners to learn. Multimodal displays can be developed to generate additional auditory, visual, and tactile stimuli for helping humans benefit from cross-modal learning and multiple attentional resources for a better understanding of the alarm situations. In designing multimodal alarm displays in work domains where the alarms are predominantly auditory-based and where accessing visual displays is not possible at all times, tactile displays can enhance the effectiveness of alarms by providing additional streams of information for understanding the alarms. However, because of low information density of tactile information presentation, the use of tactile alarms has been limited. In this thesis, by using human subjects, the learnability of auditory and tactile alarms, separately and together in an audio-tactile display were studied. The objective of the study was to test cross-modal learning when messages of an alarm (i.e. meaning, urgency level) were conveyed simultaneously in audible, tactile and audio-tactile alarm displays. The alarm signals were designed by using spatial characteristics of tactile, and temporal characteristics of audible signals separately in audible and tactile displays as well as together in an audio-tactile display. This study explored if using multimodal alarms (tactile and audible) would help learning unimodal (audible or tactile) alarm meanings and urgency levels. The findings of this study can help for design of more efficient discrete audio-tactile alarms that promote learnability of alarm meanings and urgency levels.

Bakgrunden till min studie kommer ursprungligen från mitt eget intresse att diskutera ämnesintegrerad undervisnings betydelse i skola och samhälle. Syftet med föreliggande studie är att utifrån ett multimodalt perspektiv ta reda på hur en musiklärare och en specialpedagog i matematik samarbetar för att integrera matematik och musik i sin undervisning i årskurs 1-3. Jag har använt mig av observationer och samtal för att ta reda på hur de arbetar, vilka redskap de använder sig av samt varför de arbetar som de gör.     Den undervisning jag har observerat kallas mattemusik och är en utarbetad metod för ämnesintegrerat lärande. I resultatet visas att det är läroplanens mål och kunskapskrav i matematik och musik som styr undervisningen och att de två lärarna anser att det sätt som undervisningen bedrivs på även uppfyller många sociala mål som till exempel samarbetsförmåga, hänsynstagande och turtagning. Lektionerna i mattemusik utgick från ett tema och sedan arbetade lärarna tillsammans med eleverna mot den nya kunskapen ifrån många olika håll och med flera sinnen involverade. Undervisningen var i många avseenden multimodal då även momenten under lektionerna i sig själva var det.     Det jag tar upp i diskussionen är bland annat avsaknaden av det taktila sinnet som inte fanns representerat i lika stor utsträckning som de visuella, auditiva och kinestetiska sinnena. / The background of my study is originally my own interest in discussing the role of crossover teaching in school and society. The purpose of this study is to discern how one music teacher and one special education teacher in mathematics work to integrate mathematics and music in their teaching in grades 1-3 from a multimodal perspective. I have used observations and conversations to determine how the teachers work, why they work the way they do, and what tools they are using.     The teaching I have observed is called math-music and it is an elaborate method of crossover teaching in mathematics and music. The result shows that it is the curriculum goals and knowledge in mathematics and music that governs the teaching and that the two teachers believe that the way teaching is carried on also fulfills many social goals such as teamwork, consideration and turn-taking. The classes in math-music were based on one theme, with  teachers and students working together from different perspectives, involving multiple senses. In many respects the teaching was multimodal, in which even different parts of the teaching, one by one, was multimodal in themselves.     What I discuss includes the lack of the tactile sense, which was not represented as much as the visual, auditory and kinesthetic senses.

Augmented Reality (AR) has the possibility of interacting with virtual objects and real objects at the same time since it combines the real world with computer-generated contents seamlessly. However, most AR interface research uses general Virtual Reality (VR) interaction techniques without modification. In this research we develop a multimodal interface (MMI) for AR with speech and 3D hand gesture input. We develop a multimodal signal fusion architecture based on the user behaviour while interacting with the MMI that provides more effective and natural multimodal signal fusion. Speech and 3D vision-based free hand gestures are used as multimodal input channels. There were two user observations (1) a Wizard of Oz study and (2)Gesture modelling. With the Wizard of Oz study, we observed user behaviours of interaction with our MMI. Gesture modelling was undertaken to explore whether different types of gestures can be described by pattern curves. Based on the experimental observations, we designed our own multimodal fusion architecture and developed an MMI. User evaluations have been conducted to evaluate the usability of our MMI. As a result, we found that MMI is more efficient and users are more satisfied with it when compared to the unimodal interfaces. We also describe design guidelines which were derived from our findings through the user studies.

Bakgrund: Långvarig smärta är ett stort folkhälsoproblem som innebär stora kostnader för samhället och självklart också stort lidande för individen. Multimodal smärtrehabilitering (MMR) innebär att olika yrkeskategorier arbetar gemensamt kring dessa patienter. Forskningen visar måttligt till starkt vetenskapligt stöd för MMR vid komplex smärtproblematik. Det finns kunskapsluckor kring vilken typ av fysioterapeutiska interventioner som används vid multimodal smärtrehabilitering. Syfte: Syftet med studien var att kartlägga fysioterapeuternas arbete inom ramen för multimodal smärtrehabilitering i primärvården. Metod: Enkätstudie av tvärsnittstyp med kvantitativ ansats baserad på en egenkonstruerad webbenkät. Populationen utgjordes av fysioterapeuter anslutna till Nationella Registret över Smärtrehabilitering (NRS) primärvård och resultatet bygger på de 23 fysioterapeuter som valde att svara på webbenkäten. Resultat: Resultaten baseras på svar från 71 % av de NRS-anslutna klinikerna. Sammanfattning av resultaten visade framför allt att råd/undervisning samt olika former av fysisk träning utgjorde grunden i fysioterapeutens arbete inom MMR-team i primärvården. Det framkommer att så gott som alla patienter träffade fysioterapeut under behandlingsperioden. Det var en klar övervikt mot gruppbehandling eller en kombination av grupp och individuell behandling gällande de fysioterapeutiska interventionerna. Behandlingsperioderna var för det mesta fyra till elva veckor där patienten träffade fysioterapeut oftast varje vecka eller flera gånger/vecka. I primärvårdens MMR-team var fysioterapeut, arbetsterapeut, KBT terapeut, läkare och rehabkoordinator de vanligast förekommande yrkeskategorierna i teamen. Slutsats: Fysioterapeuten har en central roll i primärvårdens MMR-team och använder sig primärt av evidensbaserade, aktiva interventioner. / Background: Chronic pain is a major public health problem which involves high costs for society and, of course, also great suffering for the individual. Multimodal rehabilitation (MMR) means that different occupational categories work together around these patients. Research shows moderately to strong scientific evidence for MMR in complex pain problems. There are a lack of evidence about which type of physiotherapeutic interventions that are used in multimodal pain rehabilitation. Aim: The aim was to study which physiotherapy interventions that are used in multimodal primary healthcare rehabilitation in Sweden. Method: Cross-sectional survey with quantitative approach based on a self-designed web questionnaire. The population consisted of physiotherapists working in clinics reporting to the Swedish Quality Registry for Pain Rehabilitation (SQRP) in primary care and the result is based on the 23 physiotherapists who responded to the web questionnaire. Results: The results are based on 71% of the NRS-affiliated clinics. Counseling/teaching and various forms of physical training formed the basis of the physiotherapist's work in MMR teams in primary care. Almost all patients meet physiotherapists during the treatment period. Group treatment or a combination of group and individual treatment were most common. The treatment periods were mostly between four to eleven weeks where the patient met a physiotherapist most often weekly or several times a week. In the primary care MMR-team, the physiotherapist, occupational therapist, KBT therapist, physician and rehab coordinator were the most common occupational categories. Conclusion: Physiotherapists have a central role in the MMR-teams in primary care, they primarily use evidence-based, active interventions.