Avatars are used more and more in online communication, in both games and socialmedia. At the same time technology for facial puppetry, where expressions of the user aretransferred to the avatar, has developed rapidly. Why is it that facial puppetry, despite this,is conspicuous by its absence? This thesis analyzes the available and upcoming solutions for facial puppetry, if a com-mon framework or library can exist and what can be done to simplify the process for de-velopers who wants to implement facial puppetry. A survey was conducted to get a better understanding of the technology. It showedthat there is no standard yet for how to describe facial expressions, but part of the marketis converging towards a common format. It also showed that there is no existing inter-face that can handle communication with tracking devices or translation between differentexpression formats. Several prototypes for recording and streaming facial expression data from differentsources were implemented as a practical test. This was done to evaluate the complexity ofimplementing real-time facial puppetry. It showed that it is not always possible to integratethe available tracking solutions into an existing project. When integration was possible itrequired a lot of work. The best way to get tracking right now seems to be to implement astandalone program for tracking that streams the tracked data to the main application. In summary it is the poor integrability of the solutions that makes it problematic forthe developers, together with a wide variety of facial expression formats. A software thatacts like a bridge between the tracking solutions and the game could allow for translationbetween different formats and simplify implementation of support. In the future, instead of working towards making all tracking solutions output stan-dardized tracking data, research further how to build a framework that can handle differ-ent configurations. / <p>Examensarbetet är utfört vid Institutionen för teknik och naturvetenskap (ITN) vid Tekniska fakulteten, Linköpings universitet</p>

The use of Virtual Reality (VR) in a variety of professional, military, governmental, and educational fields has continued to expand over the past several decades, and the recent Covid-19 pandemic has brought attention to this field. This study surveys 154 college students over 23 questions that include various demographics that can be used to look for discriminators, multiple-choice VR-related questions, as well as a few free-form questions about use of VR in learning environments. The students’ experience with, interest in, and thoughts on how to best use VR vary considerably. The Covid-19 pandemic is found to have limited impact thus far in terms of VR use, but the interest in using VR in schools since then has generally increased quite a bit. Commitment to invest in VR were it to be expanded and provide continual feedback varies quite a bit as well but is strong. A statistical 2 analysis shows that, at a high confidence level, males generally are more experienced with VR in general, have a greater interest in seeing VR implemented further, and are more committed to radical changes in educational methodology than females are. In addition, it is found that Hispanics/Latinos, Black / African Americans, Pacific Islanders, and those of mixed race are more inclined to provide continual feedback as regards the implementation of VR in the school curriculum than (non-Hispanic) White and Asian people are.

Актуальность данной работы обусловлена тем, что музейные комплексы с началом развития компьютерных технологий стали терять свою привлекательность для посетителей. Развитие технологий привело к созданию новых форм общения и социального взаимодействия. Для поддержания конкурентоспособности музеям в настоящее время необходимо все больше взаимодействовать с IT-сферой. Возможности дополненной реальности позволили создать целый ряд новых приложений, принципиально меняющих подходы к предоставлению пользователю услуг, в том числе и в музейной деятельности. Объектом исследования являются технологии дополненной реальности, применяемые в туристской отрасли. Предметом исследования является применение технологий дополненной реальности в музейных экспозициях Екатеринбурга. Цели исследования: а) оценить перспективность использования технологий дополненной реальности в экспозициях музеев Екатеринбурга; б) предложить проект создания музейной экспозиции с применением технологий дополненной реальности. Поставленная цель предопределила решение следующих задач: 1. Теоретически описать изучаемый предмет, очертить понятийные рамки изучаемого предмета. 2. Провести эмпирическое исследование, направленное на оценку. 3. Интерпретировать полученные данные, исходя из гипотез исследования. В процессе решения поставленных задач были получены следующие результаты: 1. Музейные экспозиции с технологиями дополненной реальности пользуются спросом. 2. Регулярные обновления AR-приложения помогают увеличить поток посетителей музеев. 3. Посетители музеев готовы платить за новые впечатления, полученные с помощью дополненной реальности, больше. 4. В последней главе предлагается проект создания музейной экспозиции с применением технологий дополненной реальности, посвященный Универсиаде 2023. Полученные автором результаты содержат предложения по внедрению технологий дополненной реальности в экскурсионные услуги музеев Екатеринбурга. Результаты исследования представляют ценность для туристского потенциала города и могут найти применение в музейных экспозициях. Диссертация состоит из введения, четырех глав, заключения и двух приложений. / Relevance of the research topic. The relevance of this work is due to the fact that museum complexes with the beginning of the development of computer technologies began to lose their attractiveness for visitors. Advances in technology have led to the creation of new forms of communication and social interaction. To maintain competitiveness, museums now need to increasingly interact with the IT sector. The possibilities of augmented reality have made it possible to create a number of new applications that fundamentally change the approaches to the provision of services to the user, including in museum activities. Object of study. The object of the research is augmented reality technologies used in the tourism industry. Subject of study. The subject of this research is the use of augmented reality technologies in museum exhibitions in Yekaterinburg. Goals and objectives of the study. Research objectives: a) assess the prospects of using augmented reality technologies in expositions of museums in Yekaterinburg; b) propose a project for creating a museum exposition using augmented reality technologies. This goal predetermined the solution of the following tasks: 1. Theoretically describe the studied subject, outline the conceptual framework of the studied subject. 2. Conduct empirical evaluation research. 3. Interpret the findings based on the hypotheses of the research. In the process of solving the assigned tasks, the following results were obtained: 1. Museum exhibitions with augmented reality technologies are in demand. 2. Regular updates of AR applications help to increase the flow of visitors to museums. 3. Visitors to museums are willing to pay more for new augmented reality experiences. 4. The last chapter proposes a project to create a museum exhibition using augmented reality technologies dedicated to the Universiade 2023. The results obtained by the author contain proposals for the implementation of augmented reality technologies in the excursion services of museums in Yekaterinburg. The research results are valuable for the tourist potential of the city and can be used in museum exhibitions. The dissertation consists of an introduction, four chapters, a conclusion and two appendices.

Научная новизна исследования состоит в выявлении потенциала иммерсивных цифровых технологий и их прикладного использования в продвижении территории. Практическая значимость исследования заключается в разработке путей повышения эффективности применения цифровых технологий, в том числе и иммерсивных, в продвижении территории. Эффективность исследования — предложенные автором разработки позволят рациональнее применять цифровые технологии в продвижении города Екатеринбург, внедрить их в существующие решения, а также учитывать тенденции развития цифровых технологий при реализации дальнейших шагов в продвижении территории. / The scientific novelty of the study is to identify the potential of immersive digital technologies and their applied use in the promotion of the territory. The practical significance of the research lies in the development of ways to increase the effectiveness of the use of digital technologies, including immersive ones, in the promotion of the territory. The effectiveness of the research — the developments proposed by the author will make it possible to use digital technologies more efficiently in the promotion of the city of Yekaterinburg, introduce them into existing solutions, and also take into account the trends in the development of digital technologies when implementing further steps in the promotion of the territory.

Цель работы – проектирование и методологическое обеспечение разрабатываемого сервиса. Объект исследования – пользовательский опыт взаимодействия с разрабатываемыми прототипами. Методы исследования: анализ существующих решений, методы активации творческого мышления, метод экспертных оценок, анкетирование. Результаты работы: сформулирована методика использований фотограмметрии для создания 3d моделей мебели, разработаны прототипы и вспомогательное ПО мебельного маркетплейса «МЕБЕЛЬ+Р». / The purpose of the work is the design and methodological support of the developed service. The object of the study is the experience of interaction with developed prototypes. Research methods: analysis of repeated decisions, methods of activation of creative thinking, method of expert assessments, questioning. Results of the work: a photogrammetry technique was developed for creating 3d models of furniture, prototypes and auxiliary software for the furniture marketplace "FURNITURE + R" were developed.

Со второй половины XX века всё большую значимость получает неценовая конкуренция, которая стала постепенно вытеснять ценовую. В такой же ситуации оказались и регионы, которые тоже имеют свою конкуренцию с другими территориями за туристический поток, приток квалифицированных рабочих, ученых, деятелей культуры и прочих людей, которые могут стимулировать социально-экономический рост региона. Правильные коммуникации создают желательную репутацию, которая зачастую является сильным преимуществом перед другими конкурентами, повышая стоимость продукта или услуги, или помогая потребителю в принятии решения по поводу путешествия или переезда именно в этот регион, а не в другой. Одними из новых и быстро развивающихся технологий, которые всё чаще используются в качестве инструментов продвижения территории, являются виртуальная и дополненная реальности. Всё больше менеджеров по продвижению региона начинают их осваивать, но при этом они до сих пор не являются распространёнными для этой области. В представленной работе отражены особенности использования инструментов виртуальной и дополненной реальностей в продвижении территории, проанализированы практика и результаты научных исследований в этой области. Основываясь на результатах методологической и эмпирической базы исследований, разработана коммуникативная программа продвижения Пермского края как региона с использованием инструментов виртуальной и дополненной реальностей. / Since the second half of the 20th century, non-price competition has become increasingly important, gradually replacing price competition. In the same situation, regions have also found themselves in competition with other territories for tourist flows, influx of skilled workers, scientists, cultural figures and other people who can stimulate the socio-economic growth of the region. The right communications create a desirable reputation, which is often a strong advantage over other competitors, adding value to a product or service, or helping the consumer decide to travel or relocate to that particular region rather than another. Virtual and augmented reality are some of the new and rapidly evolving technologies that are increasingly being used as tools to promote a territory. More and more promotional managers are beginning to master them, but they are still not common for this area. This paper reflects the peculiarities of virtual and augmented reality tools in territory promotion, analyses the practice and research results in this field. Based on the results of the methodological and empirical research base, a communication programme for the promotion of the Perm Territory as a region using virtual and augmented reality tools has been developed.

Fundamentally, interaction design is about getting in between: Between people and what is important to them. While it often makes sense to make mediated interactions explicit, implicit interactions can integrate harmoniously into the here and now and happen almost unnoticed. In this master’s thesis, I explore different approaches to harmoniously integrate interactions with mobile and wearable devices into cycling. Cycling can be treated as a modality. It can influence the behaviour of the operating system or provide considerations for the design of applications. Rather than adding to the existing flow of signals, I propose to tap into that flow and even take it a step further: I explore ways to add meaning to what people are already doing as if there were no interface. Wearable devices, such as the current Apple Watch, can provide comprehensive insight into ongoing activities. The name "smartwatch" suggests that this device is meant to be attended to, but the greater opportunity lies in sensing and adding meaning to what is already happening. This allows people to immerse themselves in their ‘natural’ environment with their heads up and their hands free.

Indiana University-Purdue University Indianapolis (IUPUI) / Simultaneous localization and mapping (SLAM) is a general device localization technique that uses realtime sensor measurements to develop a virtualization of the sensor's environment while also using this growing virtualization to determine the position and orientation of the sensor. This is useful for augmented reality (AR), in which a user looks through a head-mounted display (HMD) or viewfinder to see virtual components integrated into the real world. Visual SLAM (i.e., SLAM in which the sensor is an optical camera) is used in AR to determine the exact device/headset movement so that the virtual components can be accurately redrawn to the screen, matching the perceived motion of the world around the user as the user moves the device/headset. However, many potential AR applications may need access to more than device localization data in order to be useful; they may need to leverage environment data as well. Additionally, most SLAM solutions make the naive assumption that the environment surrounding the system is completely static (non-moving). Given these circumstances, it is clear that AR may benefit substantially from utilizing a SLAM solution that detects objects that move in the scene and ultimately provides localization data for each of these objects. This problem is known as the dynamic SLAM problem. Current attempts to address the dynamic SLAM problem often use machine learning to develop models that identify the parts of the camera image that belong to one of many classes of potentially-moving objects. The limitation with these approaches is that it is impractical to train models to identify every possible object that moves; additionally, some potentially-moving objects may be static in the scene, which these approaches often do not account for. Some other attempts to address the dynamic SLAM problem also localize the moving objects they detect, but these systems almost always rely on depth sensors or stereo camera configurations, which have significant limitations in real-world use cases. This dissertation presents a novel approach for registering and localizing unknown moving objects in the context of markerless, monocular, keyframe-based SLAM with no required prior information about object structure, appearance, or existence. This work also details a novel deep learning solution for determining SLAM map initialization suitability in structure-from-motion-based initialization approaches. This dissertation goes on to validate these approaches by implementing them in a markerless, monocular SLAM system called LUMO-SLAM, which is built from the ground up to demonstrate this approach to unknown moving object registration and localization. Results are collected for the LUMO-SLAM system, which address the accuracy of its camera localization estimates, the accuracy of its moving object localization estimates, and the consistency with which it registers moving objects in the scene. These results show that this solution to the dynamic SLAM problem, though it does not act as a practical solution for all use cases, has an ability to accurately register and localize unknown moving objects in such a way that makes it useful for some applications of AR without thwarting the system's ability to also perform accurate camera localization.

This study details the development of a neural network model designed for real-time semantic segmentation, specifically to distinguish sky pixels from other elements within an image. The model is incorporated into a feature for an Augmented Reality application in Unity, leveraging Unity Barracuda—a versatile neural network inference library. While Barracuda offers cross-platform compatibility, it poses challenges due to its lack of support for certain layers and operations. Consequently, it lacks the support of most state-of-the-art models, and this study aims to provide a model that works within Barracuda.  Given Unity's absence of a framework for model development, the development and training of the model was conducted in an open-source machine learning library. The model is continuously evaluated to optimize the trade-off between prediction accuracy and operational speed.   The resulting model is able to predict and classify each pixel in an image at around 137 frames per second. While its predictions might not be on par with some of the top-performing models in the industry, it effectively meets its objectives, particularly in the real-time classification of sky pixels within Barracuda. / Denna rapport beskriver utvecklingen av en neural nätverksmodell avsedd för semantisk segmentering i realtid, specifikt för att särskilja himlen från andra element inom en bild. Modellen integreras i en funktion för en applikation med augmenterad verklighet i Unity, med hjälp av Unity Barracuda - ett mångsidigt bibliotek för neurala nätverk. Även om Barracuda erbjuder kompatibilitet över olika plattformar, medför det utmaningar på grund av dess brist på stöd för vissa lager och operationer. Följaktligen saknar den stöd från de bäst presterande modellerna, och denna studie syftar till att erbjuda en modell som fungerar inom Barracuda. Med tanke på Unitys avsaknad av ett ramverk för modellutveckling valdes ett open-source maskininlärningsbibliotek. Modellen utvärderas kontinuerligt för att optimera avvägningen mellan förutsägelseprecision och driftshastighet. Den resulterande modellen kan förutsäga och klassificera varje pixel i en bild med en hastighet på cirka 137 bilder per sekund. Även om dess förutsägelseprecision inte är i nivå med några av de bäst presterande modellerna inom branschen, uppfyller den effektivt sina mål, särskilt när det gäller realtidsklassificering av himlen inom Barracuda.

This thesis investigates the use of virtual avatar heads to enhance local video see-through collaboration in a mixed reality environment. When users engage with each other using a head-mounted display, the device sits atop their head and obstructs their view, concealing their gaze and facial expressions while collaborating. This makes communication harder by removing non-verbal hints. The proposed solution aims to restore them by rendering virtual avatar heads on top of users heads. The study examines the impact of different avatar styles, including the potential effects of the uncanny valley, as well as the use of lip syncing versus facial tracking to animate avatar mouths. An application was developed using Unity3D to implement this solution, allowing two users to collaborate in a mixed reality environment with avatars on their heads. An experiment was conducted with 56 participants, where users collaborated in two tasks: the twenty question game and a collaborative object placement task. A between-subject design was used to compare with and without avatars, avatar rendering type and avatar lip animation. During the experiment, social presence, user experience and performance were measured through questionnaires (Networked Minds of social presence, NASA TLX and User Experience Questionnaire) and eye gaze data. The study found that although there was not always a strong difference between no avatar and avatars, the use of cartoon avatars with lip syncing was the most favorable option, enhancing the user’s comfort and facilitating interpretation of their partner’s emotions and feelings, as well as receiving more attention from them, which could be due to the Uncanny Valley. However, no evidence of performance improvement was found. The findings of this study have important implications for the design of collaborative mixed reality environments, highlighting the potential benefits of using virtual avatars to enhance communication and social presence. The study also underscores the importance of avatar style and facial animation as well as the potential impact of the uncanny valley on user experience. / Denna avhandling undersöker användningen av virtuella avatarhuvuden för att förbättra lokalt samarbete med genomskinlig video i en mixed reality-miljö. När användare interagerar med varandra med hjälp av en huvudmonterad display sitter enheten ovanpå huvudet och skymmer sikten, döljer blicken och ansiktsuttrycken medan de samarbetar. Detta försvårar kommunikationen genom att ta bort icke-verbala ledtrådar. Den föreslagna lösningen syftar till att återställa dem genom att rendera virtuella avatarhuvuden ovanpå användarnas huvuden. Studien undersöker effekterna av olika avatarstilar, inklusive de potentiella effekterna av uncanny valley, samt användningen av läppsynkronisering kontra ansiktsspårning för att animera avatarernas munnar. En applikation utvecklades med Unity3D för att implementera denna lösning, så att två användare kan samarbeta i en mixed reality-miljö med avatarer på sina huvuden. Ett experiment genomfördes med 56 deltagare, där användarna samarbetade i två uppgifter: spelet med tjugo frågor och en samarbetsuppgift för objektplacering. En mellanobjektsdesign användes för att jämföra med och utan avatarer, avatarens renderingstyp och avatarens läppanimering. Under experimentet mättes social närvaro, användarupplevelse och prestanda med hjälp av frågeformulär (Networked Minds of social presence, NASA TLX och User Experience Questionnaire) och ögonstyrningsdata. Studien visade att även om det inte alltid fanns en stark skillnad mellan ingen avatar och avatarer, var användningen av tecknade avatarer med läppsynkronisering det mest fördelaktiga alternativet, vilket förbättrade användarens komfort och underlättade tolkningen av deras partners känslor och sinnestämming, samt fick mer uppmärksamhet från dem, vilket kan bero på Uncanny Valley. Det fanns dock inga bevis för att prestandan förbättrades. Resultaten av denna studie har viktiga konsekvenser för utformningen av kollaborativa mixed reality-miljöer och belyser de potentiella fördelarna med att använda virtuella avatarer för att förbättra kommunikationen och den sociala närvaron. Studien understryker också vikten av avatarstil och ansiktsanimering samt den potentiella inverkan av uncanny valley på användarupplevelsen.