Virtual reality (VR) headsets are becoming more common and will require evolving input mechanisms to support a growing range of applications. Because VR devices require users to wear head-mounted displays, there are accomodations that must be made in order to support specific input devices. One such device, a keyboard, serves as a useful tool for text entry. Many users will require assistance towards using a keyboard when wearing a head-mounted display. Developers have explored new mechanisms to overcome the challenges of text-entry for virtual reality. Several games have toyed with the idea of using motion controllers to provide a text entry mechanism, however few investigations have made on how to assist users in using a physical keyboard while wearing a head-mounted display. As an alternative to controller based text input, I propose that a software tool could facilitate the use of a physical keyboard in virtual reality. Using computer vision, a user€™s hands could be projected into the virtual world. With the ability to see the location of their hands relative to the keyboard, users will be able to type despite the obstruction caused by the head-mounted display (HMD). The viability of this approach was tested and the tool released as a plugin for the Unity development platform. The potential uses for the plugin go beyond text entry, and the project can be expanded to include many physical input devices.

This dissertation examines the developments and progress of spatial calibration procedures for Optical See-Through (OST) Head-Mounted Display (HMD) devices for visual Augmented Reality (AR) applications. Rapid developments in commercial AR systems have created an explosion of OST device options for not only research and industrial purposes, but also the consumer market as well. This expansion in hardware availability is equally matched by a need for intuitive standardized calibration procedures that are not only easily completed by novice users, but which are also readily applicable across the largest range of hardware options. This demand for robust uniform calibration schemes is the driving motive behind the original contributions offered within this work. A review of prior surveys and canonical description for AR and OST display developments is provided before narrowing the contextual scope to the research questions evolving within the calibration domain. Both established and state of the art calibration techniques and their general implementations are explored, along with prior user study assessments and the prevailing evaluation metrics and practices employed within. The original contributions begin with a user study evaluation comparing and contrasting the accuracy and precision of an established manual calibration method against a state of the art semi-automatic technique. This is the first formal evaluation of any non-manual approach and provides insight into the current usability limitations of present techniques and the complexities of next generation methods yet to be solved. The second study investigates the viability of a user-centric approach to OST HMD calibration through novel adaptation of manual calibration to consumer level hardware. Additional contributions describe the development of a complete demonstration application incorporating user-centric methods, a novel strategy for visualizing both calibration results and registration error from the user’s perspective, as well as a robust intuitive presentation style for binocular manual calibration. The final study provides further investigation into the accuracy differences observed between user-centric and environment-centric methodologies. The dissertation concludes with a summarization of the contribution outcomes and their impact on existing AR systems and research endeavors, as well as a short look ahead into future extensions and paths that continued calibration research should explore.

Virtual Reality är ett media med gamla rötter, men samtidigt en ny teknologi för allmänheten. Ett av problemen med VR är att det kan frammana visst obehag då det brukas, så kallad simulatorsjuka (Kennedy m.fl., 1993), vilket forskare studerar i syftet att förstå hur det undviks. I denna undersökning har en artefakt tagits fram i form av en spelprototyp, av slaget 2,5D-side-scrollingspel till VR, och använts för att studera uppkomsten av simulatorsjuka. Artefakten använder en head-mounted display för att uppnå VR. Tre testpersoner deltog i undersökningen, och för att mäta symptomen användes ett beprövat frågeformulär samt kvalitativa intervjuer. Data visar på att en tidigare hypotes om intryckskonflikt (Kolasinski, 1995, s.7) är inkorrekt, samt att konventionella orienteringsmekaniker har vissa fördelar. För framtida studier bör simulatorsjuka undersökas från fler vinklar och/eller med ett större urval av testpersoner.

München, Techn. Univ., Diss., 2008

Over the last few years, there have been striking developments in wearable computing. Among all the different forms of wearable devices, Head Mounted Displays (HMDs) are deemed the first seamless solution to enabling workers with real time contextual information and allowing companies to integrate with existing back-end systems. The hands-free feature that come along with the HMDs is also believed a great advantage over many traditional technologies. However, few studies had discussed the impact of different design characteristics of head mounted displays on task performance. This study aimed to find out how different display positions of Head Mounted Displays may affect the performance of workers performing guided repair and maintenance tasks. A set of car maintenance tasks were performed by 20 participants with task guidance presented at four Display Conditions: above-eye HMD, eye-centered HMD, below-eye HMD and the traditional paper manual. Time and errors were measured and discussed, as well as other user experience related measurements.

Conventional stereoscopic displays present a pair of stereoscopic images on a single and fixed image plane. In consequence, these displays lack the capability of correctly rendering focus cues (i.e. accommodation and retinal blur) and may induce the discrepancy between accommodation and convergence. A number of visual artifacts associated with incorrect focus cues in stereoscopic displays have been reported, limiting the applicability of these displays for demanding applications and daily usage. Depth-fused multi-focal-plane display was proposed to create a fixed-viewpoint volumetric display capable of rendering correct or nearly-correct focus cues in a stereoscopic display through a small number of discretely placed focal planes. It effectively addresses the negative effects of conventional stereoscopic displays on depth perception accuracy and visual fatigue. In this dissertation, the fundamental design methods and considerations of depth-fused displays were refined and extended based on previous works and a high-resolution optical see-through multi-focal-plane head-mounted display enabled by state-of-the-art freeform optics was developed. The prototype system is capable of rendering nearly-correct focus cues for a large volume of 3D space extending into a depth range from 0 to 3 diopters at flicker-free speed. By incorporating freeform optics, the prototype not only achieves high quality imagery across a large 3D volume for the virtual display path but it also maintains better than 0.5 arcminutes visual resolution of the see-through view. The optical design, implementation and experimental validation of the display are presented and discussed in detail.

Zugl.: München, Techn. Univ., Diss., 2008

VR is a relatively new technology that has been shown to have potential to increase productivity andfacilitate better decision making, both within the architecture role and in the broader building industry. However, there are many difficulties that stand in the way of this new technology. In this study, the researcher uses semi-structured interviews to interview six working architects from threecities in Sweden. This is an attempt to investigate what architects think are the technical difficultiesthat stand in the way of VR use within the architect role. Both software and hardware limitations are investigated. I additionally investigate what architects think are potential solutions to those problems, and how architects think VR can be used in the future in a way that would be useful and facilitate better communication. This study found that, according to the architects interviewed, the technical limitations are mainly the lack of portability, isolation from the outside world, the need for powerful hardware, motion sickness, movement restrictions and the setup process being generally inconvenient. The future use of VR that would be useful according to the architects interviewed are the ability to have virtual meetings, VR being used in conjunction with AR, the ability to design and sketch in VR and using VR as a communication tool to convey design ideas to the public.

Harvester operators are handling large amounts of information while processing trees. Primarily by looking at the close environment, the trees, and the harvesting head, but also through the bucking display placed in front of the operator. Head-mounted displays have been under development since the 1960s but have yet to find any major applications where they excel outside military aircrafts despite numerous tries. This master thesis aims at evaluating the usability for head-mounted displays (HMDs) and augmented reality (AR) technology within forestry, by answering the research question: Are HMDs suited for displaying information in harvesters? What information is suitable to display in HMDs and how should it be displayed for best effect? A literature review about the current state of the art of HMDs and AR has been compiled which can be read independently. Qualitative user studies have been performed to map the current interactions of harvester operators by observation and interviews in the field. Interviews have been made with subject matter experts in relating fields. The insights gathered from the user studies led to three interface designs for bucking which were designed, prototyped and tested for usability in a harvester simulator. Ideas for other more immersive uses of the HMD were also designed. For bucking is assortment and species the most important information for the operators, contrary to how it is displayed today, where diameter and fed length is presented as the most significant information. Near machine navigation is a problem area which may be solved with AR. Operators participating in the test were positive towards the technology after testing. Our conclusion is that HMDs shows enough promise and performance to be evaluated further by tests in the field. Weight, brightness, and a non-occluding design are the most important properties for an HMD for harvesters. / Skördaroperatörer hanterar stora mängder information vid fällning och aptering. Främst genom att titta på omgivningen, träden och aggregatet men också via apteringsskärmen framför sig. Head-mounted Displays (HMDs) har varit under utveckling sedan 1960 talet men har hittat få tillämpningar utanför det militära flyget trots många studier. Detta examensarbete försöker utreda om HMDs och augmented reality (AR) är lämpliga tekniker för det mekaniserade skogsbruket genom att svara på frågorna: Är HMDs lämpliga för att visa information för skördaroperatörer? Vilken information skall då visas och hur ska den visas på bästa sätt? En litteraturstudie om HMDs och AR har genomförts och kan läsas fristående. Kvalitativa användarstudier har genomförts för att kartlägga skördaroperatörers arbete genom observationer och intervjuer. Intervjuer har även gjorts med experter inom närliggande områden. Från studierna har tre gränssnitt utvecklats och testats i en skördarsimulator. Idéer för AR gränssnitt har även de utformats. För aptering är sortiment och trädslag den viktigaste informationen att visas, i motsats till dagens gränssnitt där diameter och utmatad längd visas tydligast. Navigation i närheten av skördaren är ett problemområde som skulle kunna lösas med hjälp av AR. Operatörer i användartester har varit positiva till tekniken. Vår slutsats är att HMDs är tillräckligt lovande för fortsatta tester i fält för skördaroperatörer. Vikt, ljusstyrka och en design som inte skymmer sikten är de viktigaste faktorerna för en HMD för skördarbruk.

This work has investigated how head-mounted-displays can enable more efficient and better work conditions for warehouse workers. Head-mounted-displays have increased in popularity among companies because of an increase in the field of e-commerce, therefore warehouse labour was an interesting area to review. The purpose of this project has been to investigate how head-mounted-displays can simplify warehouse work and to find an area where Tangar can be utilized. Tangar is an application to facilitate indoor navigation by helping users to reach points of interest. Through a mixed methodology approach that utilizes both quantitative and qualitative methods, a broad understanding in warehouse and inventory management have been established. The potentials of head-mounted display were evaluated using empirical and theoretical studies. Based on an early concept that was evaluated by a collaboration with a warehouse-solution company, factors that are of importance in warehouse management were identified. A decision to direct the project towards order picking was taken as it is a fundamental process within warehouse management. Three concepts were generated that harness the benefits of head-mounted-displays. With an informed decision the benefits for each of the concepts were compared with important parameters for a profitable warehouse management. It turned out that "Pick-by-Light", a common system in warehouse management, can be made virtual using head-mounted-displays. Since the system had never previously been operated virtually, an extensive study needed to be done in order to evaluate the viability in order-picking to propose a final concept. An experimental environment was set for the empirical studies, and two other common order picking systems were compared to the virtual Pick-by-Light system. Quantitative data in the form of time measurements from the order picking as well as picking errors and qualitative data from a NASA-TLX survey, was extracted from twelve users. A total of 360 samples from the quantitative study and 36 questionnaires from the qualitative study was then analysed. The result resembled those from similar studies with a conventional Pick-by-Light system. Thus, parallels were drawn that indicated that the virtual system had good potential to perform at least as well as a regular Pick-by-Light. A virtual Pick-by-Light system might be able to reduce implementation-, work- and operational costs as the use of material is replaced by a virtual product, and also no installation is required. With the combination of Tangar, there is also a potential that a virtual Pick-by-Light system could be more efficient and accurate. The disadvantages of the conventional Pick-by-Light system are also that confirmations are ineffective and that workers find it difficult to get an overview of pickplaces. Which can potentially be eliminated with the proposed concept. However, a new generation of hardware and further studies are required in order to establish a final concept. The Magic Leap One, which is the head-mounted-display used in the project, is new. Many problems regarding the display have been discovered during the project and affected the results of the user studies. Further studies need to be done with other displays to determine the validity of the results of this work. In summary, this work gives an introduction in how "Mixed-reality" can be used in warehouse management and recommendations for continued work. / Det här arbetet har undersökt hur huvudmonterade skärmar kan möjliggöra ett effektivare och bättre arbete för lager-personal. Huvudmonterade skärmar har ökat i popularitet bland företaget på grund av ökningen inom e-handel och därför var lagerarbete ett intressant område att undersöka. Syftet med det här projektet har varit att undersöka hur huvudmonterade skärmar fortsatt skulle kunna förenkla lagerarbete samt att ta hitta ett område där Tangar kan användas. Tangar är en applikation som förenklar inomhus navigering genom att leda användaren till valda intressepunkter. Genom en metodisk undersökning som utnyttjar både kvantitativa och kvalitativa metoder, har en bred bakgrund inom lagerhantering kunnat upprättas. Potentialen av att använda huvudmonterade skärmar har undersökts genom empiriska och teoretiska studier. Baserat på ett tidigt koncept som utvärderas genom ett samarbete med ett sakkunnigt företag, identifierades flertalet faktorer som är av vikt i lagerhantering. Ett beslut om att rikta projektet mot order-plockning togs då det är en fundamental process inom lagerabete. Tre koncept genererades som utnyttjar fördelarna med huvudmonterade skärmar. Genom att ta ett informativt beslut, kunde fördelarna för var och ett av koncepten jämföras med viktiga parametrar för ett lönsamt lagerarbete. Det visade sig att ”Pick-by-Light”, ett vanligt system inom lagerhantering, kunde göras virtuellt med hjälp av huvudmonterade skärmar. I och med att systemet tidigare aldrig utförts virtuellt, behövdes en omfattande studie göras för att evaluera dess potential inom order-plockning för att kunna föreslå ett slutgiltigt koncept. En experimentell miljö sattes upp som ram för de empiriska studierna och två andra vanliga order-plocknings system kunde jämföras mot det virtuella Pick-by-Light systemet. Kvantitativa data i form av orderplockningstider samt plock-fel och kvalitativa data från observationer samt en NASA-TLX enkät, kunde extraheras från tolv användare. Totalt kunde 360 stickprov från den kvantitative studien och 36 enkäter från den kvalitative studien därefter analyseras. Resultatet liknade det som observerats i liknande studier där ett vanligt Pick-by-Light system evaluerats. Därmed kunde paralleller dras som visade att det virtuella systemet hade god potential till att kunna prestera åtminstone lika bra som ett vanliga Pick-by-Light systemet och ett koncept togs fram för vidare utveckling. Ett virtuellt Pick-by-Light system skulle kunna reducera implementerings-, arbetes- samt driftkostnader i och med att materialåtgången ersätts av en virtuell produkt, samt att ingen installation krävs. I och med kombinationen av Tangar finns det även potential att konceptet är mer effektivt och exakt. De nackdelar med det traditionella Pick-by-Light systemet är också att plock-bekräftelser görs ineffektivt och att arbetare har svårt att få en överblick gällande plockställen. Vilket skulle kunna elimineras med det föreslagna konceptet. Dock krävs en ny generation hårdvara och vidare studier för att kunna fastställa ett slutgiltigt koncept. Magic Leap One, som är den huvudmonterade skärmen som används i projektet är väldigt ny. Många problem gällande displayen har upptäckts under projektet och påverkat resultatet av användarstudierna. Fortsatta studier skulle behöva göras med andra displayer för att fastställa validiteten av resultaten från det här arbetet. Sammanfattningsvis ger det här arbetet en introduktion om hur ”Mixed-reality” kan användas inom lagerhantering samt rekommendationer till fortsatt arbete.