A major disadvantage of shooting on green screen with CGI is that the director cannot see or interact with the virtual elements in the scene and thus get a realistic view of how the end result will look like, and also not being able to make changes as necessary by moving objects and actors around during the shooting session. This disadvantage can be mitigated by the use of previsualization where the director is given a rough take of what the final cut will look like by combining the filmed material with the virtual environment in real-time. This master’s thesis work aims to come up with solutions for improving the film studio Stiller Studios’s previsualization system. This involves a review and integration of game engines for previsualization in a motion control green screen studio, a camera calibration process with blur detection and automatic selection of images as well as research into camera tracking and depth compositing.

Previsualization is a class of techniques for creating approximate previews of a movie sequence in order to visualize a scene prior to shooting it on the set. Often these techniques are used to convey the artistic direction of the story in terms of cinematic elements, such as camera movement, angle, lighting, dialogue, and character motion. Essentially, a movie director uses previsualization (previs) to convey movie visuals as he sees them in his "minds-eye". Traditional methods for previs include hand-drawn sketches, Storyboards, scaled models, and photographs, which are created by artists to convey how a scene or character might look or move. A recent trend has been to use 3D graphics applications such as video game engines to perform previs, which is called 3D previs. This type of previs is generally used prior to shooting a scene in order to choreograph camera or character movements. To visualize a scene while being recorded on-set, directors and cinematographers use a technique called On-set previs, which provides a real-time view with little to no processing. Other types of previs, such as Technical previs, emphasize accurately capturing scene properties but lack any interactive manipulation and are usually employed by visual effects crews and not for cinematographers or directors. This dissertation's focus is on creating a new method for interactive visualization that will automatically capture the on-set lighting and provide interactive manipulation of cinematic elements to facilitate the movie maker's artistic expression, validate cinematic choices, and provide guidance to production crews. Our method will overcome the drawbacks of the all previous previs methods by combining photorealistic rendering with accurately captured scene details, which is interactively displayed on a mobile capture and rendering platform. This dissertation describes a new hardware and software previs framework that enables interactive visualization of on-set post-production elements. A three-tiered framework, which is the main contribution of this dissertation is; 1) a novel programmable camera architecture that provides programmability to low-level features and a visual programming interface, 2) new algorithms that analyzes and decomposes the scene photometrically, and 3) a previs interface that leverages the previous to perform interactive rendering and manipulation of the photometric and computer generated elements. For this dissertation we implemented a programmable camera with a novel visual programming interface. We developed the photometric theory and implementation of our novel relighting technique called Symmetric lighting, which can be used to relight a scene with multiple illuminants with respect to color, intensity and location on our programmable camera. We analyzed the performance of Symmetric lighting on synthetic and real scenes to evaluate the benefits and limitations with respect to the reflectance composition of the scene and the number and color of lights within the scene. We found that, since our method is based on a Lambertian reflectance assumption, our method works well under this assumption but that scenes with high amounts of specular reflections can have higher errors in terms of relighting accuracy and additional steps are required to mitigate this limitation. Also, scenes which contain lights whose colors are a too similar can lead to degenerate cases in terms of relighting. Despite these limitations, an important contribution of our work is that Symmetric lighting can also be leveraged as a solution for performing multi-illuminant white balancing and light color estimation within a scene with multiple illuminants without limits on the color range or number of lights. We compared our method to other white balance methods and show that our method is superior when at least one of the light colors is known a priori.

Previsualisation är en arbetsprocess som i ett tidigt stadie ger oss preliminära versioner av bilder eller sekvenser. Den är till för att kunna uppskatta produktionskraven, visualisera ideér och en gemensam vision. En teknisk previsualisation är en mer specifik term där teknisk exakthet är avgörande. En teknisk previs kan skapas i programmet FrameForge Pré-viz Studio som denna text redogör för. Previsualisationen som beskrivs i denna text lägger grund för diskussioner kring hur den kan användas på en filminspelning. I en jämförelse mellan den previsualiserade och filmade versionen av en filmscen kan vi se att tekniska skillnader förekommer dem sinsemellan. Resultatet tyder ändå på att de är tillräckligt lika varandra för att möjliggöra att planera, ta både tekniska och estetiska beslut inför inspelning.

Previsualization is a process within pre-production of filmmaking where filmmakers can visually plan specific scenes with camera works, lighting, character movements, etc. We developed and assess a prototype for previsualization in virtual reality for collaborative purposes where multiple filmmakers can be present in a virtual environment to share a creative work experience, remotely. The costs of computer graphics-based effects are substantial within film production, using previsualization, these scenes can be planned in detail to reduce the amount of work put on effects in the later production phase. By performing a within-group study on 20 filmmakers, our findings show that the use of virtual reality for distributed, collaborative previsualization processes is useful for real-life pre-production purposes. These results provide insights on how to best design collaborative, virtual reality-applications used for remote work, and highlights the pitfalls of certain design choices. / Previsualisering är en process inom filmskapande där filmskapare visuellt kan planera specifika scener med kamerafunktioner, belysning, karaktärsrörelser, etc. I denna uppsats utvärderas en prototyp för previsualisering i virtual reality för samarbete där flera filmskapare kan vara närvarande i en virtuell miljö för att dela ett kreativt arbete i realtid. Kostnaderna för datorgrafikbaserade effekter är betydande inom filmproduktion. I och med previsualisering kan dessa scener planeras i detalj för att minska mängden arbete som påverkar den senare produktionsfasen. Genom att utföra en studie med 20 filmskapare visar resultaten att användningen av virtual reality för distribuerade previsualiseringsprocesser för samarbete är användbart för verkliga scenarion. Dessa resultat ger insikter om hur man bäst utformar virtual reality-applikationer som kan användas för distribuerat arbete och lyfter fram fallgropar inom vissa designval.

No description available.

The execution of this thesis was based on one primare question and threesecondary questions: Primary question at issue: Is it possible to develop a system to previsualize computer generatedgraphics in real-time on a film set with available equipment on theuniversity? Secondary questions at issue: Is it possible to sync the camera feed with the motions of the HTC Vive to getan acceptable result in the game engine with existing equipment? Will the result be able to be integrated with the Computer Graphicsprogramme at LTU? How much access to control and manipulate the previsualized computergenerated graphics, while filming can be achieved? The purpose of this report is to try a method with the available equipment, toproduce a prototype of a tool to previsualize computer generated graphics inreal-time. The equipment used for the method was a computer, web camera, HTC Vive anda green screen. All the equipment was linked together with the game engineUnity. This resulted in a prototype of a tool that in real-time can mimic themotions of the camera, replace colored background and has the feature of realtimescene editing, as the previsualization runs. / Utförandet av detta examensarbete baserades på en primär frågeställning, samttre ytterligare sekundära frågor: Primär frågeställning: Kan man med hjälp av befintlig utrustning på universitetet ta fram enmodell för att previsualisera datorgenererad grafik i real-tid vid en inspelning? Sekundära frågeställningar: Kommer det gå att synka bild från webcamera med fångade rörelser frånHTC Vive och få ett godtagbart resultat från spelmotorn med befintligutrustning? Kan man använda resultatet som ett verktyg för utbildningen Datorgrafikpå LTU? Hur mycket kontroll för den förhandsvisade grafiken går att uppnå i realtid? Syftet med rapporten är att framställa och bepröva en prototyp av en produkt föratt kunna förhandsvisa datorgenererad grafik i real-tid, med befintlig utrustning. Metoden utövades med en dator, web kamera, HTC Vive och en green screen.Genom att använda spelmotorn Unity knöts utrustningen samman för attresultera i en prototyp som i real-tid fångar kamerans rörelser, ersätter färgadbakgrund och kan manipuleras direkt vid inspelning.

Pour respecter les droits d’auteur, la version électronique de cette thèse a été dépouillée de certains documents visuels. La version intégrale de la thèse a été déposée à la Division de la gestion des documents et des archives. / La prévisualisation cinématographique, c’est-à-dire la mise en forme visuelle du scénario visant à préciser le film à venir, constitue une pratique dont les origines remontent aux débuts du cinéma. À travers le prisme ludique, notre thèse propose que la récente transition des outils traditionnels de prévisualisation vers des dispositifs numériques apparentés aux technologies vidéoludiques transforme cette préparation visuelle en l’apparentant sur plusieurs points à un jeu, voire à un jouet. Nos recherches portent sur la démarche créative du réalisateur, et comment l’exercice de son imagination mène à la fabrication d’un simulacre du film par le biais d’une dynamique itérative et de différents supports propres à représenter l’œuvre envisagée. Cette démarche est étudiée principalement dans le cadre de longs métrages impliquant une part substantielle d’effets visuels, un corpus particulièrement susceptible d’impliquer des stratégies élaborées de préparation logistique ou créative. À la suite d’un résumé historique impliquant une typologie des formes de prévisualisation et l’identification de variables propices à orienter les analyses subséquentes, notre thèse présente les notions d’imagination et d’attitude ludique, et précise comment ces ancrages théoriques sont utilisés pour mieux définir les aspects de la conception du film qui s’apparente au jeu. Par la suite, et au moyen d’une étude de cas impliquant un jeu vidéo remédié en un outil de machinima, elle détaille l’élaboration de la double figure dite « jeu de construction/jouet à construire » propre à représenter la nature et la dynamique de la prévisualisation selon un point de vue ludique. Nos recherches examinent ensuite les différents contextes de la production cinématographique quant à leur pertinence à l’exercice de la préparation visuelle par le réalisateur, tout en déterminant les aspects de la fabrication du film qui échappent à une telle mise en forme, lorsque réalisée au moyen des dispositifs conventionnels. À la lumière de cet examen, notre thèse propose l’émergence d’un nouvel espace de jeu, déployé transversalement à la chaine de production traditionnelle, et susceptible de favoriser la pratique d’une prévisualisation d’inspiration ludique qui répond de manière satisfaisante aux principales lacunes préalablement relevées. / Film previsualization (or previs), that is to say the visual formatting of the scenario aimed at anticipating the film to come, constitutes a practice whose origins date back to early cinema. According to a lusory perspective, our thesis proposes that the recent transition from traditional previs tools to digital devices involving videogame technologies transforms visual preparation by making it more similar to a game, or eventually to a toy. Our thesis focuses on the creative approach of the director, more specifically the process of imagination and how it leads to the design of a simulacrum of the film through an iterative method while using a variety of media fit to represent the intended work. This approach takes place mainly in the context of feature films involving a substantial part of visual effects, a corpus particularly likely to involve elaborate strategies relating to logistical or creative preparation. Following a historical background, a typology of the main forms of previsualization and the identification of variables suited to guide subsequent analysis, both notions of imagination and lusory attitude are studied in order to better define the aspects of film creation that relate to play and game. Afterwards, our thesis details the twin figure of the “construction game” and the “toy to build” representing the nature and dynamics of previsualization from a lusory perspective. This will be achieved by means of a case study involving a videogame remedied as a machinima tool. Our thesis then examines the different contexts of film production as to their relevance to previsualization practices performed by the director, while exposing the aspects of filmmaking that elude such practices when carried out by means of traditional devices. In the light of this examination, our thesis suggests the emergence of a new context of play, deployed transversally to the traditional production chain, and likely to favor the practice of a lusory-inspired previsualization that addresses in a successful way some of the shortcomings previously listed.