This thesis explores techniques for synthesizing crowds from imagery. Synthetic photorealistic crowds are desirable for cinematic gaming, special effects and architectural visualization. While motion captured-based techniques for the animation and control of crowds have been well-studied in computer graphics, the resulting control rig sequences require a laborious model-based graphics pipeline to render photorealistic videos of crowds. Over the past ten years, data-driven techniques for rendering imagery of complex phenomena have become a popular alternative to model-based graphics. This popularity is due in large part to difficulties in constructing the sufficiently-detailed models that are required to achieve photorealism. A dynamic crowd of humans is an extremely challenging example of such phenomena. Example-based synthesis methods such as video textures are an appealing alternative, but current techniques are unable to handle new challenges posed by crowds. This thesis describes how to synthesize video-based crowds by explicitly segmenting pedestrians from input videos of natural crowds and optimally placing them into an output video while satisfying environmental constraints imposed by the scene. There are three key challenges. First, the crowd layout of segmented videos must satisfy constraints imposed by environmental and crowd obstacles. This thesis addresses four types of environmental constraints: (a) ground planes in the scene which are valid for crowd traversal, such as sidewalks, (b) spatial regions of these planes where crowds may enter and exit the scene, (c) static obstacles, such as mailboxes and walls of a building, and (d) dynamic obstacles such as individuals and groups of individuals. Second, pedestrians and groups of pedestrians should be segmented from the input video with no artifacts and minimal interaction time. This is challenging in real world scenes due to significant appearance changes while traveling through the scene. Third, segmented pedestrian videos may not have enough frames or the right shape to compose a path from an artist-defined entrance to exit. Plausible temporal transitions between segmented pedestrians are therefore needed but they are difficult to identify and synthesize due to complex self occlusions. We present a novel algorithm for composing video billboards, represented by crowd tubes, to form a crowd while avoiding collisions between static and dynamic obstacles. Crowd tubes are represented in the scene using a temporal sequence of circles planted in the calibrated ground plane. The approach consists of representing crowd tube samples and constraint violations with a conflict graph. The maximal independent set yields a dense crowd composition. We present a prototype system for the capture, analysis, synthesis and control of video-based crowds. Several results demonstrate the system's ability to generate videos of crowds which exhibit a variety of natural behaviors.

A thesis submitted to the Faculty of Humanities, University of the Witwatersrand, Johannesburg, in fulfilment of the requirements for the Degree of Master of Music Johannesburg 2016 / Recently, scholarly work in the field of film sound design has emphasised the crucial significance of sound in film. Writers such as Mark Underwood (2008), Larry Sider (2003) and Danijela Kulezic-Wilson (2008) have expressed the view that film sound design ought to be approached from a musical perspective substantiating this position through analytical discussions on the relatively musical use of sound design in scenes from films such as Alfred Hitchcock’s The Birds (1963) and Darren Aronofsky’s Requiem for a Dream (2000). Building upon the work of these and other scholars, this thesis investigates the varying ways in which music and foley sound design relate and interact within a film seeking to categorise with some specificity the various ways in which foley sound design can ascend beyond its ordinary remit and in so doing function in lieu of film music as well as in cooperation with it. I consider examples from, amongst others, The Godfather, parts one (1972) and two (1974) by Francis Ford Coppola and The Matrix (1999) by Ana and Lana Wachowski. In each chapter of part one of the thesis (which I call Take 1), I explore a particular aspect of the way in which foley makes known its capacity to function quasi-musically. Chapter one looks at Walter Murch’s concept of the metaphoric use of sound and how, through this technique, foley sound can be applied so as to fulfil roles more accustomed to film music in its stead. Chapter two details some of the ways in which film music and foley interact within a film. A crucial element of this discussion is the on going debate between scholars such as Michel Chion who disavow the existence of a soundtrack and others such as Rick Altman who contradict Chion on this matter. Chapter three looks at how otherworldly diegetic contexts help to encourage creativity in designing and applying foley sounds so as to further enhance its pre-discussed ability to act in film music’s stead while chapter four focuses on the voice as the soloist within the melee of sounds that constitute the film soundtrack. Part two (or Take 2) of the thesis consists of compositions written in response to some of the theories and concepts explored in the first part of the thesis including a ‘dramatic string quartet’ in which I attempt to realise in a musical composition some of the ideas discussed in all four of the chapters. I conclude the thesis by reflecting on the main insights uncovered throughout the thesis in addition to reflecting on the process of composing the pieces in relation to the relative success of the performances thereof. / GR2017