No description available.

A digital twin project is launched by the Integrative Systems Biology (ISB) research team and led by Gunnar Cedersund. The digital twin project is based on biological models of physiological processes, that can interact and be tailored for a specific person. However, the digital twin can currently not analyse movements of a human body. In this master thesis, the aim was to create a useful pipeline that expands the digital twin project with biomechanical modelling of movements, and also visualises the twins by letting the concept take human form. The biomechanical analysis was done in the software OpenSim, where the movements of a motion captured dance were analysed. To generate a simulation of the motion with an acceptable error in a reasonable computation time, a musculoskeletal model was created in OpenSim and scaled to best fit the anthropometry of the dancer. Then, the motion was estimated with an optimised procedure by using the scaled model and the motion capture data. The Root-Mean Squared (RMS) error of the estimated dance with accuracy 10-6 was 2.39 cm. In this thesis, the torque in each joint for the dance motion was estimated. The loads and muscle forces can also be estimated in OpenSim. One useful application is for calculating energy consumption. In order to calculate muscle forces, external forces needs to be measured while recording motion capture. This is something that will be focused on in the future, when continuing with this project. The visualisation of the digital twins were made in Unreal Engine with MetaHuman avatars. The dance recorded in motion capture, were applied to the avatars in order to make them dance. The recorded dance was the same for both OpenSim and Unreal Engine, so the dance could both be viewed and analysed. In conclusion, we have added a new feature to the existing digital twin technology: movements and simulation of the musculoskeletal system. This new feature can in the future be used for both medical purposes such as movement-based rehabilitation as well as for integration into dance performances.

The boundary between offline and online worlds is rapidly shrinking with improvements in technology. Virtual Youtubers (VTubers) have emerged in recent years as a new Twitch streaming phenomenon. Replacing the use of webcams, VTubers obscure their true physical appearance and instead choose to represent themselves through a fictional character. This character is most often controlled through facial tracking, motion capture, and additional software tools. While previous livestreaming research has focused on why people watch others play video games, or what an audience wants from the streamer they watch, there is very little current research available on VTubers. Current studies are scattered, and do not attempt to deeply engage with VTubers on a personal level to explore their thoughts and motivations. In turn, this study aims to answer the following questions: 1) “How does livestreaming as a VTuber allow people to explore and/or express their identity?”; 2) “What makes VTubing unique when compared to standard facecam streaming?”. 10 different VTubers were interviewed using semi-structured interviews, and their responses were analyzed by a framework centering Erving Goffman’s (1959) theory of self-presentation. Theories of online identity, imagined audience, online disinhibition, and parasocial interaction were also utilized to support the analysis. The results suggest that VTubing is a unique form of livestreaming which allows its users to overcome personal insecurities, explore different methods of self-presentation, and to affirm the identities of members of queer and ostracized communities.

Entrainment is the rhythmic coordination of movement with a signal or other person. Most studies on entrainment have looked at synchronization with auditory or visual signals, whereas much less is known about how entrainment emerges mutually between individuals, especially when they are in physical contact with one another. In this dissertation, I empirically explored dance partnering as an ecological model for understanding interpersonal entrainment through haptic interaction. I began by performing a statistical meta-analysis of functional neuroimaging articles devoted to the most common experimental paradigm for entrainment, namely externally-paced finger tapping to an acoustic rhythmic stimulus (Chapter 2). The results showed that the cerebellar vermis was a strong neural marker of entrainment, as it was more activated by externally-paced tapping than by self-paced tapping, whereas the basal ganglia was activated by both types of rhythmic movements. Next, I used functional magnetic resonance imaging (fMRI) with a group of participants trained at couple dancing in order to explore the neural basis of haptic mutual entrainment, with a focus on the dynamics of leading and following (Chapter 3). While mutual interaction overall engaged brain networks involved in somatosensation, internal-body sensation and social cognition, leading showed enhanced activity principally in areas for motor control and self-initiated action, whereas following showed enhanced activity mainly in sensory and social-cognition areas. Finally, I used 3D motion capture to explore multisensory coupling for mutual entrainment at the group level during folk dancing (Chapter 4). The results showed that dancers relied most extensively on haptic coupling to synchronize as a group, whereas auditory and visual coupling were dependent on the spatiotemporal context. These studies advance our understanding of the neural and behavioural mechanisms underlying joint actions in which entrainment emerges mutually through haptic interaction. / Thesis / Doctor of Philosophy (PhD) / Entrainment is the rhythmic coordination of movement with a signal or other person. Most studies on entrainment have looked at synchronization with auditory or visual signals, whereas much less is known about how entrainment emerges mutually between individuals, especially when they are in physical contact with one another. I began my research by performing a statistical analysis of the literature examining the brain basis of synchronization with auditory signals, identifying a key brain area for entrainment. Next, using a group of participants trained at couple dancing, I explored the brain areas engaged when two individuals in physical contact improvised movement together, focusing on who is leading or following the interaction. Finally, I explored how folk dancers use multiple sensory signals (auditory, visual and tactile) to synchronize as a group. These studies advance our understanding of the neural and behavioural mechanisms by which people mutually entrain through physical interaction.

Bats have many impressive flight characteristics such as the ability to rapidly change direction, carry substantial loads, and maintain good flight efficiency. For several years, researchers have been working towards an understanding of the specific aerodynamic phenomena which relate the unique wing structure of bats to their flight abilities. Computational fluid dynamics, a powerful tool used extensively across aerospace research, has led to substantial progress in the understanding of insect flight. However, due to technical challenges, numerical simulation has seen limited use in bat flight research. For this research, we develop, validate, and apply computational modeling techniques to three modes of bat flight: straight flight, sweeping turn, and U-turn maneuver. 3D kinematic data collection was achieved using a 28 camera multi-perspective optical motion capture system. The calibration of the cameras was conducted using a multi-camera self-calibration method. Point correspondences between cameras and frames was achieved using a human-supervised software package developed for this project. After the collection of kinematic data, we carried out aerodynamic flow simulations using the incompressible Navier-Stokes solver, GenIDLEST. The immersed boundary method (IBM) was used to impose moving boundary conditions representing the wing kinematics. Validation of the computational model was preformed through a grid independence study as well as careful evaluation of other relevant simulation parameters. Verification of the model was performed by comparing simulated aerodynamic loads to the expected loads based on the observed flight trajectories. Additionally, we established that we had a sufficient resolution of the wing kinematics, by calculating the sensitivity of the simulation results to the number of kinematic markers used during motion capture. For this study, three particular flights are analyzed—a straight and level flight, a sweeping turn, and a sharp 180 degree turn. During straight flight, typical flight velocities observed in the flight tunnel were 2-3 m/s resulting in a Reynolds number of about 12,000. Lift generation occurred almost exclusively during the downstroke, and peaks mid-downstroke. At the beginning of each downstroke, the effective angle of attack of the wings transitions from negative to positive and a leading edge vortex (LEV) quickly forms. LEVs are known to augment lift generation in flapping flight and allow lift to remain high at large angles of attack. During the end of each downstroke, the LEVs break up and lift drops substantially. As the wingbeat cycle transitions from downstroke to upstroke, the wings rotate such that the wing chordline is vertical as the wing moves upward. This wing rotation is critical for mitigating negative lift during the upstroke. Many of the basic flight mechanisms used for straight flight—i.e. LEV formation, wing rotation during upstrokes—were also observed during the sweeping turn. In addition, asymmetries in the wing kinematics and consequently the aerodynamics were observed. Early in the turn, the bank angle was low and elevated levels of thrust were generated by the outer wing during both the upstroke and downstroke causing a yaw moment. As the bat moved towards the middle of the turn, the bank angle increased to 20-25 degrees. Although the bank angle remained nominally constant during the middle and later portion of the turn, there was variation within each wingbeat cycle. Specifically, the bank angle dropped during each upstroke and subsequently was recovered during each downstroke as a consequence of elevated lift on the outer wing. Banking served to redirect the net force vector laterally causing a radial, centripetal force. Considering the mass of the bat, the nominal flight velocity, and the radius of curvature, the magnitude of the radial force fully explained the expected centripetal acceleration during the middle and later portion of the turn. Over the entire turn, yaw was found to be important in initiating the turn while banking was more important during the middle part of the turn. Over the course of 5 wingbeat cycles, the change in bearing angle (direction of flight) was about 45 degrees. Analysis of the U-turn flight showed many of the same characteristics as were observed during the sweeping turn, as well as a few key differences. The bat's ability to rotate its body rapidly appears to be more limited than its ability to change its trajectory. For this reason, the yaw rotation began about one to two cycles before the rapid bearing angle change and was stretched out over several wingbeat cycles. At the apex of the U-turn, the bat combined a high roll angle with a low flight velocity magnitude to very rapidly redirect its bearing direction and negotiate a low radius of curvature flight trajectory. Increases in roll angle occurred almost exclusively during the downstrokes, while both the upstroke and downstroke were active in generating yaw. Elevated thrust on the left outer wing during the end of the upstroke was observed throughout the flight, and elevated drag on the right inside wing did not appear to have an impact on the turn. We hope that this project motivates and facilitates further computational analysis into bat flight aerodynamics. Additionally, the data and findings will be useful for applications such as the design of bioinspired MAVs or flexible membrane energy harvesting technology. / Doctor of Philosophy / Bats have many impressive flight characteristics such as the ability to rapidly change direction, carry substantial loads, and maintain good flight efficiency. A better understanding of the physics of how bats fly can help scientists and engineers build more maneuverable, quieter, and more efficient bioinspired micro air vehicles. This engineering approach leverages the incredible capabilities observed in nature, but requires detailed knowledge of the animal as a prerequisite. Computational fluid dynamics, a powerful tool used extensively across aerospace research, has led to substantial progress in the understanding of animal flight broadly. However, due to technical challenges, numerical simulation has seen limited use in bat flight research. For this research, we develop, validate, and apply computer modeling techniques to the investigation of bat flight aerodynamics. Three particular modes of flight were analyzed—a straight and level flight, a sweeping turn, and a sharp 180 degree turn. During straight flight, typical flight velocities observed in the flight tunnel were 2-3 m/s. Lift generation, the force keeping the bat aloft, occurred almost exclusively during the downstroke, and peaks mid-downstroke. As the wing flap transitions from downstroke to upstroke, the wings rotate such that the wing is vertical as it moves upward. This wing rotation is critical for maximizing lift force during flight. During the sweeping turn, asymmetries in the wing kinematics and consequently the aerodynamics were observed. Early in the turn, the bank angle was low and elevated levels of thrust were generated by the outer wing during both the upstroke and downstroke causing rotation of the bat. As the bat moved towards the middle of the turn, the bank angle increased to 20-25 degrees. Banking served to redirect the net force vector laterally causing a turning force. Over the course of 5 wingbeat cycles, the change in direction of flight was about 45 degrees. Analysis of the U-turn flight showed many of the same characteristics as were observed during the sweeping turn, as well as a few key differences. At the apex of the U-turn, the bat combined a high roll angle with a low flight velocity magnitude to very rapidly redirect its bearing direction and negotiate a low radius of curvature flight trajectory. We hope that this project motivates and facilitates further computer simulations studying bat flight aerodynamics. Additionally, the data and findings will be useful for applications such as the design of bioinspired MAVs or flexible membrane energy harvesting technology.

In recent years, the design, fabrication, and control of robotic systems inspired by biology has gained renewed attention due to the potential improvements in efficiency, maneuverability, and adaptability with which animals interact with their environments. Motion studies of biological systems such as humans, fish, insects, birds and bats are often used as a basis for robotic system design. Often, these studies are conducted by recording natural motions of the system of interest using a few high-resolution, high-speed cameras. Such equipment enables the use of standard methods for corresponding features and producing three-dimensional reconstructions of motion. These studies are then interpreted by a designer for kinematic, dynamic, and control systems design of a robotic system. This methodology generates impressive robotic systems which imitate their biological counter parts. However, the equipment used to study motion is expensive and designer interpretation of kinematics data requires substantial time and talent, can be difficult to identify correctly, and often yields kinematic inconsistencies between the robot and biology. To remedy these issues, this dissertation leverages the use of low-cost, low-speed, low-resolution cameras for tracking bat flight and presents a methodology for automatically learning physical geometry which restricts robotic joints to a motion submanifold identified from motion capture data. To this end, we present a spatially recursive state estimator which incorporates inboard state correction for producing accurate state estimates of bat flight. Using these state estimates, we construct a Gaussian process dynamic model (GPDM) of bat flight which is the first nonlinear dimensionality reduction of flapping flight in bats. Additionally, we formulate a novel method for learning robotic joint geometry directly from the experimental observations. To do this, we leverage recent developments in learning theory which derive analytical-empirical potential energy fields for identifying an underlying motion submanifold. We use these energy fields to optimize a compliant structure around a single degree-of-freedom elbow joint and to design rigid structures around spherical joints for an entire bat wing. Validation experiments show that the learned joint geometry restricts the motion of the joints to those observed during experiment. / Ph. D. / In recent years, robots modeled after biological systems have become increasingly prevalent. Such robots are often designed based on motion capture experiments of the animal they aim to imitate. The motion studies are typically conducted using commercial motion capture systems such as ViconTM or OptiTrackTM or a few high-speed, high-resolution cameras such as those marketed by PhotronTM or PhantomTM. These systems allow for automated processing of video sequences into three-dimensional reconstructions of the biological motion using standard image processing and state estimation techniques. The motion data is then used to drive robotic system designs such as the SonyTM AiboTM dog and the Boston Dynamics Atlas humanoid robot. While the motion capture data forms a basis for these impressive robots, the progression from data to robotic system is neither algorithmic nor rigorous and requires substantial interpretation by a human. In contrast, this dissertation presents a novel experimental and computational framework which uses low-speed, low-resolution cameras for capturing the complex motion of bats in flight and introduces a methodology which uses the motion capture data to directly design geometry which restricts the motion of joints to the motions observed in experiment. The advantage of our method is that the designer only needs to specify a general joint geometry such as a ball or pin joint, and geometry which restricts the motion is automatically identified. To do this, we learn an energy field over the set of kinematic configurations observed during experiment. This energy field “pushes” system trajectories towards those experimentally observed trajectories. We then learn compliant or rigid geometry which approximates this energy field to physically restrict the range of motion of the joint. We validate our method by fabricating joint geometry designed using both these approaches and present experiments which confirm that the reachable set of the joint is approximately the same as the set of configurations observed during experiments.

With the maturity of conventional industrial robots, there has been increasing interest in designing robots that emulate realistic animal motions. This discipline requires careful and systematic investigation of a wide range of animal motions from biped, to quadruped, and even to serpentine motion of centipedes, millipedes, and snakes. Collecting optical motion capture data of such complex animal motions can be complicated for several reasons. Often there is the need to use many high-quality cameras for detailed subject tracking, and self-occlusion, loss of focus, and contrast variations challenge any imaging experiment. The problem of self-occlusion is especially pronounced for animals. In this thesis, we walk through the process of collecting motion capture data of a running lizard. In our collected raw video footage, it is difficult to make temporal correspondences using interpolation methods because of prolonged blurriness, occlusion, or the limited field of vision of our cameras. To work around this, we first make a model data set by making our best guess of the points' locations through these corruptions. Then, we randomly eclipse the data, use Gappy POD to repair the data and then see how closely it resembles the initial set, culminating in a test case where we simulate the actual corruptions we see in the raw video footage. / Master of Science / There has been increasing interest over the past few years in designing robots that emulate realistic animal motions. To make these designs as accurate as possible requires thorough analysis of animal motion. This is done by recording video and then converting it into numerical data, which can be analyzed in a rigorous way. But this conversion cannot be made when the raw video footage is ambiguous, for instance, when the footage is blurry, the shot is too dark or too light, the subject (or parts of the subject) are out of view of the camera, etc. In this thesis, we walk through the process of collecting video footage of a lizard running and then converting it into data. Ambiguities in the video footage result in an incomplete translation into numerical data and we use a mathematical technique called the Gappy Proper Orthogonal Decomposition to fill in this incompleteness in an intelligible way. And in the process, we lay your hands on the fundamental drivers of the animal’s motion.

No / We introduce a new "marker-based" model for use in capturing equine movement. This model is informed by a sound biomechanical study of the animal and can be deployed in the pursuit of many undertakings. Unlike many other approaches, our method provides a high level of automation and hides the intricate biomechanical knowledge required to produce realistic results. Due to this approach, it is possible to acquire solved data with minimal manual intervention even in real-time conditions. The approach introduced can be replicated for the production of many other animals. The model is first informed by the veterinary world through studies of the subject's anatomy. Second, further medical studies aimed at understanding and addressing surface processes, inform model creation. The latter studies address items such as skin sliding. If not otherwise corrected these processes may hinder marker based capture. The resultant model has been tested in feasibility studies for practicality and subject acceptance during production. Data is provided for scrutiny along with the subject digitally captured through a variety of methods. The digital subject in mesh form as well as the motion capture model aid in comparison and show the level of accurateness achieved. The video reference and digital renders provide an insight into the level of realism achieved.

Special effects (SFX) for film, television, and any type of video are usually digital, practical, or a mix of both. Animatronics is the use of robotics to mimic living movements, which is often used in the SFX industry. Currently, there is a gap between practical and digital effects, which means that post-production of practical effects leads to an inefficient workload. A robot is recorded on video, and if any digital enhancements or corrections are needed, the digital artists must start from nothing but the recorded video. Since the practical effects – controlling the animatronic- are using RC signals to manipulate servomotors, there is movement information that could be used to “record” movement into a digital space. This thesis project aims to create a way to bridge the gap between digital and physical by translating RC signals into digital animation. The RC signals used in this project were sent via a Taranis remote control, mixed using Open-TX software, and then translated into a digital animation rig in the 3D software Blender.

I describe and reflect on the creation of my exhibition Making Space for Daydreams, containing two large sculptures and a collection of conversational audio recordings. The sculptural artworks utilize interdisciplinary techniques and both digital fabrication and handcrafted approaches, including: woodworking, hand-cut paper, laser-cut illustration, motion capture, projection mapping and pencil drawings. The exhibition explores themes related to daydreaming, and the value of recognizing and building spaces that help make daydreaming possible. In addition, the work explores presence, memory, and inner child relationships as way to understand and frame my own relationship to daydreaming. By engaging in an act of imaginative play, I collaborated with my inner child and benefited from a strengthened sense of self-identity. Through acts of obscuring in the artwork I create allowances for imagination and invite the viewer to reflect on their own experiences and daydreams. / Master of Fine Arts / I describe and reflect on the creation of my exhibition Making Space for Daydreams, containing two large sculptures and a collection of conversational audio recordings. The exhibition explores themes related to daydreaming and the value of recognizing and building spaces that help make daydreaming possible. By engaging in an act of imaginative play, I collaborated with my inner child and benefited from a strengthened sense of self-identity.