Dissertation (Ph.D.)--Auburn University, 2006. / Abstract. Vita. Includes bibliographic references.

A new technique, called nanoinjection, is being developed to insert foreign DNA into a living cell. Such DNA transfection is commonly used to create transgenic organisms vital to the study of genetics, immunology, and many other biological sciences. In nanoinjection, DNA, which has a net negative charge, is electrically attracted to a micromachined lance. The lance then pierces the cell membranes, and the voltage on the lance is reversed, repelling the DNA into the cell. It is shown that DNA motion is strongly correlated to ion transport through a process called electrophoresis. Gel electrophoresis is used to move DNA using an electric field through a gel matrix (electrolytic solution). Understanding and using electrophoretic principals, a mathematical model was created to predict the motion (trajectory) of DNA particles as they are attracted to and repulsed from the nanoinjector lance. This work describes the protocol and presents the results for DNA motion experiments using fabricated gel electrophoresis devices. Electrophoretic systems commonly use metal electrodes in their construction. This work explores and reports the differences in electrophoretic motion of DNA (decomposition voltage, electrical field, etc.) when one electrode is constructed from a semiconductor, silicon rather than metal. Experimental results are used to update and validate the mathematical model to reflect the differences in material selection. Accurately predicting DNA motion is crucial for nanoinjection. The mathematical model allows investigation of the attraction/repulsion process by varying specific parameters. Result show that the ground electrode placement, lance orientation and lance penetration significantly affect attraction or repulsion efficiency while the gap, lance direction, lance tip width, lance tip half angle and lance tip height do not. It is also shown that the electric field around the lance is sufficient to cause localized electroporation of cell membranes, which may significantly improve the efficiency of transport.

Over the past decade, there has been an explosion in the volume of video data, including internet videos and surveillance camera footage. These videos often feature extended durations with unedited content, predominantly filled with background clutter, while the relevant activities of interest occupy only a small portion of the footage. Consequently, there is a compelling need for advanced processing techniques to automatically analyze this vast reservoir of video data, specifically with the goal of identifying the segments that contain the events of interest. Given that humans are the primary subjects in these videos, comprehending human activities plays a pivotal role in automated video analysis. This thesis seeks to tackle the challenge of detecting human activities from untrimmed videos, aiming to classify and pinpoint these activities both in their spatial and temporal dimensions. To achieve this, we propose a modular approach. We begin by developing a temporal activity detection framework, and then progressively extend the framework to support activity detection in the spatio-temporal dimension. To perform temporal activity detection, we introduce an end-to-end trainable deep learning model leveraging 3D convolutions. Additionally, we propose a novel and adaptable fusion strategy to combine both the appearance and motion information extracted from a video, using RGB and optical flow frames. Importantly, we incorporate the learning of this fusion strategy into the activity detection framework. Building upon the temporal activity detection framework, we extend it by incorporating a spatial localization module to enable activity detection both in space and time in a holistic end-to-end manner. To accomplish this, we leverage shared spatio-temporal feature maps to jointly optimize both spatial and temporal localization of activities, thus making the entire pipeline more effective and efficient. Finally, we introduce several novel techniques for modeling actor motion, specifically designed for efficient activity recognition. This is achieved by harnessing 2D pose information extracted from video frames and then representing human motion through bone movement, bone orientation, and body joint positions. Our experimental evaluations, conducted using benchmark datasets, showcase the effectiveness of the proposed temporal and spatio-temporal activity detection methods when compared to the current state-of-the-art methods. Moreover, the proposed motion representations excel in both performance and computational efficiency. Ultimately, this research shall pave the way forward towards imbuing computers with social visual intelligence, enabling them to comprehend human activities in any given time and space, opening up exciting possibilities for the future.

Single view reconstruction is fundamentally an under-constrained problem. We aim to develop new approaches to model human face and motion with model priors that restrict the space of possible solutions. First, we develop a novel approach to recover the 3D shape from a single view image under challenging conditions, such as large variations in illumination and pose. The problem is addressed by employing the techniques of non-linear manifold embedding and alignment. Specifically, the local image models for each patch of facial images and the local surface models for each patch of 3D shape are learned using a non-linear dimensionality reduction technique, and the correspondences between these local models are then learned by a manifold alignment method. Local models successfully remove the dependency of large training databases for human face modeling. By combining the local shapes, the global shape of a face can be reconstructed directly from a single linear system of equations via least square. Unfortunately, this learning-based approach cannot be successfully applied to the problem of human motion modeling due to the internal and external variations in single view video-based marker-less motion capture. Therefore, we introduce a new model-based approach for capturing human motion using a stream of depth images from a single depth sensor. While a depth sensor provides metric 3D information, using a single sensor, instead of a camera array, results in a view-dependent and incomplete measurement of object motion. We develop a novel two-stage template fitting algorithm that is invariant to subject size and view-point variations, and robust to occlusions. Starting from a known pose, our algorithm first estimates a body configuration through temporal registration, which is used to search the template motion database for a best match. The best match body configuration as well as its corresponding surface mesh model are deformed to fit the input depth map, filling in the part that is occluded from the input and compensating for differences in pose and body-size between the input image and the template. Our approach does not require any makers, user-interaction, or appearance-based tracking. Experiments show that our approaches can achieve good modeling results for human face and motion, and are capable of dealing with variety of challenges in single view reconstruction, e.g., occlusion.

The ability to learn is essential for robots to function and perform services within a dynamic human environment. Robot programming by demonstration facilitates learning through a human teacher without the need to develop new code for each task that the robot performs. In order for learning to be generalizable, the robot needs to be able to grasp the underlying structure of the task being learned. This requires appropriate knowledge abstraction and representation. The goal of this thesis is to develop a learning by imitation system that abstracts knowledge of human demonstrations of a task and represents the abstracted knowledge in a hierarchical framework. The learning by imitation system is capable of performing both action and object recognition based on video stream data at the lower level of the hierarchy, while the sequence of actions and object states observed is reconstructed at the higher level of the hierarchy in order to form a coherent representation of the task. Furthermore, error recovery capabilities are included in the learning by imitation system to improve robustness to unexpected situations during task execution. The first part of the thesis focuses on motion learning to allow the robot to both recognize the actions for task representation at the higher level of the hierarchy and to perform the actions to imitate the task. In order to efficiently learn actions, the actions are segmented into meaningful atomic units called motion primitives. These motion primitives are then modeled using dynamic movement primitives (DMPs), a dynamical system model that can robustly generate motion trajectories to arbitrary goal positions while maintaining the overall shape of the demonstrated motion trajectory. The DMPs also contain weight parameters that are reflective of the shape of the motion trajectory. These weight parameters are clustered using affinity propagation (AP), an efficient exemplar clustering algorithm, in order to determine groups of similar motion primitives and thus, performing motion recognition. The approach of DMPs combined with APs was experimentally verified on two separate motion data sets for its ability to recognize and generate motion primitives. The second part of the thesis outlines how the task representation is created and used for imitating observed tasks. This includes object and object state recognition using simple computer vision techniques as well as the automatic construction of a Petri net (PN) model to describe an observed task. Tasks are composed of a sequence of actions that have specific pre-conditions, i.e. object states required before the action can be performed, and post-conditions, i.e. object states that result from the action. The PNs inherently encode pre-conditions and post-conditions of a particular event, i.e. action, and can model tasks as a coherent sequence of actions and object states. In addition, PNs are very flexible in modeling a variety of tasks including tasks that involve both sequential and parallel components. The automatic PN creation process has been tested on both a sequential two block stacking task and a three block stacking task involving both sequential and parallel components. The PN provides a meaningful representation of the observed tasks that can be used by a robot to imitate the tasks. Lastly, error recovery capabilities are added to the learning by imitation system in order to allow the robot to readjust the sequence of actions needed during task execution. The error recovery component is able to deal with two types of errors: unexpected, but known situations and unexpected, unknown situations. In the case of unexpected, but known situations, the learning system is able to search through the PN to identify the known situation and the actions needed to complete the task. This ability is useful not only for error recovery from known situations, but also for human robot collaboration, where the human unexpectedly helps to complete part of the task. In the case of situations that are both unexpected and unknown, the robot will prompt the human demonstrator to teach how to recover from the error to a known state. By observing the error recovery procedure and automatically extending the PN with the error recovery information, the situation encountered becomes part of the known situations and the robot is able to autonomously recover from the error in the future. This error recovery approach was tested successfully on errors encountered during the three block stacking task.

The ability to learn is essential for robots to function and perform services within a dynamic human environment. Robot programming by demonstration facilitates learning through a human teacher without the need to develop new code for each task that the robot performs. In order for learning to be generalizable, the robot needs to be able to grasp the underlying structure of the task being learned. This requires appropriate knowledge abstraction and representation. The goal of this thesis is to develop a learning by imitation system that abstracts knowledge of human demonstrations of a task and represents the abstracted knowledge in a hierarchical framework. The learning by imitation system is capable of performing both action and object recognition based on video stream data at the lower level of the hierarchy, while the sequence of actions and object states observed is reconstructed at the higher level of the hierarchy in order to form a coherent representation of the task. Furthermore, error recovery capabilities are included in the learning by imitation system to improve robustness to unexpected situations during task execution. The first part of the thesis focuses on motion learning to allow the robot to both recognize the actions for task representation at the higher level of the hierarchy and to perform the actions to imitate the task. In order to efficiently learn actions, the actions are segmented into meaningful atomic units called motion primitives. These motion primitives are then modeled using dynamic movement primitives (DMPs), a dynamical system model that can robustly generate motion trajectories to arbitrary goal positions while maintaining the overall shape of the demonstrated motion trajectory. The DMPs also contain weight parameters that are reflective of the shape of the motion trajectory. These weight parameters are clustered using affinity propagation (AP), an efficient exemplar clustering algorithm, in order to determine groups of similar motion primitives and thus, performing motion recognition. The approach of DMPs combined with APs was experimentally verified on two separate motion data sets for its ability to recognize and generate motion primitives. The second part of the thesis outlines how the task representation is created and used for imitating observed tasks. This includes object and object state recognition using simple computer vision techniques as well as the automatic construction of a Petri net (PN) model to describe an observed task. Tasks are composed of a sequence of actions that have specific pre-conditions, i.e. object states required before the action can be performed, and post-conditions, i.e. object states that result from the action. The PNs inherently encode pre-conditions and post-conditions of a particular event, i.e. action, and can model tasks as a coherent sequence of actions and object states. In addition, PNs are very flexible in modeling a variety of tasks including tasks that involve both sequential and parallel components. The automatic PN creation process has been tested on both a sequential two block stacking task and a three block stacking task involving both sequential and parallel components. The PN provides a meaningful representation of the observed tasks that can be used by a robot to imitate the tasks. Lastly, error recovery capabilities are added to the learning by imitation system in order to allow the robot to readjust the sequence of actions needed during task execution. The error recovery component is able to deal with two types of errors: unexpected, but known situations and unexpected, unknown situations. In the case of unexpected, but known situations, the learning system is able to search through the PN to identify the known situation and the actions needed to complete the task. This ability is useful not only for error recovery from known situations, but also for human robot collaboration, where the human unexpectedly helps to complete part of the task. In the case of situations that are both unexpected and unknown, the robot will prompt the human demonstrator to teach how to recover from the error to a known state. By observing the error recovery procedure and automatically extending the PN with the error recovery information, the situation encountered becomes part of the known situations and the robot is able to autonomously recover from the error in the future. This error recovery approach was tested successfully on errors encountered during the three block stacking task.

Submitted by Renata Lopes (renatasil82@gmail.com) on 2017-03-02T19:31:32Z No. of bitstreams: 1 virginiafernandesmota.pdf: 2597727 bytes, checksum: df1d36b8c756398774e8649591f66a32 (MD5) / Approved for entry into archive by Adriana Oliveira (adriana.oliveira@ufjf.edu.br) on 2017-03-06T20:02:23Z (GMT) No. of bitstreams: 1 virginiafernandesmota.pdf: 2597727 bytes, checksum: df1d36b8c756398774e8649591f66a32 (MD5) / Made available in DSpace on 2017-03-06T20:02:23Z (GMT). No. of bitstreams: 1 virginiafernandesmota.pdf: 2597727 bytes, checksum: df1d36b8c756398774e8649591f66a32 (MD5) Previous issue date: 2011-02-28 / CAPES - Coordenação de Aperfeiçoamento de Pessoal de Nível Superior / Movimento é uma das características fundamentais que refletem a informação semântica em vídeos. Uma das técnicas de estimativa do movimento é o cálculo do fluxo óptico. Este é uma representação 2D (bidimensional) das velocidades aparentes de uma sequência de quadros (frames) adjacentes, ou seja, a projeção 2D do movimento 3D (tridimensional) projetado na câmera. Neste trabalho é proposto um descritor global de movimento baseado no tensor de orientação. O mesmo é formado à partir dos coeficientes dos polinômios de Legendre calculados para cada quadro de um vídeo. Os coeficientes são encontrados através da projeção do fluxo óptico nos polinômios de Legendre, obtendo-se uma representação polinomial do movimento. O descritor tensorial criado é avaliado classificando-se a base de vídeos KTH com um classificador SVM (máquina de vetor de suporte). É possível concluir que a precisão da abordagem deste trabalho supera às encontradas pelos descritores globais encontrados na literatura. / Motion is one of the main characteristics that describe the semantic information of videos. One of the techniques of motion estimation is the extraction of optical flow. The optical flow is a bidimensional representation of velocities in a sequence of adjacent frames, in other words, is the 2D projection of the 3D motion projected on the camera. In this work it is proposed a global video descriptor based on orientation tensor. This descriptor is composed by coefficients of Legendre polynomials calculated for each video frame. The coefficients are found though the projection of the optical flow on Legendre polynomials, obtaining a polynomial representation of the motion. The tensorial descriptor created is evaluated by a classification of the KTH video database with a SVM (support vector machine) classifier. Results show that the precision of our approach is greater than those obtained by global descriptors in the literature.

Dans cette thèse, nous nous intéressons à la modélisation de mouvements de foules. Nous proposons un modèle microscopique basé sur la théorie des jeux. Chaque individu a une certaine vitesse souhaitée, celle qu'il adopterait en l'absence des autres. Une personne est influencée par certains de ses voisins, pratiquement ceux qu'elle voit devant elle. Une vitesse réelle est considérée comme possible si elle réalise un équilibre de Nash instantané: chaque individu fait son mieux par rapport à un objectif personnel (vitesse souhaitée), en tenant compte du comportement des voisins qui l'influencent. Nous abordons des questions relatives à la modélisation ainsi que les aspects théoriques du problème dans diverses situations, en particulier dans le cas où chaque individu est influencé par tous les autres, et le cas où les relations d'influence entre les individus présentent une structure hiérarchique. Un schéma numérique est développé pour résoudre le problème dans le second cas (modèle hiérarchique) et des simulations numériques sont proposées pour illustrer le comportement du modèle. Les résultats numériques sont confrontés avec des expériences réelles de mouvements de foules pour montrer la capacité du modèle à reproduire certains effets.Nous proposons une version macroscopique du modèle hiérarchique en utilisant les mêmes principes de modélisation au niveau macroscopique, et nous présentons une étude préliminaire des difficultés posées par cette approche.La dernière problématique qu'on aborde dans cette thèse est liée aux cadres flot gradient dans les espaces de Wasserstein aux niveaux continu et discret. Il est connu que l'équation de Fokker-Planck peut s'interpréter comme un flot gradient pour la distance de Wasserstein continue. Nous établissons un lien entre une discrétisation spatiale du type Volume Finis pour l'équation de Fokker-Planck sur une tesselation de Voronoï et les flots gradient sur le réseau sous-jacent, pour une distance de type Wasserstein récemment introduite sur l'espace de mesures portées par les sommets d'un réseaux. / We are interested in the modeling of crowd motion. We propose a microscopic model based on game theoretic principles. Each individual is supposed to have a desired velocity, it is the one he would like to have in the absence of others. We consider that each individual is influenced by some of his neighbors, practically the ones that he sees. A possible actual velocity is an instantaneous Nash equilibrium: each individual does its best with respect to a personal objective (desired velocity), considering the behavior of the neighbors that influence him. We address theoretical and modeling issues in various situations, in particular when each individual is influenced by all the others, and in the case where the influence relations between individuals are hierarchical. We develop a numerical strategy to solve the problem in the second case (hierarchical model) and propose numerical simulations to illustrate the behavior of the model. We confront our numerical results with real experiments and prove the ability of the hierarchical model to reproduce some phenomena.We also propose to write a macroscopic counterpart of the hierarchical model by translating the same modeling principles to the macroscopic level and make the first steps towards writing such model.The last problem tackled in this thesis is related to gradient flow frameworks in the continuous and discrete Wasserstein spaces. It is known that the Fokker-Planck equation can be interpreted as a gradient flow for the continuous Wasserstein distance. We establish a link between some space discretization strategies of the Finite Volume type for the Fokker- Planck equation in general meshes (Voronoï tesselations) and gradient flows on the underlying networks of cells, in the framework of discrete Wasserstein-like distance on graphs recently introduced.

Submitted by Renata Lopes (renatasil82@gmail.com) on 2017-03-06T15:14:46Z No. of bitstreams: 1 ederdealmeidaperez.pdf: 749381 bytes, checksum: 7338f694cc850057100e730b520d74eb (MD5) / Approved for entry into archive by Adriana Oliveira (adriana.oliveira@ufjf.edu.br) on 2017-03-06T20:25:35Z (GMT) No. of bitstreams: 1 ederdealmeidaperez.pdf: 749381 bytes, checksum: 7338f694cc850057100e730b520d74eb (MD5) / Made available in DSpace on 2017-03-06T20:25:35Z (GMT). No. of bitstreams: 1 ederdealmeidaperez.pdf: 749381 bytes, checksum: 7338f694cc850057100e730b520d74eb (MD5) Previous issue date: 2012-08-24 / CAPES - Coordenação de Aperfeiçoamento de Pessoal de Nível Superior / O reconhecimento de padrões de movimentos tem se tornado um campo de pesquisa muito atrativo nos últimos anos devido, entre outros fatores, à grande massificação de dados em vídeos e a tendência na criação de interfaces homem-máquina que utilizam expressões faciais e corporais. Esse campo pode ser considerado um dos requisitos chave para análise e entendimento de vídeos. Neste trabalho é proposto um descritor de movimentos baseado em tensores de 2a ordem e histogramas de gradientes (HOG - Histogram of Oriented Gradients). O cálculo do descritor é rápido, simples e eficaz. Além disso, nenhum aprendizado prévio é necessário sendo que a adição de novas classes de movimentos ou novos vídeos não necessita de mudanças ou que se recalculem os descritores já existentes. Cada quadro do vídeo é particionado e em cada partição calcula-se o histograma de gradientes no espaço e no tempo. A partir daí calcula-se o tensor do quadro e o descritor final é formado por uma série de tensores de cada quadro. O descritor criado é avaliado classificando-se as bases de vídeos KTH e Hollywood2, utilizadas na literatura atual, com um classificador Máquina Vetor Suporte (SVM). Os resultados obtidos na base KTH são próximos aos descritores do estado da arte que utilizam informação local do vídeo. Os resultados obtidos na base Hollywood2 não superam o estado da arte, mas são próximos o suficiente para concluirmos que o método proposto é eficaz. Apesar de a literatura apresentar descritores que possuem resultados superiores na classificação, suas abordagens são complexas e de alto custo computacional. / The motion pattern recognition has become a very attractive research field in recent years due to the large amount of video data and the creation of human-machine interfaces that use facial and body expressions. This field can be considered one of the key requirements for analysis and understanding in video. This thesis proposes a motion descriptor based on second order tensor and histograms of oriented gradients. The calculation of the descriptor is fast, simple and effective. Furthermore, no prior knowledge of data basis is required and the addition of new classes of motion and videos do not need to recalculate the existing descriptors. The frame of a video is divided into a grid and the histogram of oriented gradients is computed in each cell. After that, the frame tensor is computed and the final descriptor is built by a series of frame tensors. The descriptor is evaluated in both KTH and Hollywood2 data basis, used in the current literature, with a Support Vector Machine classifier (SVM). The results obtained on the basis KTH are very close to the descriptors of the state-of-the-art that use local information of the video. The results obtained on the basis Hollywood2 not outweigh the state-of-the-art but are close enough to conclude that the proposed method is effective. Although the literature presents descriptors that have superior results, their approaches are complex and with computational cost.