Master of Science / Department of Mechanical and Nuclear Engineering / Akira T. Tokuhiro / Robots will eventually become common everyday items. However before this becomes a reality, robots would need to learn be socially interactive. Since humans communicate much more information through expression than through actual spoken words, expression recognition is an important aspect in the development of social robots. Automatic recognition of emotional expressions has a number of potential applications other than just social robots. It can be used in systems that make sure the operator is alert at all times, or it can be used for psycho-analysis or cognitive studies. Emotional expressions are not always deliberate and can also occur without the person being aware of them. Recognizing these involuntary expressions provide an insight into the persons thought, state of mind and could be used as indicators for a hidden intent. In this research we developed an initial multi-modal emotion recognition system using cues from emotional expressions in face and voice. This is achieved by extracting features from each of the modalities using signal processing techniques, and then classifying these features with the help of artificial neural networks. The features extracted from the face are the eyes, eyebrows, mouth and nose; this is done using image processing techniques such as seeded region growing algorithm, particle swarm optimization and general properties of the feature being extracted. In contrast features of interest in speech are pitch, formant frequencies and mel spectrum along with some statistical properties such as mean and median and also the rate of change of these properties. These features are extracted using techniques such as Fourier transform and linear predictive coding. We have developed a toolbox that can read an audio and/or video file and perform emotion recognition on the face in the video and speech in the audio channel. The features extracted from the face and voices are independently classified into emotions using two separate feed forward type of artificial neural networks. This toolbox then presents the output of the artificial neural networks from one/both the modalities on a synchronized time scale. Some interesting results from this research is consistent misclassification of facial expressions between two databases, suggesting a cultural basis for this confusion. Addition of voice component has been shown to partially help in better classification.

Master of Science / Department of Electrical and Computer Engineering / Sanjoy Das / Stephen M. Welch / Multi-objective optimization problems deal with finding a set of candidate optimal solutions to be presented to the decision maker. In industry, this could be the problem of finding alternative car designs given the usually conflicting objectives of performance, safety, environmental friendliness, ease of maintenance, price among others. Despite the significance of this problem, most of the non-evolutionary algorithms which are widely used cannot find a set of diverse and nearly optimal solutions due to the huge size of the search space. At the same time, the solution set produced by most of the currently used evolutionary algorithms lacks diversity. The present study investigates a new optimization method to solve multi-objective problems based on the widely used swarm-intelligence approach, Particle Swarm Optimization (PSO). Compared to other approaches, the proposed algorithm converges relatively fast while maintaining a diverse set of solutions. The investigated algorithm, Partially Informed Fuzzy-Dominance (PIFD) based PSO uses a dynamic network topology and fuzzy dominance to guide the swarm of dominated solutions. The proposed algorithm in this study has been tested on four benchmark problems and other real-world applications to ensure proper functionality and assess overall performance. The multi-objective gene regulatory network (GRN) problem entails the minimization of the coefficient of variation of modified photothermal units (MPTUs) across multiple sites along with the total sum of similarity background between ecotypes. The results throughout the current research study show that the investigated algorithm attains outstanding performance regarding optimization aspects, and exhibits rapid convergence and diversity.