The brightness variance, caused by relative velocity of the camera and environment in a sequence of images, is called optical flow. The advantage of the optical-flow-based visual servo method is that feature of the object dose not need to be known in advance. Therefore, it can be applied for positioning and tracking implement tasks. The purpose of this thesis is to implement the image servo technique and the sliding-mode control method to track an unknown image pattern in three dimensional motion. The goal of tracking is to maintain identical image captured by the camera based on the relative movement calculated from the optical flow.

We present investigation and designing of a system that interacts with big audience, sitting in a dimmed theater environment. The goal is to automatically detect audiences and some of their actions. Test results indicate that because of low light condition we can’t rely on RGB camera footage in a dimmed environment. We use Microsoft Kinect Sensor to collect data from environment. Kinect is designed to be used with Microsoft Xbox 360 for gaming purposes. It has both RGB and Infrared depth camera. Change in amount of visible light doesn’t affect data from depth camera. Kinect is not a strong camera so it has limitations that we should deal with. Viewing angles of both cameras and depth range of Infrared camera are limited. Viewing angles of depth camera are 43° vertical and 57° horizontal. Most accurate range of depth camera is 1 meter to 4 meters from camera. Non-infrared reflective surfaces cause gaps in depth data. We evaluate possibility of using Kinect camera in a large environment with big audience. “Dome 3D theater” in Norrkoping Visualization Center C, is selected as environment to investigate and test the system. We ran some tests to find the best place and best height for camera to have most coverage. Our system works with optimized image processing algorithms that use 3D depth data instead of regular RGB or Grayscale image. We use “libfreenect”, Open Kinect library to get Kinect sensor up and running. C++ and OpenGL are used as programing languages and graphics interface, respectively. Open GLUT (OpenGL Utility Toolkit) is used for system’s user interface. It was not possible to use Dome environment for every test during the programming period so we recorded some depth footage and used for later tests. While evaluating the possibility of using Kinect in Dome environment, we realized that implementing a voting system would make a good demonstration and test application. Our system counts votes after audiences raise their hands to vote for something.

The purpose of this study is to develop an automatic facial feature extraction system, which is able to identify the detailed shape of eyes, eyebrows and mouth from facial images. The developed system not only extracts the location information of the features, but also estimates the parameters pertaining the contours and parts of the features using parametric deformable templates approach. In order to extract facial features, deformable models for each of eye, eyebrow, and mouth are developed. The development steps of the geometry, imaging model and matching algorithms, and energy functions for each of these templates are presented in detail, along with the important implementation issues. In addition, an eigenfaces based multi-scale face detection algorithm which incorporates standard facial proportions is implemented, so that when a face is detected the rough search regions for the facial features are readily available. The developed system is tested on JAFFE (Japanese Females Facial Expression Database), Yale Faces, and ORL (Olivetti Research Laboratory) face image databases. The performance of each deformable templates, and the face detection algorithm are discussed separately.