With the rapid development in the areas of mobile manufacturing and multimedia communications, there is an increasing demand for Networked Virtual Environment (NVE) applications, such as Augmented Reality (AR), virtual walk-throughs, and massively multiplayer online games (MMOGs), on hand-held devices. Unfortunately, downloading and rendering a complex 3D scene is very computationally intensive and is not compatible with current mobile hardware specifications nor with available wireless bandwidth. Existing NVE applications deploy client/server based 3D streaming over thin mobile devices, which suffer from single point of failure, latency, and scalability issues. To address these issues, image-based rendering (IBR) and cloud-based 3D streaming have been introduced. The former introduces visual artifacts that reduce, and usually cancel, the realistic behaviors of the Virtual Environment (VE) application, while the latter is considered very expensive to implement. Peer-to-peer (P2P) 3D streaming is promising and affordable, but it has to tackle issues in object discovery and selection as well as content provider strategies. Distributing VE content over a mobile ad-hoc network (MANET) makes the system difficult to update due to the dynamic nature of the mobile clients. In order to tackle these issues, we came up with a novel protocol that combines the pros of both central and distributed approaches. Our proposed hybrid protocol, called OCTET, enables 3D scene streaming over thin devices in a way that can cope with current mobile hardware capabilities and mitigate the challenges of client/server and P2P 3D streaming. In fact, OCTET provides strategies that select, prioritize, and deliver only those objects that contribute to the user’s visible scene. OCTET is implemented using the "ns-2" simulation environment, and extensive experiments have clearly demonstrated significant achievements in mobile resource utilization, throughput, and system scalability.

With the rapid development in the areas of mobile manufacturing and multimedia communications, there is an increasing demand for Networked Virtual Environment (NVE) applications, such as Augmented Reality (AR), virtual walk-throughs, and massively multiplayer online games (MMOGs), on hand-held devices. Unfortunately, downloading and rendering a complex 3D scene is very computationally intensive and is not compatible with current mobile hardware specifications nor with available wireless bandwidth. Existing NVE applications deploy client/server based 3D streaming over thin mobile devices, which suffer from single point of failure, latency, and scalability issues. To address these issues, image-based rendering (IBR) and cloud-based 3D streaming have been introduced. The former introduces visual artifacts that reduce, and usually cancel, the realistic behaviors of the Virtual Environment (VE) application, while the latter is considered very expensive to implement. Peer-to-peer (P2P) 3D streaming is promising and affordable, but it has to tackle issues in object discovery and selection as well as content provider strategies. Distributing VE content over a mobile ad-hoc network (MANET) makes the system difficult to update due to the dynamic nature of the mobile clients. In order to tackle these issues, we came up with a novel protocol that combines the pros of both central and distributed approaches. Our proposed hybrid protocol, called OCTET, enables 3D scene streaming over thin devices in a way that can cope with current mobile hardware capabilities and mitigate the challenges of client/server and P2P 3D streaming. In fact, OCTET provides strategies that select, prioritize, and deliver only those objects that contribute to the user’s visible scene. OCTET is implemented using the "ns-2" simulation environment, and extensive experiments have clearly demonstrated significant achievements in mobile resource utilization, throughput, and system scalability.

With the rapid development in the areas of mobile manufacturing and multimedia communications, there is an increasing demand for Networked Virtual Environment (NVE) applications, such as Augmented Reality (AR), virtual walk-throughs, and massively multiplayer online games (MMOGs), on hand-held devices. Unfortunately, downloading and rendering a complex 3D scene is very computationally intensive and is not compatible with current mobile hardware specifications nor with available wireless bandwidth. Existing NVE applications deploy client/server based 3D streaming over thin mobile devices, which suffer from single point of failure, latency, and scalability issues. To address these issues, image-based rendering (IBR) and cloud-based 3D streaming have been introduced. The former introduces visual artifacts that reduce, and usually cancel, the realistic behaviors of the Virtual Environment (VE) application, while the latter is considered very expensive to implement. Peer-to-peer (P2P) 3D streaming is promising and affordable, but it has to tackle issues in object discovery and selection as well as content provider strategies. Distributing VE content over a mobile ad-hoc network (MANET) makes the system difficult to update due to the dynamic nature of the mobile clients. In order to tackle these issues, we came up with a novel protocol that combines the pros of both central and distributed approaches. Our proposed hybrid protocol, called OCTET, enables 3D scene streaming over thin devices in a way that can cope with current mobile hardware capabilities and mitigate the challenges of client/server and P2P 3D streaming. In fact, OCTET provides strategies that select, prioritize, and deliver only those objects that contribute to the user’s visible scene. OCTET is implemented using the "ns-2" simulation environment, and extensive experiments have clearly demonstrated significant achievements in mobile resource utilization, throughput, and system scalability.

There have been substantial improvements in the area of rehabilitation robotics in the recent past. However, these advances are inaccessible to a large number of people with disabilities who are in most need of such assistance. This group includes people who are in a severely paralyzed state, that they are completely "locked-in" in their own bodies. Such persons usually retain full cognitive abilities, but have no voluntary muscle control. For these persons, a Brain Computer Interface (BCI) is often the only way to communicate with the outside world and/or control an assistive device. One major drawback to BCI devices is their low information transfer rate, which can take as long as 30 seconds to select a single command. This can result in mental fatigue to the user, specially if it necessary to make multiple selections over the BCI to complete a single task. Therefore, P300 based BCI control is not efficient for controlling a assistive robotic device such as a robotic arm. To address this shortcoming, a novel vision based Brain Robot Interface (BRI) is presented in this thesis. This visual user interface allows for selecting an object from an unstructured environment and then performing an action on the selected object using a robotic arm mounted to a power wheelchair. As issuing commands through BCI is slow, this system was designed to allow a user to perform a complete task via a BCI using an autonomous robotic system while issuing as few commands as possible. Furthermore, the new visual interface allows the user to perform the task without losing concentration on the stimuli or the task. In our interface, a scene image is captured by a camera mounted on the wheelchair, from which, a dynamically sized non-uniform stimulus grid is created using edge information. Dynamically sized grids improve object selection efficiency. Oddball paradigm and P300 Event Related Potentials (ERP) are used to select stimuli, where the stimuli being each cell in the grid. Once selected, object segmentation and matching is used to identify the object. Then the user, using BRI, chooses an action to be performed on the object by the wheelchair mounted robotic arm (WMRA). Tests on 8 healthy human subjects validated the functionality of the system. An average accuracy of 85.56% was achieved for stimuli selection over all subjects. With the proposed system, it took the users an average of 5 commands to perform a task on an object. The system will eventually be useful for completely paralyzed or locked-in patients for performing activities of daily living (ADL) tasks.

This thesis is concerned with the task of autonomous selection of objects to remove (unload) them from a pile in robotic manipulation systems. Applications such as the automation of logistics processes and service robots require an ability to autonomously manipulate objects in the environment. A collapse of a pile of objects due to an inappropriate choice of the object to be removed from the pile cannot be afforded for an autonomous robotic manipulation system. This dissertation presents an indepth analysis of the problem and proposes methods and algorithms to empower robotic manipulation systems to select a safe object from a pile elaborately and autonomously. The contributions presented in this thesis are three-fold. First, a set of algorithms is proposed for extracting a minimal set of high level symbolic relations, namely, gravitational act and support relations, of physical interactions between objects composing a pile. The symbolic relations, extracted by a geometrical reasoning method and a static equilibrium analysis can be readily used by AI paradigms to analyze the stability of a pile and reason about the safest set of objects to be removed. Considering the problem of undetected objects and the uncertainty in the estimated poses as they exist in realistic perception systems, a probabilistic approach is proposed to extract the support relations and to make a probabilistic decision about the set of safest objects using notions from machine learning and decision theory. Second, an efficient search based algorithm is proposed in an internal representation to automatically resolve the inter-penetrations between the shapes of objects due to errors in the poses estimated by an existing object detection module. Refining the poses by resolving the inter-penetrations results in a geometrically consistent model of the environment, and was found to reduce the overall pose error of the objects. This dissertation presents the concept of minimum translation search for object pose refinement and discusses a discrete search paradigm based on the concept of depth of penetration between two polyhedrons. Third, an application centric evaluation of ranging sensors for selecting a set of appropriate sensors for the task of object detection in the design process of a real-world robotics manipulation system is presented. The performance of the proposed algorithms are tested on data sets generated in simulation and from real-world scenarios.

This thesis is concerned with modeling geometric patterns. Specifically, a clear and practical definition for regular patterns is proposed. Based on this definition, this thesis proposes the following modeling setting to describe the semantic transfer of a model between various forms of pattern regularity: (1) recognition or identification of patterns in digital models of 3D assemblies and scenes, (2) pattern regularization, (3) pattern modification and editing by varying the repetition parameters, and (4) establishing exceptions (designed irregularities) in regular patterns. In line with this setting, this thesis describes a representation and approach for designing and editing hierarchical assemblies based on grouped, nested, and recursively nested patterns. Based on this representation, this thesis presents the OCTOR approach for specifying, recording, and producing exceptions in regular patterns. To support editing of free-form shape patterns on surfaces, this thesis also presents the imprint-mapping approach which can be used to identify, extract, process, and apply relief features on surfaces. Pattern regularization, modification, and exceptions are addressed for the case of relief features on surfaces.