Traditional laparoscopic surgery continues to require significant training on the part of the surgeon before entering the operating room. Augmented Reality (AR) has been investigated for use in visual guidance in training and during surgery, but little work is available investigating the effectiveness of AR techniques in providing the user better awareness of depth and space. In this work we propose several 2D AR overlays for visual guidance in training for laparoscopic surgery, with the goal of aiding the user's perception of depth and space in that limiting environment. A pilot study of 30 subjects (22 male and 8 female) was performed with results showing the effect of the various overlays on subject performance of a path following task in the Computer Assisted Surgical Trainer (CAST-III) system developed in the Model Based Design Lab. Deviation, economy of movement, and completion time are considered as metrics. Providing a reference indicator for the nearest point on the optimal path is found to result in significant reduction (p < 0.05) in subject deviation from the path. The data also indicates a reduction in subject deviation along the depth axis and total path length with overlays designed to provide depth information. Avenues for further investigation are presented.

捕捉攝像機運動和重建攝像機成像場景深度圖的測定是在計算機視覺和機器任務包括可視化控制和自主導航是非常重要。在執行上述任務時，一個攝像機（或攝像機群組）通常安裝在機器的執行端部。攝像機和執行端部之間的手眼校準在視覺控制的正常操作中是不可缺少的。同樣，在對於需要使用多個攝像機的情况下，它們的相對幾何關係也是對各種計算機視覺應用來說也是非常重要。 / 攝像機和場景的相對運動通常產生出optical flow。問題的困難主要在於，在直接觀察視頻中的optical flow通常不是完全由運動誘導出的optical flow，而只是它的一部分。這個部分就是空間圖像等光線輪廓的正交。這部分的流場被稱為normal flow。本論文提出直接利用normal flow，而不是由normal flow引申出的optical flow，去解決以下的問題：尋找攝像機運動，場景深度圖和手眼校準。這種方法有許多顯著的貢獻，它不需引申流場，進而不要求平滑的成像場景。跟optical flow相反，normal flow不需要複雜的優化處理程序去解決流場不連續性的問題，這種技術一般是需要用大量的計算量。這也打破了傳統攝像機運動與場景深度之間的問題，在沒有預先知道不連續位置的情況下也可找出攝像機的運動。這篇論提出了幾個直接方法運用在三種不同類型的視覺系統，分別是單個攝像機，雙攝像機和多個攝像機，去找出攝像機的運動。 / 本論文首先提通過Apparent Flow 正深度 (AFPD) 約束去利用所有觀察到的normal flow去找出單個攝像機的運動參數。AFPD約束是利用一個優化問題來估計運動參數。一個反复由粗到細雙重約束的投票框架能使AFPD約束尋找出運動參數。 / 由於有限的視頻採樣率，normal flow在提取方向比其幅度部分更準確。本論文提出了兩個約束條件：一個是Apparent Flow方向（AFD）的約束，另外一個是Apparent Flow 幅度（AFM）的約束去尋找運動參數。第一個約束本身是作為一個線性不等式系統去約束運動方向的參數，第二個是利用所有圖像位置的旋轉幅度的統一性去進一步限制運動參數。一個兩階段從粗到細的約束框架能使AFD及AFM約束尋找出運動參數。 / 然而，如果沒有optical flow，normal flow是唯一的原始資料，它通常遭受到有限影像分辨率和有限視頻採樣率的問題而產生出錯誤。本文探討了這個問題的補救措施，方法是把一些攝像機併在一起，形成一個近似球形的攝像機，以增加成像系統的視野。有了一個加寬視野，normal flow的數量可更大，這可以用來抵銷normal flow在每個成像點的提取錯誤。更重要的是，攝像頭的平移和旋轉運動方向可以透過Apparent Flow分離 (AFS) 約束 及 延伸Apparent Flow分離 (EAFS) 約束來獨立估算。 / 除了使用單攝像機或球面成像系統之外，立體視覺成像系統提供了其它的視覺線索去尋找攝像機在沒有被任意縮放大小的平移運動和深度圖。傳統的立體視覺方法是確定在兩個輸入圖像特徵的對應。然而，對應的建立是非常困難。本文探討了兩個直接方法來恢復完整的攝像機運動，而沒有需要利用一對影像明確的點至點對應。第一種方法是利用AFD和AFM約束伸延到立體視覺系統，並提供了一個穩定的幾何方法來確定平移運動的幅度。第二個方法需要利用有一個較大的重疊視場，以提供一個不需反覆計算的closed-form算法。一旦確定了運動參數，深度圖可以沒有任何困難地重建。從normal flow產生的深度圖一般是以稀疏的形式存在。我們可以通過擴張深度圖，然後利用它作為在常見的TV-L₁框架的初始估計。其結果不僅有一個更好的重建性能，也產生出更快的運算時間。 / 手眼校準通常是基於像圖特徵對應。本文提出一個替代方法，是從動態攝像系統產生的normal flow來做自我校準。為了使這個方法有更強防備噪音的能力，策略是使用normal flow的流場方向去尋找手眼幾何的方向部份。偏離點及部分的手眼幾何可利用normal flow固有的流場屬性去尋找。最後完整的手眼幾何可使用穩定法來變得更可靠。手眼校準還可以被用來確定多個攝像機的相對幾何關係，而不需要求它們有重疊的視場。 / Determination of general camera motion and reconstructing depth map from a captured video of the imaged scene relative to a camera is important for computer vision and various robotics tasks including visual control and autonomous navigation. A camera (or a cluster of cameras) is usually mounted on the end-effector of a robot arm when performing the above tasks. The determination of the relative geometry between the camera frame and the end-effector frame which is commonly referred as hand-eye calibration is essential to proper operation in visual control. Similarly, determining the relative geometry of multiple cameras is also important to various applications requiring the use of multi-camera rig. / The relative motion between an observer and the imaged scene generally induces apparent flow in the video. The difficulty of the problem lies mainly in that the flow pattern directly observable in the video is generally not the full flow field induced by the motion, but only partial information of it, which is orthogonal to the iso-brightness contour of the spatial image intensity profile. The partial flow field is known as the normal flow field. This thesis addresses several important problems in computer vision: determination of camera motion, recovery of depth map, and performing hand-eye calibration from the apparent flow (normal flow) pattern itself in the video data directly but not from the full flow interpolated from the apparent flow. This approach has a number of significant contributions. It does not require interpolating the flow field and in turn does not demand the imaged scene to be smooth. In contrast to optical flow, no sophisticated optimization procedures that account for handling flow discontinuities are required, and such techniques are generally computational expensive. It also breaks the classical chicken-and-egg problem between scene depth and camera motion. No prior knowledge about the locations of the discontinuities is required for motion determination. In this thesis, several direct methods are proposed to determine camera motion using three different types of imaging systems, namely monocular camera, stereo camera, and multi-camera rig. / This thesis begins with the Apparent Flow Positive Depth (AFPD) constraint to determine the motion parameters using all observable normal flows from a monocular camera. The constraint presents itself as an optimization problem to estimate the motion parameters. An iterative process in a constrained dual coarse-to-fine voting framework on the motion parameter space is used to exploit the constraint. / Due to the finite video sampling rate, the extracted normal flow field is generally more accurate in direction component than its magnitude part. This thesis proposes two constraints: one related to the direction component of the normal flow field - the Apparent Flow Direction (AFD) constraint, and the other to the magnitude component of the field - the Apparent Flow Magnitude (AFM) constraint, to determine motion. The first constraint presents itself as a system of linear inequalities to bind the direction of motion parameters; the second one uses the globality of rotational magnitude to all image positions to constrain the motion parameters further. A two-stage iterative process in a coarse-to-fine framework on the motion parameter space is used to exploit the two constraints. / Yet without the need of the interpolation step, normal flow is only raw information extracted locally that generally suffers from flow extraction error arisen from finiteness of the image resolution and video sampling rate. This thesis explores a remedy to the problem, which is to increase the visual field of the imaging system by fixating a number of cameras together to form an approximate spherical eye. With a substantially widened visual field, the normal flow data points would be in a much greater number, which can be used to combat the local flow extraction error at each image point. More importantly, the directions of translation and rotation components in general motion can be separately estimated with the use of the novel Apparent Flow Separation (AFS) and Extended Apparent Flow Separation (EAFS) constraints. / Instead of using a monocular camera or a spherical imaging system, stereo vision contributes another visual clue to determine magnitude of translation and depth map without the problem of arbitrarily scaling of the magnitude. The conventional approach in stereo vision is to determine feature correspondences across the two input images. However, the correspondence establishment is often difficult. This thesis explores two direct methods to recover the complete camera motion from the stereo system without the explicit point-to-point correspondences matching. The first method extends the AFD and AFM constraints to stereo camera, and provides a robust geometrical method to determine translation magnitude. The second method which requires the stereo image pair to have a large overlapped field of view provides a closed-form solution, requiring no iterative computation. Once the motion parameters are here, depth map can be reconstructed without any difficulty. The depth map resulted from normal flows is generally sparse in nature. We can interpolate the depth map and then utilizing it as an initial estimate in a conventional TV-L₁ framework. The result is not only a better reconstruction performance, but also a faster computation time. / Calibration of hand-eye geometry is usually based on feature correspondences. This thesis presents an alternative method that uses normal flows generated from an active camera system to perform self-calibration. In order to make the method more robust to noise, the strategy is to use the direction component of the flow field which is more noise-immune to recover the direction part of the hand-eye geometry first. Outliers are then detected using some intrinsic properties of the flow field together with the partially recovered hand-eye geometry. The final solution is refined using a robust method. The method can also be used to determine the relative geometry of multiple cameras without demanding overlap in their visual fields. / Detailed summary in vernacular field only. / Detailed summary in vernacular field only. / Detailed summary in vernacular field only. / Detailed summary in vernacular field only. / Detailed summary in vernacular field only. / Detailed summary in vernacular field only. / Detailed summary in vernacular field only. / Hui, Tak Wai. / Thesis (Ph.D.)--Chinese University of Hong Kong, 2013. / Includes bibliographical references (leaves 159-165). / Abstracts in English and Chinese. / Acknowledgements --- p.i / Abstract --- p.ii / Lists of Figures --- p.xiii / Lists of Tables --- p.xix / Chapter Chapter 1 --- Introduction --- p.1 / Chapter 1.1 --- Background --- p.1 / Chapter 1.2 --- Motivation --- p.4 / Chapter 1.3 --- Research Objectives --- p.6 / Chapter 1.4 --- Thesis Outline --- p.7 / Chapter Chapter 2 --- Literature Review --- p.10 / Chapter 2.1 --- Introduction --- p.10 / Chapter 2.2 --- Recovery of Optical Flows --- p.10 / Chapter 2.3 --- Egomotion Estimation Based on Optical Flow Field --- p.14 / Chapter 2.3.1 --- Bilinear Constraint --- p.14 / Chapter 2.3.2 --- Subspace Method --- p.15 / Chapter 2.3.3 --- Partial Search Method --- p.16 / Chapter 2.3.4 --- Fixation --- p.17 / Chapter 2.3.5 --- Region Alignment --- p.17 / Chapter 2.3.6 --- Linearity and Divergence Properties of Optical Flows --- p.18 / Chapter 2.3.7 --- Constraint Lines and Collinear Points --- p.18 / Chapter 2.3.8 --- Multi-Camera Rig --- p.19 / Chapter 2.3.9 --- Discussion --- p.21 / Chapter 2.4 --- Determining Egomotion Using Direct Methods --- p.22 / Chapter 2.4.1 --- Introduction --- p.22 / Chapter 2.4.2 --- Classical Methods --- p.23 / Chapter 2.4.3 --- Pattern Matching --- p.24 / Chapter 2.4.4 --- Search Subspace Method --- p.25 / Chapter 2.4.5 --- Histogram-Based Method --- p.26 / Chapter 2.4.6 --- Multi-Camera Rig --- p.26 / Chapter 2.4.7 --- Discussion --- p.27 / Chapter 2.5 --- Determining Egomotion Using Feature Correspondences --- p.28 / Chapter 2.6 --- Hand-Eye Calibration --- p.30 / Chapter 2.7 --- Summary --- p.31 / Chapter Chapter 3 --- Determining Motion from Monocular Camera Using Merely the Positive Depth Constraint --- p.32 / Chapter 3.1 --- Introduction --- p.32 / Chapter 3.2 --- Related Works --- p.33 / Chapter 3.3 --- Background --- p.34 / Chapter 3.3 --- Apparent Flow Positive Depth (AFPD) Constraint --- p.39 / Chapter 3.4 --- Numerical Solution to AFPD Constraint --- p.40 / Chapter 3.5 --- Constrained Coarse-to-Fine Searching --- p.40 / Chapter 3.6 --- Experimental Results --- p.43 / Chapter 3.7 --- Conclusion --- p.47 / Chapter Chapter 4 --- Determining Motion from Monocular Camera Using Direction and Magnitude of Normal Flows Separately --- p.48 / Chapter 4.1 --- Introduction --- p.48 / Chapter 4.2 --- Related Works --- p.50 / Chapter 4.3 --- Apparent Flow Direction (AFD) Constraint --- p.51 / Chapter 4.3.1 --- The Special Case: Pure Translation --- p.51 / Chapter 4.3.1.1 --- Locus of Translation Using Full Flow as a Constraint --- p.51 / Chapter 4.3.1.2 --- Locus of Translation Using Normal Flow as a Constraint --- p.53 / Chapter 4.3.2 --- The Special Case: Pure Rotation --- p.54 / Chapter 4.3.2.1 --- Locus of Rotation Using Full Flow as a Constraint --- p.54 / Chapter 4.3.2.2 --- Locus of Rotation Using Normal Flow as a Constraint --- p.54 / Chapter 4.3.3 --- Solving the System of Linear Inequalities for the Two Special Cases --- p.55 / Chapter 4.3.5 --- Ambiguities of AFD Constraint --- p.59 / Chapter 4.4 --- Apparent Flow Magnitude (AFM) Constraint --- p.60 / Chapter 4.5 --- Putting the Two Constraints Together --- p.63 / Chapter 4.6 --- Experimental Results --- p.65 / Chapter 4.6.1 --- Simulation --- p.65 / Chapter 4.6.2 --- Video Data --- p.67 / Chapter 4.6.2.1 --- Pure Translation --- p.67 / Chapter 4.6.2.2 --- General Motion --- p.68 / Chapter 4.7 --- Conclusion --- p.72 / Chapter Chapter 5 --- Determining Motion from Multi-Cameras with Non-Overlapping Visual Fields --- p.73 / Chapter 5.1 --- Introduction --- p.73 / Chapter 5.2 --- Related Works --- p.75 / Chapter 5.3 --- Background --- p.76 / Chapter 5.3.1 --- Image Sphere --- p.77 / Chapter 5.3.2 --- Planar Case --- p.78 / Chapter 5.3.3 --- Projective Transformation --- p.79 / Chapter 5.4 --- Constraint from Normal Flows --- p.80 / Chapter 5.5 --- Approximation of Spherical Eye by Multiple Cameras --- p.81 / Chapter 5.6 --- Recovery of Motion Parameters --- p.83 / Chapter 5.6.1 --- Classification of a Pair of Normal Flows --- p.84 / Chapter 5.6.2 --- Classification of a Triplet of Normal Flows --- p.86 / Chapter 5.6.3 --- Apparent Flow Separation (AFS) Constraint --- p.87 / Chapter 5.6.3.1 --- Constraint to Direction of Translation --- p.87 / Chapter 5.6.3.2 --- Constraint to Direction of Rotation --- p.88 / Chapter 5.6.3.3 --- Remarks about the AFS Constraint --- p.88 / Chapter 5.6.4 --- Extension of Apparent Flow Separation Constraint (EAFS) --- p.89 / Chapter 5.6.4.1 --- Constraint to Direction of Translation --- p.90 / Chapter 5.6.4.2 --- Constraint to Direction of Rotation --- p.92 / Chapter 5.6.5 --- Solution to the AFS and EAFS Constraints --- p.94 / Chapter 5.6.6 --- Apparent Flow Magnitude (AFM) Constraint --- p.96 / Chapter 5.7 --- Experimental Results --- p.98 / Chapter 5.7.1 --- Simulation --- p.98 / Chapter 5.7.2 --- Real Video --- p.103 / Chapter 5.7.2.1 --- Using Feature Correspondences --- p.108 / Chapter 5.7.2.2 --- Using Optical Flows --- p.108 / Chapter 5.7.2.3 --- Using Direct Methods --- p.109 / Chapter 5.8 --- Conclusion --- p.111 / Chapter Chapter 6 --- Motion and Shape from Binocular Camera System: An Extension of AFD and AFM Constraints --- p.112 / Chapter 6.1 --- Introduction --- p.112 / Chapter 6.2 --- Related Works --- p.112 / Chapter 6.3 --- Recovery of Camera Motion Using Search Subspaces --- p.113 / Chapter 6.4 --- Correspondence-Free Stereo Vision --- p.114 / Chapter 6.4.1 --- Determination of Full Translation Using Two 3D Lines --- p.114 / Chapter 6.4.2 --- Determination of Full Translation Using All Normal Flows --- p.115 / Chapter 6.4.3 --- Determination of Full Translation Using a Geometrical Method --- p.117 / Chapter 6.5 --- Experimental Results --- p.119 / Chapter 6.5.1 --- Synthetic Image Data --- p.119 / Chapter 6.5.2 --- Real Scene --- p.120 / Chapter 6.6 --- Conclusion --- p.122 / Chapter Chapter 7 --- Motion and Shape from Binocular Camera System: A Closed-Form Solution for Motion Determination --- p.123 / Chapter 7.1 --- Introduction --- p.123 / Chapter 7.2 --- Related Works --- p.124 / Chapter 7.3 --- Background --- p.125 / Chapter 7.4 --- Recovery of Camera Motion Using a Linear Method --- p.126 / Chapter 7.4.1 --- Region-Correspondence Stereo Vision --- p.126 / Chapter 7.3.2 --- Combined with Epipolar Constraints --- p.127 / Chapter 7.4 --- Refinement of Scene Depth --- p.131 / Chapter 7.4.1 --- Using Spatial and Temporal Constraints --- p.131 / Chapter 7.4.2 --- Using Stereo Image Pairs --- p.134 / Chapter 7.5 --- Experiments --- p.136 / Chapter 7.5.1 --- Synthetic Data --- p.136 / Chapter 7.5.2 --- Real Image Sequences --- p.137 / Chapter 7.6 --- Conclusion --- p.143 / Chapter Chapter 8 --- Hand-Eye Calibration Using Normal Flows --- p.144 / Chapter 8.1 --- Introduction --- p.144 / Chapter 8.2 --- Related Works --- p.144 / Chapter 8.3 --- Problem Formulation --- p.145 / Chapter 8.3 --- Model-Based Brightness Constraint --- p.146 / Chapter 8.4 --- Hand-Eye Calibration --- p.147 / Chapter 8.4.1 --- Determining the Rotation Matrix R --- p.148 / Chapter 8.4.2 --- Determining the Direction of Position Vector T --- p.149 / Chapter 8.4.3 --- Determining the Complete Position Vector T --- p.150 / Chapter 8.4.4 --- Extrinsic Calibration of a Multi-Camera Rig --- p.151 / Chapter 8.5 --- Experimental Results --- p.151 / Chapter 8.5.1 --- Synthetic Data --- p.151 / Chapter 8.5.2 --- Real Image Data --- p.152 / Chapter 8.6 --- Conclusion --- p.153 / Chapter Chapter 9 --- Conclusion and Future Work --- p.154 / Related Publications --- p.158 / Bibliography --- p.159 / Appendix --- p.166 / Chapter A --- Apparent Flow Direction Constraint --- p.166 / Chapter B --- Ambiguity of AFD Constraint --- p.168 / Chapter C --- Relationship between the Angle Subtended by any two Flow Vectors in Image Plane and the Associated Flow Vectors in Image Sphere --- p.169

Many studies have found that fixating or directing spatial attention to different regions can bias the perception of the Necker cube, but whether this effect of spatial attention is due to attended areas perceived as being closer have yet to be examined. This issue was directly investigated in this study. The stimulus used was the diamond stimulus, containing four occluders and four moving lines that can be perceived as coherent or separate motions. The results of Experiment 1 show that coherent motion was perceived more often under the attending-to-occluders condition than under the attending-to-moving-lines condition, indicating that spatial attention can bias multistable perception. The results of Experiment 2 show that the mean probability of reporting lines behind occluders in small binocular disparities was significantly higher under the attending-to-occluders condition than under the attending-to-lines condition, indicating that spatial attention can make attended areas look slightly closer. The results of Experiments 3 and 4 show that the effect of spatial attention on biasing multistable perception was weakened when there were binocular or monocular depth cues to define the depth relationship between the occluders and the lines. These results are all consistent with the notion that spatial attention can bias multistable perception through affecting depth perception, making attended areas look closer.

<p>In normal image coding the image quality is the same in all parts of the image. When it is known where in the image a single viewer is focusing it is possible to lower the image quality in other parts of the image without lowering the perceived image quality. This master's thesis introduces a coding scheme based on depth perception where the quality of the parts of the image that correspond to out-of-focus scene objects is lowered to obtain data reduction. To obtain further data reduction the method is combined with angular perception coding where the quality is lowered in parts of the image corresponding to the peripheral visual field. It is concluded that depth perception coding can be done without lowering the perceived image quality and that the coding gain increases as the two methods are combined.</p>

In normal image coding the image quality is the same in all parts of the image. When it is known where in the image a single viewer is focusing it is possible to lower the image quality in other parts of the image without lowering the perceived image quality. This master's thesis introduces a coding scheme based on depth perception where the quality of the parts of the image that correspond to out-of-focus scene objects is lowered to obtain data reduction. To obtain further data reduction the method is combined with angular perception coding where the quality is lowered in parts of the image corresponding to the peripheral visual field. It is concluded that depth perception coding can be done without lowering the perceived image quality and that the coding gain increases as the two methods are combined.

Stereoscopic displays provide 3D vision usually with the help of additional equipment such as shutter glasses and head gears. As a new stereoscopic display technology, autostereoscopic 3D Displays provide 3D vision without additional equipment. Previous studies of depth and distance estimation with autostereoscopic displays indicate the users do not exhibit better performance in 3D. Yet, they claim 3D displays provide higher immersiveness. In this study, perception of the angle of a 3D shape is investigated by comparing 2D, 3D and Real perception cases. An experiment is conducted using an autostereoscopic 3D display. Forty people have participated in the experiment. They were asked to estimate the vertex angle and draw the projections of the object from two different viewpoints. It is found that users can better estimate the angles on a cone when viewed from the top on an autostereoscopic display. This may contribute positively to 3D understanding of the scene. Results revealed that participants make more accurate angle estimation in autostereoscopic 3D displays than in traditional 2D displays. In general, the participants&rsquo / angle drawings were slightly higher than their angle estimations. Moreover, the participants overestimated 35, 65 and 90 degree angles and underestimated 115 degree angle in autostereoscopic 3D display.

Thesis (Ph. D.)--Virginia Polytechnic Institute and State University, 1991. / Vita. Abstract. Includes bibliographical references (leaves 174-179). Also available via the Internet

In very few mobile robotic applications stereo vision based navigation and mapping is used because dealing with stereo images is very hard and very time consuming. Despite all the problems, stereo vision still becomes one of the most important resources of knowing the world for a mobile robot because imaging provides much more information than most other sensors. Real robotic applications are very complicated because besides the problems of finding how the robot should behave to complete the task at hand, the problems faced while controlling the robot&rsquo / s internal parameters bring high computational load. Thus, finding the strategy to be followed in a simulated world and then applying this on real robot for real applications is preferable. In this study, we describe an algorithm for object recognition and cognitive map formation using stereo image data in a 3D virtual world where 3D objects and a robot with active stereo imaging system are simulated. Stereo imaging system is simulated so that the actual human visual system properties are parameterized. Only the stereo images obtained from this world are supplied to the virtual robot. By applying our disparity algorithm, depth map for the current stereo view is extracted. Using the depth information for the current view, a cognitive map of the environment is updated gradually while the virtual agent is exploring the environment. The agent explores its environment in an intelligent way using the current view and environmental map information obtained up to date. Also, during exploration if a new object is observed, the robot turns around it, obtains stereo images from different directions and extracts the model of the object in 3D. Using the available set of possible objects, it recognizes the object.

Thesis (M.S.)--Virginia Polytechnic Institute and State University, 1993. / Vita. Abstract. Includes bibliographical references (leaves 52-55). Also available via the Internet.

Thesis (M.S.)--Mississippi State University. Department of Computer Science and Engineering. / Title from title screen. Includes bibliographical references.