京都大学 / 0048 / 新制・課程博士 / 博士(情報学) / 甲第21910号 / 情博第693号 / 新制||情||119(附属図書館) / 京都大学大学院情報学研究科知能情報学専攻 / (主査)講師 延原 章平, 教授 西野 恒, 准教授 飯山 将晃 / 学位規則第4条第1項該当 / Doctor of Informatics / Kyoto University / DFAM

The aim of this thesis is to investigate how well a generic monocular camera, placed on the vehicle, can be employed to localize an autonomous vehicle in a warehouse setting. The main function is to ascertain which zone the vehicle is currently in, as well as update the status when entering a new zone. Two zones are defined, where one has a lower allowed top velocity and the other a higher one. For this purpose ArUco markers are used to signal the system as to where it currently is. Markers are strategically placed around the laboratory area to saturate the environment with possible detections. Multiple sequences are recorded while varying camera placement, angles, and paths to determine the optimal number and placement of markers. In addition to this, a SLAM solution is tested in order to explore what benefits can be found. The idea is to provide fine-grained localization as well as a map of the warehouse environment, to provide more options for further development. To solve the SLAM problem, an implemented particle filter approach initializes a set of particles uniformly distributed within the world frame. For each frame, the particles undergo pose prediction, weight assignment based on likelihood, and resampling. This iterative process gradually converges the particles toward the camera's true position. Visual odometry techniques are used to estimate the camera's ego-motion. The process involves acquiring a sequence of images, detecting distinctive features, matching features between consecutive frames, estimating camera motion, and optionally applying local optimization techniques for further refinement. The implementation shows promise and all test cases performed during the project have been successful as for the zone localization. The SLAM solution can detect and track specific features or landmarks over consecutive frames. By triangulating the positions of these features, their depth and distance can be determined. However, the visualization of these features on a top-down map, which was part of the plan, has not been completed yet despite finishing the particle filter implementation.

No description available.

We propose a methodology for calibrating the activity space and the cameras involved in video capture systems for upper extremity stroke rehabilitation. We discuss an in-home stroke rehabilitation system called Semi-Automated Rehabilitation At Home System (SARAH) and a clinic-based system called Action Research Arm Test (ARAT) developed by the Interactive Neuro-Rehabilitation Lab (INR) at Virginia Tech. We propose a calibration workflow for achieving invariant video capture across multiple therapy sessions. This ensures that the captured data is less noisy. In addition, there is prior knowledge of the captured activity space and patient location in the video frames provided to the Computer Vision algorithms analyzing the captured data. Such a standardized calibration approach improved machine learning analysis of patient movements and a higher rate of agreement across multiple therapists regarding the captured patient performance. We further propose a Multi-Camera Calibration approach to perform stereo camera calibration in SARAH and ARAT capture systems to help perform a 3D reconstruction of the activity space from 2D videos. The importance of the proposed activity space and camera calibration workflows, including new research paths opened as a result of our approach, are discussed in this thesis. / Master of Science / In this thesis, I describe the workflows I developed to perform calibration of stroke rehabilitation activity spaces, including the calibration of cameras involved in video capture systems for analyzing patient movements in stroke rehabilitation practices. The proposed workflows are designed to facilitate convenient user involvement in calibrating the video capture systems to provide invariant and consistent video captures, including the extraction of fine-grain information utilizing camera calibration results, to the therapists and computer vision-based automated systems for improved analysis of patient performance in stroke rehabilitation practices. The importance of human-in-the-loop systems, including future research paths to strengthen the symbiotic relationship between humans and Artificial Intelligence systems in stroke rehabilitation practices, is discussed. The quantitative and qualitative results generated from the workshops conducted to test and evaluate the calibration workflows align with the stakeholder's needs in stroke rehabilitation systems.

When considering the operation of an Unmanned Aerial Vehicle (UAV) or an Unmanned Ground Vehicle (UGV), such problems as landing site estimation or robot path planning become a concern. Deciding if an area of terrain has a level enough slope and a wide enough area to land a Vertical Take Off and Landing (VTOL) UAV or if an area of terrain is traversable by a ground robot is reliant on data gathered from sensors, such as cameras. 3-D models, which can be built from data extracted from digital cameras, can help facilitate decision making for such tasks by providing a virtual model of the surrounding environment the system is in. A stereo vision system utilizes two or more cameras, which capture images of a scene from two or more viewpoints, to create 3-D point clouds. A point cloud is a set of un-gridded 3-D points corresponding to a 2-D image, and is used to build gridded surface models. Designing a stereo system for distant terrain modeling requires an extended baseline, or distance between the two cameras, in order to obtain a reasonable depth resolution. As the width of the baseline increases, so does the flexibility of the system, causing the orientation of the cameras to deviate from their original state. A set of tools have been developed to generate 3-D point clouds from rigid and flexible stereo systems, along with a method for applying corrections to a flexible system to regain distance accuracy in a flexible system. / Master of Science

Computed tomography for flame chemiluminescence emissions allows for 3D spatially resolved flame measurements to be acquired using a series of discrete viewing angle camera images. To determine fuel/air ratios, the ratio of excited radical species (OH*/CH*) emissions using chemiluminescence can be employed. Following the process of high-resolution tomography reconstructions in this work allowed for flame tomography coupled with chemiluminescence emissions to be used for spatially resolved phase averaged equivalence ratio measurements. This is important as variations in local equivalence ratios can have a profound effect on flame behavior including but not limited to thermoacoustic instability, NOx and CO formation, and flame stabilization. Local equivalence ratios are determined from a OH*/CH* ratio of tomographically reconstructed intensity fields and relating them to equivalence ratio. The correlation of OH*/CH* to equivalence ratio is derived from an axisymmetric, commercially available flat flame burner (Holthuis and Associates Burner). To relate intensity field imaging (camera coordinate system) during the tomographic reconstruction to the real-world coordinate system of the burner a calibration procedure was performed and then validated. A calibration plate with 39 non-coplanar points was used in this procedure. It was then validated by comparing the Abel inverted flame images of the axisymmetric Holthuis and Associates burner with the tomographic reconstructed images. Results show a successful tomographic reconstruction of thermoacoustic self-excited cycle concluding equivalence ratio fluctuations coinciding with the 1st dominate frequency of the pressure fluctuations and influenced by a 2nd harmonic frequency. / Master of Science / In recent years tomographic reconstruction of flames have gained significant focus in understanding different flame phenomenon. One specific flame phenomenon is known as a thermoacoustic instability. Using highspeed cameras for chemiluminescence imaging of specific species can help define heat release rate, air/fuel ratio/equivalence ratio spatially. Coupling of pressure measurements to imaging methods can be used to determine the flames response to acoustic perturbations in the flow field. Every optics system has inherently different light transmission characteristics and therefore, needs to be calibrated/correlated using a known flame source. The work done in this paper used a Holthuis and Associates flat flame as the known flame source in conjunction with an optics system to correlate OH*/CH* ratio to equivalence ratio. This is possible due to the perfectly premixed nature the flat flame provides. The correlation curve for the optics system is then applied to the tomographically reconstructed chemiluminescence intensities during a self-excited thermo-acoustic instability. In addition, a flat flame burner was used to validate the tomography approach and calibration procedure. In conclusion the objective of this work develops and validates a method for use in tomographic reconstruction of spatially varying equivalence ratios.

Assessment of finger range of motion (ROM) is often required for monitoring the effectiveness of rehabilitative treatments and for evaluating patients' functional impairment. There are several devices which are used to measure this motion, such as wire tracing, tracing onto paper and mechanical and electronic goniometry. These devices are quite cheap, excluding electronic goniometry; however the drawbacks of these devices are their lack of accuracy and the time- consuming nature of the measurement process. The work described in this thesis considers the design, implementation and validation of a new medical measurement system utilized in the evaluation of the range of motion of the human finger joints instead of the current measurement tools. The proposed system is a non-contact measurement device based on computer vision technology and has many advantages over the existing measurement devices. In terms of accuracy, better results are achieved by this system, it can be operated by semi-skilled person, and is time saving for the evaluator. The computer vision system in this study consists of CCD cameras to capture the images, a frame-grabber to change the analogue signal from the cameras to digital signals which can be manipulated by a computer, Ultra Violet light (UV) to illuminate the measurement space, software to process the images and perform the required computation, a darkened enclosure to accommodate the cameras and UV light and to shield the working area from any undesirable ambient light. Two calibration techniques were used to calibrate the cameras, Direct Linear Transformation and Tsai. A calibration piece that suits this application was designed and manufactured. A steel hand model was used to measure the fingers joint angles. The average error from measuring the finger angles using this system was around 1 degree compared with 5 degrees for the existing used techniques.

Syftet med aktuellt arbete var att kalibrera en kamera i MATLAB, med hjälp av tillägget Computer Vision Toolbox, samt utforma ett skript som kan konvertera pixelkoordinater i MATLAB till rumskoordinater. Resultatet av arbetet agerar som ett komplement till en studie på Gymnastik- och idrottshögskolan, där kompression av skosulor undersöks med hjälp av kamera och MATLAB-skript. Flertalet tester utfördes för att säkerställa kalibreringens tillförlitlighet samt kompatibiliteten mellan de två MATLAB-skripten. Kalibreringen anses tillförlitlig, och kompatibiliteten mellan skripten anses god. Vidare utfördes en mindre kompressionsstudie på löpband. Resultaten från denna studie är ej tillförlitliga, eftersom väldigt stora fel behövde tillåtas för att möjliggöra skriptets så kallade pixelspårning. Detta anses vara på grund av ljussättningen vid testerna, samt att kamerans videoupptagning ej hade hög nog bildupptagning per sekund till att följa skornas höga hastighet på löpbandet. Vidare studier av kompression rekommenderas att utföras på stilla underlag, där foten är huvudsakligen stilla i bildens synfält under markkontakt. / The purpose of this work was to calibrate a camera in MATLAB using the Computer Vision Toolbox add-on and to design a script to convert pixel coordinates in MATLAB into spatial coordinates in the room. The result of this work serves as a complement to a study conducted at Gymnastik- och idrottshögskolan in Stockholm, where they are investigating the compression of shoe soles with a camera and a MATLAB-script. Several tests were conducted to ensure the reliability of the calibration along with the compatibility of the two MATLAB scripts. The calibration is considered reliable, and the compatibility of the two scripts is considered satisfactory. Furthermore, a smaller compression study was performed on a treadmill. The results from this study are considered unreliable, as large errors were allowed in the so-called pixel tracking of the MATLAB script. This is considered to be due to bad lighting, and because the video recording did not have a high enough frames-per-second to follow the high velocity of the shoes on the treadmill. Further compression studies are recommended to be performed on stable, non-moving surfaces, where the foot is principally still in the camera’s field of view during ground contact.

A identificação de pontos em padrões de calibração é relatada como uma fase trabalhosa nos processos de calibração de câmeras. Essa etapa é sensível à presença de ruídos e geralmente requer a identificação de muitos pontos de controle. A detecção automática de padrões de calibração é fundamental para a automatização desse processo. Os trabalhos existentes são poucos e implementam soluções semi-automáticas ou, quando automáticas, não lidam com imagens distorcidas, com inclinações acentuadas ou exigem a detecção de todos os pontos apresentados pelo padrão de calibração. O presente trabalho vem contribuir com essa área propondo uma metodologia, denominada Chessboard Topological Detection (ChTD), completamente automática, aplicável a imagens com alta distorção, independente da detecção completa do padrão e que apresenta melhor aproveitamento dos pontos e do conjunto de imagens de calibração. A metodologia proposta é baseada em três etapas: a detecção estrutural dos pontos de interesse; triangulação e filtragem dos pontos encontrados com base em regras topológicas; e a propagação e ajuste de coordenadas via conectividade dos pontos vizinhos. A partir dessa metodologia foi implementada uma ferramenta que permite extrair pontos de calibração de um padrão xadrez de forma simples e completamente automática. Na avaliação do método ChTD foi realizada uma comparação com a função findChessBoardCorners da biblioteca OpenCV usando conjuntos de imagens reais e sintéticas. Os testes realizados com imagens reais possibilitaram comparações numéricas da quantidade de pontos e de padrões detectados, e os com imagens sintéticas permitiram avaliar o ChTD diante de inclinações do padrão e ruídos controlados. Pelos resultados obtidos foi possível verificar que o ChTD foi superior ao método do OpenCV, apresentando menor dependência à inclinação do padrão, melhor aproveitamento dos pontos e das imagens de calibração e realizando a detecção em imagens distorcidas. O ChTD é executado em um único passo, diferenciando-se do método do OpenCV que faz transformações sucessivas da imagem. A metodologia desenvolvida é modularizada, possibilitando o uso de outros algoritmos em suas fases intermediárias. / The detection of calibration points is reported as a time consuming task in camera calibration systems. This task is sensitive to noise and usually requires identification of a large set of control points. A methodology of automatic detection of calibration patterns is essential for the automation of this process. Existing works are few and usually implement semi-automatic solutions. Automatic methods do not deal with distorted images, and patterns with high tilt angles or require the detection of all the calibration points. The present work aims to contribute to this area proposing a methodology named Chessboard Topological Detection (ChTD), which is completely automatic, applicable to images with high distortion, making better use of all detected points, avoind calibration frames losses, regardless of the complete detection of the pattern features. The proposed methodology is based on three steps: the structural detection of points of interest, filtering and triangulation of the points based on topological rules and the adjust and coordinate propagation via connectivity of neighboring points inside the mesh. Based on this methodology, was implemented a tool that allows to extract calibration points of a chessboard pattern in a simple and automatic way. For the evaluation of the method ChTD, a comparison was made with the function findChessBoardCorners, which belongs to OpenCV library, using sets of synthetic and real images. Tests performed with real images allowed numerical comparisons of the number of points and detected patterns. The tests with synthetic images allowed to evaluate the ChTD facing the controlled tilt angle of the chessboard pattern and controlled image noise. The obtained results showed that the ChTD method was superior to OpenCV, with less dependence on the tilt angle of the calibration pattern, detecting more points, avoiding calibration images losses and detecting the pattern in distorted images. The ChTD runs in a single step, differing from OpenCV method that makes successive transformations of the image. The developed methodology is modular, enabling the use of different algorithms in its intermediates and some principal stages.

The NASA Airborne Topographic Mapper (ATM) is a laser scanning instrument which is used mainly to collect dense topographic data over much of the conterminous US coastline. The inclusion of two digital cameras in consonance with the ATM instrument now gives 3-band (RGB) imagery apart from a very rich topographic data. This imagery, in its crude form, has limited applications due to it being not georeferenced and having a heavy camera lens distortions. As thesis, a processing system - Park-View - is developed to bring this imagery into a more suitable format for the scientists for analytical and interpretational purposes. Park-View utilizes the well gridded elevation data from layer four of another processing system called LaserMap for georeferencing the digital camera images. Camera lens behavior is modeled using a 2D grid image and all of its intrinsic parameters ascertained. These parameters are then incorporated into correcting the lens distortions of georeferenced images. Errors in time-stamping of images and in the mounting angles of the camera are calculated using well known tie-points. Georeferenced images can be stored either in GeoTiff format or jpeg format. Individual images can be georeferenced or put in a mosaic form with the mosaic color equalized for adjoining images. Park-View also provides the main GUI displaying the entire surveyed area, mapper GUI for a batch processing of all the images and a display window for displaying georeferenced images or mosaics. Additional capabilities could be added to the processing system for performing some specific image processing operations on the images such as edge detection and image enhancement.