京都大学 / 新制・課程博士 / 博士(情報学) / 甲第23312号 / 情博第748号 / 新制||情||128(附属図書館) / 京都大学大学院情報学研究科知能情報学専攻 / (主査)准教授 飯山 将晃, 教授 西野 恒, 教授 中村 裕一, 教授 美濃 導彦(京都大学 名誉教授) / 学位規則第4条第1項該当 / Doctor of Informatics / Kyoto University / DFAM

The field of Computer Vision continues to be revolutionized by advances in Convolutional Neural Networks. These networks are well suited for the regular grid structure of image data. However, there are many irregular image types that do not fit within such a framework, such as multi-view images, spherical images, superpixel representations, and texture maps for 3D meshes. These kinds of representations usually have specially designed networks that only operate and train on that unique form of data, thus requiring large datasets for each data domain. This dissertation aims to bridge the gap between standard convolutional networks and specialized ones. It proposes selection-based convolution. This technique operates on graph representations, giving it the flexibility to represent many irregular image domains, but maintains the spatially-oriented nature of an image convolution. Thus, it is possible to train a network on standard images, then use those same network weights for any kind graph-based representation. The effectiveness of this technique is evaluated on image types such as spherical images and 3D meshes for tasks such as segmentation and style transfer. Improvements to the selection mechanism through various forms of interpolation are also presented. Finally, this work demonstrates the generality of selection and its ability to be applied to various forms of graph networks and graph data, not just those specific to the image domain.

For many years, light fields have been a unique way of capturing a scene. By using a particular set of optics, a light field camera is able to, in a single moment, take images of the same scene from multiple perspectives. These perspectives can be used to calculate the scene geometry and allow for effects not possible with standard photographs, such as refocus and the creation of novel views.Neural style transfer is the process of training a neural network to render photographs in the style of a particular painting or piece of art. This is a simple process for a single photograph, but naively applying style transfer to each view in a light field generates inconsistencies in coloring between views. Because of these inconsistencies, common light field effects break down.We propose a style transfer method for light fields that maintains consistencies between different views of the scene. This is done by using warping techniques based on the depth estimation of the scene. These warped images are then used to compare areas of similarity between views and incorporate differences into the loss function of the style transfer network. Additionally, this is done in a post-training fashion, which removes the need for a light field training set.

The field of computer vision has long pursued the challenge of understanding the three-dimensional world. This endeavour is further fuelled by the increasing demand for technologies that rely on accurate perception of the 3D environment such as autonomous driving and augmented reality. However, the labelled data scarcity in the 3D domain continues to be a hindrance to extensive research and development. Semi-Supervised Learning is a valuable tool to overcome data scarcity yet most of the state-of-art methods are primarily developed and tested for two-dimensional vision problems. To address this challenge, there is a need to explore innovative approaches that can bridge the gap between 2D and 3D domains. In this work, we propose a technique that both leverages the existing abundance of two-dimensional data and makes the state-of-art semi-supervised learning methods directly applicable to 3D tasks. Multi-View Meta Pseudo Labelling (MV-MPL) combines one of the best-performing architectures in 3D shape recognition, Multi-View Convolutional Neural Networks, together with the state-of-art semi-supervised method, Meta Pseudo Labelling. To evaluate the performance of MV-MPL, comprehensive experiments are conducted on widely used shape recognition benchmarks ModelNet40, ShapeNetCore-v1, and ShapeNetCore-v2, as well as, Objaverse-LVIS. The results demonstrate that MV-MPL achieves competitive accuracy compared to fully supervised models, even when only \(10%\) of the labels are available. Furthermore, the study reveals that the object descriptors extracted from the MV-MPL model exhibit strong performance on shape retrieval tasks, indicating the effectiveness of the approach beyond classification objectives. Further analysis includes the evaluation of MV-MPL under more restrained scenarios, the enhancements to the view aggregation and pseudo-labelling processes; and the exploration of the potential of employing multi-views as augmentations for semi-supervised learning. / Forskningsområdet för datorseende har länge strävat efter utmaningen att förstå den tredimensionella världen. Denna strävan drivs ytterligare av den ökande efterfrågan på teknologier som är beroende av en korrekt uppfattning av den tredimensionella miljön, såsom autonom körning och förstärkt verklighet. Dock fortsätter bristen på märkt data inom det tredimensionella området att vara ett hinder för omfattande forskning och utveckling. Halv-vägledd lärning (semi-supervised learning) framträder som ett värdefullt verktyg för att övervinna bristen på data, ändå är de flesta av de mest avancerade semisupervised-metoderna primärt utvecklade och testade för tvådimensionella problem inom datorseende. För att möta denna utmaning krävs det att utforska innovativa tillvägagångssätt som kan överbrygga klyftan mellan 2D- och 3D-domänerna. I detta arbete föreslår vi en teknik som både utnyttjar den befintliga överflöd av tvådimensionella data och gör det möjligt att direkt tillämpa de mest avancerade semisupervised-lärandemetoderna på 3D-uppgifter. Multi-View Meta Pseudo Labelling (MV-MPL) kombinerar en av de bästa arkitekturerna för 3D-formigenkänning, Multi-View Convolutional Neural Networks, tillsammans med den mest avancerade semisupervised-metoden, Meta Pseudo Labelling. För att utvärdera prestandan hos MV-MPL genomförs omfattande experiment på väl använda uvärderingar för formigenkänning., ModelNet40, ShapeNetCore-v1 och ShapeNetCore-v2. Resultaten visar att MV-MPL uppnår konkurrenskraftig noggrannhet jämfört med helt vägledda modeller, även när endast \(10%\) av etiketterna är tillgängliga. Dessutom visar studien att objektbeskrivningarna som extraherats från MV-MPL-modellen uppvisar en stark prestanda i formåterhämtningsuppgifter, vilket indikerar effektiviteten hos tillvägagångssättet bortom klassificeringsmål. Vidare analys inkluderar utvärderingen av MV-MPL under mer begränsade scenarier, förbättringar av vyaggregerings- och pseudomärkningsprocesserna samt utforskning av potentialen att använda bilder från flera vinklar som en metod att få mer data för halv-vägledd lärande.

No description available.

No description available.

How to handle large multi-dimensional datasets such as hyperspectral images and video information both efficiently and effectively plays an important role in big-data processing. The characteristics of tensor low-rank decomposition in recent years demonstrate the importance of capturing the tensor structure adequately which usually yields efficacious approaches. In this dissertation, we first aim to explore the tensor singular value decomposition (t-SVD) with the nonconvex regularization on the multi-view subspace clustering (MSC) problem, then develop two new tensor decomposition models with the Bayesian inference framework on the tensor completion and tensor robust principal component analysis (TRPCA) and tensor completion (TC) problems. Specifically, the following developments for multi-dimensional datasets under the mathematical tensor framework will be addressed. (1) By utilizing the t-SVD proposed by Kilmer et al. \cite{kilmer2013third}, we unify the Hyper-Laplacian (HL) and exclusive $\ell_{2,1}$ (L21) regularization with Tensor Log-Determinant Rank Minimization (TLD) to identify data clusters from the multiple views' inherent information. Whereby the HL regularization maintains the local geometrical structure that makes the estimation prune to nonlinearities, and the mixed $\ell_{2,1}$ and $\ell_{1,2}$ regularization provides the joint sparsity within-cluster as well as the exclusive sparsity between-cluster. Furthermore, a log-determinant function is used as a tighter tensor rank approximation to discriminate the dimension of features. (2) By considering a tube as an atom of a third-order tensor and constructing a data-driven learning dictionary from the observed noisy data along the tubes of a tensor, we develop a Bayesian dictionary learning model with tensor tubal transformed factorization to identify the underlying low-tubal-rank structure of the tensor substantially with the data-adaptive dictionary for the TRPCA problem. With the defined page-wise operators, an efficient variational Bayesian dictionary learning algorithm is established for TPRCA that enables to update of the posterior distributions along the third dimension simultaneously. (3) With the defined matrix outer product into the tensor decomposition process, we present a new decomposition model for a third-order tensor. The fundamental idea is to decompose tensors mathematically in a compact manner as much as possible. By incorporating the framework of Bayesian probabilistic inference, the new tensor decomposition model on the subtle matrix outer product (BPMOP) is developed for the TC and TRPCA problems. Extensive experiments on synthetic data and real-world datasets are conducted for the multi-view clustering, TC, and TRPCA problems to demonstrate the desirable effectiveness of the proposed approaches, by detailed comparison with currently available results in the literature.

<p>Video super-resolution for dual-mode cameras in single-view and mono-view scenarios is studied in this thesis. Dual-mode cameras are capable of generating high-resolution still images while shooting video sequences at low-resolution. High-resolution still images are used to form a regularization function for solving the inverse problem of super-resolution. Exploiting proposed regularization function in this thesis obviates the need for classic regularization function. Experimental results show that using proposed regularization function instead of classic regularization functions for super-resolution of single-view video leads to improved results. In this thesis, super-resolution problem is divided into low-resolution frame fusion and de-blurring. A frame fusion scheme for multi-view video is proposed and performance improvement when exploiting multi-view sequence instead of single-view for frame fusion is studied. Experimental results show that information taken by a set of cameras instead of a single camera can improve super-resolution process, especially when video contains fast motions. As a side work, we applied our low-resolution multi-view frame fusion algorithm to 3D frame-compatible format resolution enhancement. Multi-view video super-resolution using high-resolution still images is performed at the decoder to prevent increasing computation complexity of the encoder. Experimental results show that this method delivers comparable compression efficiency for lower bit-rates.</p> / Master of Applied Science (MASc)

No description available.

Tesis por compendio / [ES] Esta tesis se centra en el desarrollo de nuevas técnicas automatizadas que permiten inspeccionar nematodos Caenorhabidits elegans (C. elegans) en placas de Petri estándar, para el análisis de sus comportamientos. C. elegans es un nemátodo de 1mm de longitud, con el cual se pueden realizar distintos experimentos para analizar los efectos de fármacos, compuestos o alteraciones genéticas en su longevidad, su salud física o su cognición. El campo principal metodológico del presente trabajo para el análisis de esos efectos es la visión por computador; y con ello, el desarrollo completo del sistema de visión activo: sistema de iluminación inteligente, sistema de captura óptimo, procesamiento de las imágenes para detección y clasificación de nematodos. Los campos secundarios en esta investigación son el control y robotización. Los C. elegans son animales sensibles a la luz y por ello el primero de los métodos está en la rama de la iluminación inteligente, con el cual se permite regular la intensidad y las longitudes de onda de la luz que reciben los nematodos. El siguiente método es el procesado para la detección y clasificación de movimiento a partir de las imágenes obtenidas con esa iluminación controlada. Tener el ambiente controlado es fundamental, los nematodos son muy sensibles a las condiciones ambientales por lo que puede alterarse su actividad biológica, y con ello los resultados, así que el tercer método es la integración de las técnicas en un nuevo dispositivo que permite automatizar ensayos de lifespan y validar los resultados automáticos comparándolos con los manuales. El movimiento del animal es clave para poder realizar inferencias estadísticas que puedan mostrar tendencias en sus comportamientos, por ello la estimulación automatizada que provoque una reacción de su movilidad es el cuarto de los métodos. Por último, el aumento de la resolución en las imágenes muestra mayor detalle, mejorando el procesamiento y extracción de características. El quinto método es un robot multivista que posibilita tomar imágenes a distintas resoluciones, lo que permite mantener el seguimiento global de los gusanos, al mismo tiempo que se toman imágenes con un encuadre de mayor detalle del nematodo objetivo. / [CA] Esta tesi doctoral se centra en el desentrollament de noves tècniques automatitzades que permeten inspeccionar nemàtodes Caenorhabidits elegans (C. elegans) en plaques de Petri estàndar, per a l'anàlisi dels seus comportaments. C. elegans és un nemàtode d'1mm de llargària, ab el qual se poden realitzar distints experiments per a analitzar els efectes de fàrmacs, composts o alteracions genètiques en sa longevitat, la seua salut física o la seua cognició. El camp principal metodològic del present treball per a l'anàlisi d'eixos efectes és la visió per computador; i ab açò, el desentrollament complet del sistema de visió actiu: sistema d'il.luminació inteligent, sistema de captura òptim, processament de les imàtgens per a detecció i classificació de nematode. Els camps secundaris en esta investigació són el control i robotització. Els C. elegans són animals sensibles a la llum i por ello el primer dels mètodes està en la branca de la il.luminació intel.ligent, ab el qual es permet regular la intensitat i les longituds d'ona de la llum que reben els nematodes. El següent mètode és el processat per a la detecció i classificació de moviment a partir de les imàtgens obtinguda ab eixa il.luminació controlada. Tindre l'ambient controlat és fonamental, els nemàtodes són molt sensibles a les condicions ambientals per lo que pot alterar-se la seua activitat biològica, i ab aço els resultats, aixina que el tercer mètode és la integració de les tècniques en un nou dispositiu que permet automatitzar ensajos de lifespan i validar els resultats automàtics comparant-los ab els manuals. El moviment de l'animal és clau per a poder realitzar inferencies estadístiques que puguen mostrar tendències en el seus comportaments, per això la estimulació automatitzada que provoque una reacció de la seua mobilitat és el quart dels mètodes. Per últim, l'augment de la resolució en les imàtgens mostra major detall, millorant el processament i extracció de característiques. El quint mètode és un robot multivista que possibilita prendre imàtgens a distintes resolucions, lo que permet mantindre el seguiment global dels cucs, al mateix temps que se prenguen imàtgens ab un enquadrament de major detall del nematode objectiu. / [EN] This thesis focuses on the development of new automated techniques that allow the inspection of Caenorhabidits elegans nematodes (C. elegans) in Petri dishes, for the analysis of their behavior. This nematode is a 1mm long worm, with which different experiments can be carried out to analyze the effects of drugs, compounds or genetic alterations on its longevity, physical health or cognition. The main methodological field of the present work for the analysis of these effects is computer vision; and with it, the complete development of the active vision system: intelligent lighting system, optimal capture system, image processing for detection and classification of nematodes. The secondary fields in this research are control and robotization. C. elegans are light-sensitive animals and therefore the first method is in the field of intelligent lighting, with which it is possible to regulate the intensity and wavelength of the light that nematodes receive. The next method is the processing for the detection and classification of movement from the images obtained with that controlled lighting. Having a controlled environment is essential, worms are very sensitive to environmental conditions so it can alter biological activity, and with it the results, so the third method is the integration of techniques in a new device that allows automating tests of lifespan and validate the automatic results comparing them with the manual ones. The movement of the animal is key to be able to carry out statistical conferences that can show trends in its behaviors, therefore the automated stimulation that causes a reaction of its mobility is the fourth of the methods. Finally, increasing the resolution in the images shows greater detail, improving the processing and extraction of features. The fifth method is a multiview robot that enables images to be taken at different resolutions, allowing global tracking of worms to be maintained, while at the same time taking images with a more detailed frame of the target worm. / Puchalt Rodríguez, JC. (2022). Development and Evaluation of New Methods for Automating Experiments with C. Elegans Based on Active Vision [Tesis doctoral]. Universitat Politècnica de València. https://doi.org/10.4995/Thesis/10251/181359 / TESIS / Compendio