Single-cell genomics (SCG) are methods for investigating heterogeneity between biological cells, among these is Smart-seq which sequences from RNA molecules. A more recent version of this method is Smart-seq3xpress which is currently in the process of being automated by the Sandberg lab at Karolinska Institutet. As part of this automated lab system, microwell plates are moved by a robot arm between molecular biology instuments. The purpose of this project was to create and integrate an image-based classification solution to validate the placement of these plates. This was done by building upon the VGG-16 convolutional neural network (CNN) model and specialising it through transfer learning to train models which classify microwell plate placement as correct or incorrect. These models were then integrated into the automated lab pipeline so that the system could self-correct or warn lab personnel of misplacement, removing the need for constant human supervision. / Enskild cellgenomik (eng. single-cell genomics) är metoder för att undersöka heterogenitet mellan biologiska celler, bland dessa metoder är Smart-seq vilken sekvenserar från RNA molekyler. En nyare version av denna metod är Smart-seq3xpress vilken nu håller på att automatiseras av Sandberglabbet vid Karolinska Institutet. Som del av detta automatiserade labbsystem förflyttas mikrobrunnplattor av en robotarm mellan molekylärbiologiska mätinstrument. Syftet med detta projekt var att skapa samt integrera en bildbaserad klassificeringslösning för att säkerställa placeringen av dessa plattor. Detta gjordes genom att bygga på djupinlärningsmodellen VGG-16 och specialisera den med överförd inlärning för att kunna träna modeller vilka klassificerar om mikrobrunnplattornas placeringar är korrekta eller inkorrekta. Sedan integrerades dessa modeller som en del av det automatiserade labbsystemet sådan att systemet kunde självkorrigera eller varna labbpersonal vid felplaceringar, och därmed ta bort behovet av konstant mänsklig tillsyn.

[pt] O segmento offshore de produção de petróleo é o principal produtor nacional desse insumo. Nesse contexto, inspeções submarinas são cruciais para a manutenção preventiva dos equipamentos, que permanecem toda a vida útil em ambiente oceânico. A partir dos dados de imagem e sensor coletados nessas inspeções, especialistas são capazes de prevenir e reparar eventuais danos. Tal processo é profundamente complexo, demorado e custoso, já que profissionais especializados têm que assistir a horas de vídeos atentos a detalhes. Neste cenário, o presente trabalho explora o uso de modelos de classificação de imagens projetados para auxiliar os especialistas a encontrarem o(s) evento(s) de interesse nos vídeos de inspeções submarinas. Esses modelos podem ser embarcados no ROV ou na plataforma para realizar inferência em tempo real, o que pode acelerar o ROV, diminuindo o tempo de inspeção e gerando uma grande redução nos custos de inspeção. No entanto, existem alguns desafios inerentes ao problema de classificação de imagens de inspeção submarina, tais como: dados rotulados balanceados são caros e escassos; presença de ruído entre os dados; alta variância intraclasse; e características físicas da água que geram certas especificidades nas imagens capturadas. Portanto, modelos supervisionados tradicionais podem não ser capazes de cumprir a tarefa. Motivado por esses desafios, busca-se solucionar o problema de classificação de imagens submarinas a partir da utilização de modelos que requerem menos supervisão durante o seu treinamento. Neste trabalho, são explorados os métodos DINO (Self-DIstillation with NO labels, auto-supervisionado) e uma nova versão multi-label proposta para o PAWS (Predicting View Assignments With Support Samples, semi-supervisionado), que chamamos de mPAWS (multi-label PAWS). Os modelos são avaliados com base em sua performance como extratores de features para o treinamento de um classificador simples, formado por uma camada densa. Nos experimentos realizados, para uma mesma arquitetura, se obteve uma performance que supera em 2.7 por cento o f1-score do equivalente supervisionado. / [en] The offshore oil production segment is the main national producer of this input. In this context, underwater inspections are crucial for the preventive maintenance of equipment, which remains in the ocean environment for its entire useful life. From the image and sensor data collected in these inspections,experts are able to prevent and repair damage. Such a process is deeply complex, time-consuming and costly, as specialized professionals have to watch hours of videos attentive to details. In this scenario, the present work explores the use of image classification models designed to help experts to find the event(s) of interest in under water inspection videos. These models can be embedded in the ROV or on the platform to perform real-time inference,which can speed up the ROV, monitor notification time, and greatly reduce verification costs. However, there are some challenges inherent to the problem of classification of images of armored submarines, such as: balanced labeled data are expensive and scarce; the presence of noise among the data; high intraclass variance; and some physical characteristics of the water that achieved certain specificities in the captured images. Therefore, traditional supervised models may not be able to fulfill the task. Motivated by these challenges, we seek to solve the underwater image classification problem using models that require less supervision during their training. In this work, they are explorers of the DINO methods (Self-Distillation with NO labels, self-supervised) anda new multi-label version proposed for PAWS (Predicting View AssignmentsWith Support Samples, semi-supervised), which we propose as mPAWS (multi-label PAWS). The models are evaluated based on their performance as features extractors for training a simple classifier, formed by a dense layer. In the experiments carried out, for the same architecture, a performance was obtained that exceeds by 2.7 percent the f1-score of the supervised equivalent.

Given the explosive growth of web pages on the Internet in the last decade, automatic classification and categorization of web pages have grown into an important task. This thesis sets out to evaluate whether or not methods for text and image analysis, which had not been evaluated for web page classification, could improve on the state-of-the-art methods in web page classification. In web page classification, there is no dataset that is used for benchmarking. Therefore, in order to make comparisons, baseline models are implemented. The methods implemented are Bidirectional Encoder Representations from Transformers (BERT) for text and EfficientNet B4 for images. This thesis also sets out to evaluate methods for combining knowledge from two models. The thesis concludes that the proposed methods do improve on the state-of-the- art methods in web page classification. The proposed methods achieve approximately 92% accuracy while the baselines achieve approximately 87%. The proposed methods and the baselines are shown to be different using McNemar’s test at a significance level 0.05. The thesis also concludes that weighted average of logits could be preferable to weighted average of probabilities; weighted average of logits could be a more robust method, although more research is needed. / Givet den explosiva tillväxten av webbsidor på Internet under det senaste decenniet har automatisk klassificering och kategorisering av webbsidor vuxit till en viktig uppgift. Denna avhandling syftar till att utvärdera huruvida nya metoder för text- och bildanalys, som inte hade utvärderats för klassificering av webbsidor, skulle kunna prestera bättre än de senaste metoderna som har använts i området. Inom webbsideklassificering finns det inget dataset som används för jämförelser. För att göra jämförelser implementeras därför referensmodeller. De nya metoderna som implementerats är Bidirectional Encoder Representations from Transformers (BERT) för text och EfficientNet B4 för bilder. Den här avhandlingen syftar också till att utvärdera metoder för att kombinera kunskap från två modeller. Avhandlingen drar slutsatsen att de nya metoderna presterar bättre än de senaste metoderna inom klassificering av webbsidor. De nya metoderna uppnår cirka 92% noggrannhet medan referensmodellerna uppnår cirka 87%. De nya metoderna och referensmodellerna visar sig vara olika med hjälp av McNemars test med en signifikansnivå av 0.05. Avhandlingen drar också slutsatsen att det viktat genomsnitt av logits skulle kunna vara att föredra framför viktat genomsnitt av sannolikheter; viktat genomsnitt av logits skulle kunna vara en mer robust metod men måste undersökas mer.

Vitiligo is a skin disease where the pigment cells that produce melanin die or stop functioning, which causes white patches to appear on the body. Although vitiligo is not considered a serious disease, there is a risk that something is wrong with a person's immune system. In recent years, the use of medical image processing techniques has grown, and research continues to develop new techniques for analysing and processing medical images. In many medical image classification tasks, deep convolutional neural network technology has proven its effectiveness, which means that it may also perform well in vitiligo classification. Our study uses four deep convolutional neural networks in order to classify images of vitiligo and normal skin. The architectures selected are VGG-19, ResNeXt101, InceptionResNetV2 and Inception V3. ROC and AUC metrics are used to assess each model's performance. In addition, the authors investigate the economic benefits that this technology may provide to the healthcare system and patients. To train and evaluate the CNN models, the authors used a dataset that contains 1341 images in total. Because the dataset is limited, 5-fold cross validation is also employed to improve the model's prediction. The results demonstrate that InceptionV3 achieves the best performance in the classification of vitiligo, with an AUC value of 0.9111, and InceptionResNetV2 has the lowest AUC value of 0.8560. / Vitiligo är en hudsjukdom där pigmentcellerna som producerar melanin dör eller slutar fungera, vilket får vita fläckar att dyka upp på kroppen. Även om Vitiligo inte betraktas som en allvarlig sjukdom, det finns fortfarande risk att något är fel på en persons immun. Under de senaste åren har användningen av medicinska bildbehandlingstekniker vuxit och forskning fortsätter att utveckla nya tekniker för att analysera och bearbeta medicinska bilder. I många medicinska bildklassificeringsuppgifter har djupa konvolutionella neurala nätverk bevisat sin effektivitet, vilket innebär att den också kan fungera bra i Vitiligo klassificering. Vår studie använder fyra djupa konvolutionella neurala nätverk för att klassificera bilder av vitiligo och normal hud. De valda arkitekturerna är VGG-19, RESNEXT101, InceptionResNetV2 och Inception V3. ROC- och AUC mätvärden används för att bedöma varje modells prestanda. Dessutom undersöker författarna de ekonomiska fördelarna som denna teknik kan ge till sjukvårdssystemet och patienterna. För att träna och utvärdera CNN modellerna använder vi ett dataset som innehåller totalt 1341 bilder. Eftersom datasetet är begränsat används också 5-faldigt korsvalidering för att förbättra modellens förutsägelse. Resultaten visar att InceptionV3 uppnår bästa prestanda i klassificeringen av Vitiligo, med ett AUC -värde på 0,9111, och InceptionResNetV2 har det lägsta AUC -värdet på 0,8560.

No description available.

<p>The computer vision task of classifying natural images is a primary driving force behind modern AI algorithms. Deep Convolutional Neural Networks (CNNs) demonstrate state of the art performance in large scale multi-class image classification tasks. However, due to the many layers and millions of parameters these models are considered to be black box algorithms. The decisions of these models are further obscured due to a cumbersome multi-class decision process. There exists another approach called class binarization in the literature which determines the multi-class prediction outcome through a sequence of binary decisions.The focus of this dissertation is on the integration of the class-binarization approach to multi-class classification with deep learning models, such as CNNs, for addressing large scale image classification problems. Three works are presented to address the integration.</p> <p>In the first work, Error Correcting Output Codes (ECOCs) are integrated into CNNs by inserting a latent-binarization layer prior to the CNNs final classification layer.  This approach encapsulates both encoding and decoding steps of ECOC into a single CNN architecture. EM and Gibbs sampling algorithms are combined with back-propagation to train CNN models with Latent Binarization (LB-CNN). The training process of LB-CNN guides the model to discover hidden relationships similar to the semantic relationships known apriori between the categories. The proposed models and algorithms are applied to several image recognition tasks, producing excellent results.</p> <p>In the second work, Hierarchically Decodeable Output Codes (HD-OCs) are proposedto compactly describe a hierarchical probabilistic binary decision process model over the features of a CNN. HD-OCs enforce more homogeneous assignments of the categories to the dichotomy labels. A novel concept called average decision depth is presented to quantify the average number of binary questions needed to classify an input. An HD-OC is trained using a hierarchical log-likelihood loss that is empirically shown to orient the output of the latent feature space to resemble the hierarchical structure described by the HD-OC. Experiments are conducted at several different scales of category labels. The experiments demonstrate strong performance and powerful insights into the decision process of the model.</p> <p>In the final work, the literature of enumerative combinatorics and partially ordered sets isused to establish a unifying framework of class-binarization methods under the Multivariate Bernoulli family of models. The unifying framework theoretically establishes simple relationships for transitioning between the different binarization approaches. Such relationships provide useful investigative tools for the discovery of statistical dependencies between large groups of categories. They are additionally useful for incorporating taxonomic information as well as enforcing structural model constraints. The unifying framework lays the groundwork for future theoretical and methodological work in addressing the fundamental issues of large scale multi-class classification.</p> <p><br></p>

Machine learning and AI are growing rapidly and they are being implemented more often than before due to their high accuracy and performance. One of the biggest challenges to machine learning is data collection. The training data is the most important part of any machine learning project since it determines how the trained model will behave. In the case of object classification and detection, capturing a large number of images per object is not always possible and can be a very time-consuming and tedious process. This thesis explores options specific to image classification that help reducing the need to capture many images per object while still keeping the same performance accuracy. In this thesis, experiments have been performed with the goal of achieving a high classification accuracy with a limited dataset. One method that is explored is to create artificial training images using a game engine. Ways to expand a small dataset such as different data augmentation methods, and regularization methods, are also employed. / Maskininlärning och AI växer snabbt och de implementeras allt oftare på grund av deras höga noggrannhet och prestanda. En av de största utmaningarna för maskininlärning är datainsamling. Träningsdata är den viktigaste delen av ett maskininlärningsprojekt eftersom den avgör hur den tränade modellen kommer att bete sig. När det gäller objektklassificering och detektering är det inte alltid möjligt att ta många bilder per objekt och det kan vara en process som kräver mycket tid och arbete. Det här examensarbetet utforskar alternativ som är specifika för bildklassificering som minskar behovet av att ta många bilder per objekt samtidigt som prestanda bibehålls. I det här examensarbetet, flera experiment har utförts med målet att uppnå en hög klassificeringsprestanda med en begränsad dataset. En metod som utforskas är att skapa träningsbilder med hjälp av en spelmotor. Metoder för att utöka antal bilder i ett litet dataset, som data augmenteringsmetoder och regleringsmetoder, används också.

<p dir="ltr">This dissertation delves into a significant challenge for Autonomous Vehicles (AVs): achieving efficient and robust perception under adverse weather and lighting conditions. Systems that rely solely on cameras face difficulties with visibility over long distances, while radar-only systems struggle to recognize features like stop signs, which are crucial for safe navigation in such scenarios.</p><p dir="ltr">To overcome this limitation, this research introduces a novel deep camera-radar fusion approach using neural networks. This method ensures reliable AV perception regardless of weather or lighting conditions. Cameras, similar to human vision, are adept at capturing rich semantic information, whereas radars can penetrate obstacles like fog and darkness, similar to X-ray vision.</p><p dir="ltr">The thesis presents NeXtFusion, an innovative and efficient camera-radar fusion network designed specifically for robust AV perception. Building on the efficient single-sensor NeXtDet neural network, NeXtFusion significantly enhances object detection accuracy and tracking. A notable feature of NeXtFusion is its attention module, which refines critical feature representation for object detection, minimizing information loss when processing data from both cameras and radars.</p><p dir="ltr">Extensive experiments conducted on large-scale datasets such as Argoverse, Microsoft COCO, and nuScenes thoroughly evaluate the capabilities of NeXtDet and NeXtFusion. The results show that NeXtFusion excels in detecting small and distant objects compared to existing methods. Notably, NeXtFusion achieves a state-of-the-art mAP score of 0.473 on the nuScenes validation set, outperforming competitors like OFT by 35.1% and MonoDIS by 9.5%.</p><p dir="ltr">NeXtFusion’s excellence extends beyond mAP scores. It also performs well in other crucial metrics, including mATE (0.449) and mAOE (0.534), highlighting its overall effectiveness in 3D object detection. Visualizations of real-world scenarios from the nuScenes dataset processed by NeXtFusion provide compelling evidence of its capability to handle diverse and challenging environments.</p>

<p>Deep ‘Analog Artificial Neural Networks’ (AANNs) perform complex classification problems with high accuracy. However, they rely on humongous amount of power to perform the calculations, veiling the accuracy benefits. The biological brain on the other hand is significantly more powerful than such networks and consumes orders of magnitude less power, indicating some conceptual mismatch. Given that the biological neurons are locally connected, communicate using energy efficient trains of spikes, and the behavior is non-deterministic, incorporating these effects in Artificial Neural Networks (ANNs) may drive us few steps towards a more realistic neural networks. </p> <p> </p> <p>Emerging devices can offer a plethora of benefits including power efficiency, faster operation, low area in a vast array of applications. For example, memristors and Magnetic Tunnel Junctions (MTJs) are suitable for high density, non-volatile Random Access Memories when compared with CMOS implementations. In this work, we analyze the possibility of harnessing the characteristics of such emerging devices, to achieve neuro-inspired solutions to intricate problems.</p> <p> </p> <p>We propose how the inherent stochasticity of nano-scale resistive devices can be utilized to realize the functionality of spiking neurons and synapses that can be incorporated in deep stochastic Spiking Neural Networks (SNN) for image classification problems. While ANNs mainly dwell in the aforementioned classification problem solving domain, they can be adapted for a variety of other applications. One such neuro-inspired solution is the Cellular Neural Network (CNN) based Boolean satisfiability solver. Boolean satisfiability (k-SAT) is an NP-complete (k≥3) problem that constitute one of the hardest classes of constraint satisfaction problems. We provide a proof of concept hardware based analog k-SAT solver that is built using MTJs. The inherent physics of MTJs, enhanced by device level modifications, is harnessed here to emulate the intricate dynamics of an analog, CNN based, satisfiability (SAT) solver. </p> <p> </p> <p>Furthermore, in the effort of reaching human level performance in terms of accuracy, increasing the complexity and size of ANNs is crucial. Efficient algorithms for evaluating neural network performance is of significant importance to improve the scalability of networks, in addition to designing hardware accelerators. We propose a scalable approach for evaluating Liquid State Machines: a bio-inspired computing model where the inputs are sparsely connected to a randomly interlinked reservoir (or liquid). It has been shown that biological neurons are more likely to be connected to other neurons in the close proximity, and tend to be disconnected as the neurons are spatially far apart. Inspired by this, we propose a group of locally connected neuron reservoirs, or an ensemble of liquids approach, for LSMs. We analyze how the segmentation of a single large liquid to create an ensemble of multiple smaller liquids affects the latency and accuracy of an LSM. In our analysis, we quantify the ability of the proposed ensemble approach to provide an improved representation of the input using the Separation Property (SP) and Approximation Property (AP). Our results illustrate that the ensemble approach enhances class discrimination (quantified as the ratio between the SP and AP), leading to improved accuracy in speech and image recognition tasks, when compared to a single large liquid. Furthermore, we obtain performance benefits in terms of improved inference time and reduced memory requirements, due to lower number of connections and the freedom to parallelize the liquid evaluation process.</p>

台灣位於環太平洋地震帶上，地形為山地居多，且地質脆弱，加上位於西太平洋副熱帶地區，使得山區常受到颱風的侵擾而發生崩塌，導致土石流和洪水等災害發生，進而影響人民的生命和財產安全。因此，如何有效地建置崩塌地區域資料庫，成為國土保育與災害防治的重要課題。以往利用遙測與航測技術於崩塌地萃取的研究中，大多是於幾何糾正後衛星影像或是航測正射影像上分析崩塌地，但產製正射影像或是糾正衛星影像時，都需要花費較多的時間，對於講求時效性的救災行動而言頗為不利。本研究之目的為發展一套不需使用正射影像萃取崩塌地的方法，以物件導向影像分類法，於DMC（Digital Mapping Camera）航測原始影像上直接萃取崩塌地資訊。首先採取多重解析影像分割的技術，將航測影像依像元光譜和形狀同質性分割成不同區塊（物件），接著利用影像光譜統計值搭配區域成長法，偵測影像中的雲覆蓋地區並過濾。其次，根據光譜亮度統計特徵值，將影像區分成陰暗地區、正常地區以及較亮地區之三種土地覆蓋類型，使用線性相關糾正法（Linear-correlation Correction）將陰暗地區光譜亮度值轉換至正常地區，並利用物件的特徵值，如光譜、面積、形狀以及相關性依序萃取此三種土地覆蓋類型內的崩塌地。最後，使用光線追蹤法 （Ray-tracing），將崩塌地區塊從影像坐標轉換至地圖坐標，使其可以套疊地形資料如坡度、坡向，並進行空間分析以提升崩塌地的判釋精度。研究結果顯示，崩塌地萃取之使用者精度和生產者精度，均有82%以上，並且整個實驗可大量批次處理影像，及快速建立崩塌地資料庫，本研究之方法和崩塌地資料庫將有助於國土保育與崩塌地的災害防治。 / Being located in a subtropical and seismic zone of the West Pacific, the geology is fragile and topography is mountainous in Taiwan. Landslides, floods and other disasters induced by typhoons occur frequently, and it cause the life-threatening and property loss of human beings in Taiwan. Therefore, how to establish landslides data effectively become an important issue of land conservation and disasters management. In recently years, most of the researchers used aerial ortho-images or satellite georeferencing images to detect landslides sites. However, it spent a lot of time generating aerial ortho-images and rectifying satellite images, and it also reduced the efficiency of landslides analysis. Thus, this study developed an object-oriented classification method, which can be directly applied in raw image data, to detect landslides sites. Firstly, this study used multi-resolution image segmentation technique to segment images acquired by Z/I DMC（Digital Mapping Camera） into individual regions （objects） according to the homogeneity of spectral and shape features, and then removed cloud areas by using brightness features depended on the spectral information of images. Secondly, the study divided the entire image into three areas, which are darker area, normal area and lighter area, according to brightness value. Next, Linear-correlation correction （LCC） method was used in this study to transform darker area to normal area so that it can easily detect the landslides sites in darker area, and the object features, such as spectral, area, shape and space correlation indices, were used to extract landslide sites in images. Finally, in order to enhance the accuracy of landslide, the initial landslides were converted from image coordinate system to map coordinate system by ray-tracing method, so the initial landslides data can be further extracted by using topographic data, including slope and aspect data. The results of this study showed that the user and producer accuracies of detecting landslides can reach up to 82%. Moreover, the entire experiments process of this study can batch analyze automatically and establish landslides database quickly. It is expected that the method and landslides data of this study may have contribution to land conservation and disasters management.