With the advancement in deep learning in the past few years, we are able to create complex machine learning models for detecting objects in images, regardless of the characteristics of the objects to be detected. This development has enabled engineers to replace existing heuristics-based systems in favour of machine learning models with superior performance. In this report, we evaluate the viability of using deep learning models for object detection in real-time video feeds on mobile devices in terms of object detection performance and inference delay as either an end-to-end system or feature extractor for existing algorithms. Our results show a significant increase in object detection performance in comparison to existing algorithms with the use of transfer learning on neural networks adapted for mobile use. / Utvecklingen inom djuplärning de senaste åren innebär att vi är kapabla att skapa mer komplexa maskininlärningsmodeller för att identifiera objekt i bilder, oavsett objektens attribut eller karaktär. Denna utveckling har möjliggjort forskare att ersätta existerande heuristikbaserade algoritmer med maskininlärningsmodeller med överlägsen prestanda. Den här rapporten syftar till att utvärdera användandet av djuplärningsmodeller för exekvering av objektigenkänning i video på mobila enheter med avseende på prestanda och exekveringstid. Våra resultat visar på en signifikant ökning i prestanda relativt befintliga heuristikbaserade algoritmer vid användning  av djuplärning och överförningsinlärning i artificiella neurala nätverk.

Commercialization potential is important to data science. Whether the problems encountered by data science in production can be solved determines the success or failure of the commercialization of data science. Recent research shows that DevOps theory is a great approach to solve the problems that software engineering encounters in production. And from the product perspective, data science and software engineering both need to provide digital services to customers. Therefore it is necessary to study the feasibility of applying DevOps in data science. This paper describes an approach of developing a delivery pipeline line for a data science system applying DevOps practices. I applied four practices in the pipeline: version control, model server, containerization, and continuous integration and delivery. However, DevOps is not a theory designed specifically for data science. This means the currently available DevOps practices cannot cover all the problems of data science in production. I expended the set of practices of DevOps to handle that kind of problem with a practice of data science. I studied and involved transfer learning in the thesis project. This paper describes an approach of parameterbased transfer where parameters learned from one dataset are transferred to another dataset. I studied the effect of transfer learning on model fitting to a new dataset. First I trained a convolutional neural network based on 10,000 images. Then I experimented with the trained model on another 10,000 images. I retrained the model in three ways: training from scratch, loading the trained weights and freezing the convolutional layers. The result shows that for the problem of image classification when the dataset changes but is similar to the old one, transfer learning a useful practice to adjust the model without retraining from scratch. Freezing the convolutional layer is a good choice if the new model just needs to achieve a similar level of performance as the old one. Loading weights is a better choice if the new model needs to achieve better performance than the original one. In conclusion, there is no need to be limited by the set of existing practices of DevOps when we apply DevOps to data science. / Kommersialiseringspotentialen är viktig för datavetenskapen. Huruvida de problem som datavetenskapen möter i produktionen kan lösas avgör framgången eller misslyckandet med kommersialiseringen av datavetenskap. Ny forskning visar att DevOps-teorin är ett bra tillvägagångssätt för att lösa de problem som programvaruteknik möter i produktionen. Och ur produktperspektivet behöver både datavetenskap och programvaruteknik tillhandahålla digitala tjänster till kunderna. Därför är det nödvändigt att studera genomförbarheten av att tillämpa DevOps inom datavetenskap. Denna artikel beskriver en metod för att utveckla en leverans pipeline för ett datavetenskapssystem som använder DevOps-metoder. Jag använde fyra metoder i pipeline: versionskontroll, modellserver, containerisering och kontinuerlig integration och leverans. DevOps är dock inte en teori som utformats specifikt för datavetenskap. Detta innebär att de för närvarande tillgängliga DevOps-metoderna inte kan täcka alla problem med datavetenskap i produktionen. Jag spenderade uppsättningen av DevOps för att hantera den typen av problem med en datavetenskap. Jag studerade och involverade överföringslärande i avhandlingsprojektet. I det här dokumentet beskrivs en metod för parameterbaserad överföring där parametrar lärda från en datasats överförs till en annan datasats. Jag studerade effekten av överföringsinlärning på modellanpassning till ett nytt datasystem. Först utbildade jag ett invecklat neuralt nätverk baserat på 10 000 bilder. Sedan experimenterade jag med den tränade modellen på ytterligare 10 000 bilder. Jag omskolade modellen på tre sätt: träna från grunden, ladda de tränade vikterna och frysa de invändiga lagren. Resultatet visar att för problemet med bildklassificering när datasättet ändras men liknar det gamla, överföra lärande en användbar praxis för att justera modellen utan omskolning från början. Att frysa det invändiga lagret är ett bra val om den nya modellen bara behöver uppnå en liknande prestanda som den gamla. Att ladda vikter är ett bättre val om den nya modellen behöver uppnå bättre prestanda än den ursprungliga. Sammanfattningsvis finns det inget behov att begränsas av uppsättningen av befintliga metoder för DevOps när vi tillämpar DevOps på datavetenskap.

In recent years, the state-of-the-art in computer vision has improved immensely due to increased use of convolutional neural networks (CNN). However, the best-performing models are typically complex and too slow or too large for mobile use. We investigate whether the power of these large models can be transferred to smaller models and used in mobile applications. A small CNN model was designed based on VGG Net. Using transfer learning, three pre-trained ImageNet networks were tuned to perform hand-drawn image classification. The models were evaluated on their predictive power and the best model was compressed to the small CNN model using knowledge distillation, a flavor of model compression. We found a small but significant improvement in classification performance compared to training the small CNN model directly on training data. No such improvement was found in localization abilities. We claim that model compression, and knowledge distillation in particular, presents a valuable tool for mobile deep learning development. / De senaste åren har system för datorseende markant förbättrats, genom användning av djupa faltningsnäterk (‘‘convolutional neural network’’ - CNN). De bästa modellerna är dock komplexa och för långsamma eller för stora för användning på mobila enheter. Vi undersöker huruvida styrkan i dessa stora modeller kan överföras till mindre modeller för mobila applikationer. En liten CNN-modell designades baserat på VGG Net. Genom användning av transfer learning justerades tre ImageNet-modeller till att klassificera handskrivna dokument. Modellerna evaluerades på deras förmåga att kategorisera innehållet. Den bästa modellen komprimerades sedan till den mindre modellen genom modellkomprimering, mer specifikt en teknik kallad knowledge distillation. Vi fann en liten men signifikant förbättring av den lilla modellens förmåga att kategorisera innehållet, jämfört med att träna modellen direkt på data. Någon sådan förbättring upptäcktes dock inte för lokalisering av objekt. Vi påstår att modellkomprimering, och speciellt knowledge distillation, kan vara ett värdefullt verktyg för utveckling av djupa neurala nätverk för mobila applikationer.

This thesis empirically studies transfer learning as a calibration framework for Convolutional Neural Network (CNN) based appearance-based gaze estimation models. A dataset of approximately 1,900,000 eyestripe images distributed over 1682 subjects is used to train and evaluate several gaze estimation models. Each model is initially trained on the training data resulting in generic gaze models. The models are subsequently calibrated for each test subject, using the subject's calibration data, by applying transfer learning through network fine-tuning on the final layers of the network. Transfer learning is observed to reduce the Euclidean distance error of the generic models within the range of 12-21%, which is in line with current state-of-the-art. The best performing calibrated model shows a mean error of 29.53mm and a median error of 22.77mm. However, calibrating heatmap output-based gaze estimation models decreases the performance over the generic models. It is concluded that transfer learning is a viable calibration framework for improving the performance of CNN-based appearance based gaze estimation models. / Detta examensarbete är en empirisk studie på överföringsträning som ramverk för kalibrering av neurala faltningsnätverks (CNN)-baserade bildbaserad blickapproximationsmodeller. En datamängd på omkring 1 900 000 ögonrandsbilder fördelat över 1682 personer används för att träna och bedöma flertalet blickapproximationsmodeller. Varje modell tränas inledningsvis på all träningsdata, vilket resulterar i generiska modeller. Modellerna kalibreras därefter för vardera testperson med testpersonens kalibreringsdata via överföringsträning genom anpassning av de sista lagren av nätverket. Med överföringsträning observeras en minskning av felet mätt som eukilidskt avstånd för de generiska modellerna inom 12-21%, vilket motsvarar de bästa nuvarande modellerna. För den bäst presterande kalibrerade modellen uppmäts medelfelet 29,53mm och medianfelet 22,77mm. Dock leder kalibrering av regionella sannolikhetsbaserade blickapproximationsmodeller till en försämring av prestanda jämfört med de generiska modellerna. Slutsatsen är att överföringsträning är en legitim kalibreringsansats för att förbättra prestanda hos CNN-baserade bildbaserad blickapproximationsmodeller.

Strawberry is an important cash crop in California, and its supply accounts for 80% of the US market [2]. However, in current practice, strawberries are picked manually, which is very labor-intensive and time-consuming. In addition, the farmers need to hire an appropriate number of laborers to harvest the berries based on the estimated volume. When overestimating the yield, it will cause a waste of human resources, while underestimating the yield will cause the loss of the strawberry harvest [3]. Therefore, accurately estimating harvest volume in the field is important to farmers. This paper focuses on an image-based solution to detect strawberries in the field by using the traditional computer vision technique and deep learning method. When strawberries are in different growth stages, there are considerable differences in their color. Therefore, various color spaces are first studied in this work, and the most effective color components are used in detecting strawberries and differentiating mature and immature strawberries. In some color channels such as the R color channel from the RGB color model, Hue color channel from the HSV color model, 'a' color channel from the Lab color model, the pixels belonging to ripe strawberries are clearly distinguished from the background pixels. Thus, the color-based K-mean cluster algorithm to detect red strawberries will be exploited. Finally, it achieves a 90.5% truth-positive rate for detecting red strawberries. For detecting the unripe strawberry, this thesis first trained the Support Vector Machine classifier based on the HOG feature. After optimizing the classifier through hard negative mining, the truth-positive rate reached 81.11%. Finally, when exploring the deep learning model, two detectors based on different pre-trained models were trained using TensorFlow Object Detection API with the acceleration of Amazon Web Services' GPU instance. When detecting in a single strawberry plant image, they have achieved truth-positive rates of 89.2% and 92.3%, respectively; while in the strawberry field image with multiple plants, they have reached 85.5% and 86.3%.

A brain tumor is detected and classified by biopsy that is conducted after the brain surgery. Advancement in technology and machine learning techniques could help radiologists in the diagnosis of tumors without any invasive measures. We utilized a deep learning-based approach to detect and classify the tumor into Meningioma, Glioma, Pituitary tumors. We used registration and segmentation-based skull stripping mechanism to remove the skull from the MRI images and the grab cut method to verify whether the skull stripped MRI masks retained the features of the tumor for accurate classification. In this research, we proposed a transfer learning based approach in conjunction with discriminative learning rates to perform the classification of brain tumors. The data set used is a 3064 T MRI images dataset that contains T1 flair MRI images. We achieved a classification accuracy of 98.83%, 96.26%, and 95.18% for training, validation, and test sets and an F1 score of 0.96 on the T1 Flair MRI dataset.

Laser powder bed fusion (LPBF) remains a predominately open-loop additive manufacturing process with minimal in-situ quality and process control. Some machines feature optical monitoring systems but lack automated analytical capabilities for real-time defect detection. Recent advances in machine learning (ML) and convolutional neural networks (CNN) present compelling solutions to analyze images in real-time and to develop in-situ monitoring. Approximately 30,000 selective laser melting (SLM) build images from 31 previous builds are gathered and labeled as either “okay” or “defect”. Then, 14 open-sourced CNN were trained using transfer learning to classify the SLM build images. These models were evaluated by F1 score and down selected to the top 3 models. The top 3 models were then retrained and evaluated using Dietterich’s 5x2 cross-validation and compared with pairwise student t-tests. The pairwise t-test results show no statistically significant difference in performance between VGG- 19, Xception, and InceptionResNet. All models are strong candidates for future development and refinement. Additional work addresses the entire model development process and establishes a foundation for future work. Collaborations with computer science students has produced an image pre-processing program to enhance as-taken SLM images. Other outcomes include initial work to overlay CAD layer images and preliminary hardware integration plan for the SLM machine. The results from this work have demonstrated the potential of an optical layer-wise image defect detection system when paired with a CNN.

<p>The prediction of chromatin accessibility for cancer and neuron cell lines using genomic sequences is quite challenging. Advances in machine learning and deep learning techniques allow such challenges to be addressed. This thesis investigates the use of both the transfer learning and the multi-task learning techniques. In particular, this research demonstrates the potential of transfer learning and multi-task learning in improving the prediction accu?racy for twenty-three cancer types in human and neuron cell lines. Three different network architectures are used: the Basset network, the network, and the DeepSEA network. In addition, two transfer learning techniques are also used. In the first technique data relevant to the desired prediction task is not used during the pre-training stage while the second technique includes limited data about the desired prediction task in the pre-training phase. The preferred performance evaluation metric used to evaluate the performance of the models was the AUPRC due to the numerous negative samples. Our results demonstrate an average improvement of 4% of the DeepSEA network in predicting all twenty-three cancer cell line types when using the first technique, a decrease of 0.42% when using the second technique, and an increase of 0.40% when using multi-task learning. Also, it had an average improvement of 3.09% when using the first technique, 1.16% when using the second technique and 4.60% for the multi-task learning when predicting chromatin accessibility for the 14 neuron cell line types. The DanQ network had an average improvement of 1.18% using the first transfer learning technique, the second transfer learning technique showed an average decrease of 1.93% and also, a decrease of 0.90% for the multi-task learning technique when predicting for the different cancer cell line types. When predicting for the different neuron cell line types the DanQ had an average improvement of 1.56% using the first technique, 3.21% when using the second technique, and 5.35% for the multi-task learning techniques. The Basset network showed an average improvement of 2.93% using the first transfer learning technique and an average decrease of 0.02%, and 0.63% when using the second technique and multi-task learning technique respectively. Using the Basset network for prediction of chromatin accessibility in the different neuron types showed an average increase of 2.47%, 9 3.80% and 5.50% for the first transfer learning technique, second transfer learning technique and the multi-task learning technique respectively. The results show that the best technique for the cancer cell lines prediction is the first transfer learning model as it showed an improvement for all three network types, while the best technique for predicting chromatin accessibility in the neuron cell lines is the multi-task learning technique which showed the highest average improvement among all networks. The DeepSEA network showed the greatest improvement in performance among all techniques when predicting the different cancer cell line types. Also, it showed the greatest improvement when using the first transfer learning technique for predicting chromatin accessibility for neuron cell lines in the brain. The basset network showed the greatest improvement for the multi-task learning technique and the second transfer learning technique when predicting the accessibility for neuron cell lines. </p>

The influx of unmanned aerial systems over the last decade has increased need for airspace awareness. Monitoring solutions such as drone detection, tracking, and classification become increasingly important to maintain compliance for regulatory and security purposes, as well as for recognizing aircraft that may not be so. Vision systems offer significant size, weight, power, and cost (SWaP-C) advantages, which motivates exploration of algorithms to further aid with monitoring performance. A method to classify aircraft using vision systems to measure their motion characteristics is explored. It builds on the assumption that at least continuous visual detection or at most visual tracking of an object of interest is already accomplished. Monocular vision is in part limited by range/scale ambiguity, where range and scale information of an object projected onto the image plane of a camera using a pin- hole model is generally lost. In an indirect effort to attempt to recover scale information via identity, classification of aircraft can aid in improvement of. These measured motion characteristics can then be used to classify the perceived object based on its unique motion profile over time, using signal classification techniques. The study is not limited to just unmanned aircraft, but includes full scale aircraft in the simulated dataset used to provide a representative set of aircraft scale and motion. / Doctor of Philosophy / The influx of small drones over the last decade has increased need for airspace awareness to ensure they do not become a nuisance when operated by unqualified or ill-intentioned personnel. Monitoring airspace around locations where drone usage would be unwanted or a security issue is increasingly necessary, especially for more range and endurance capable fixed wing (airplane) drones. This work presents a solution utilizing a single camera to address the classification part of fixed wing drone monitoring, as cameras are extremely common, generally cheap, information rich sensors. Once an aircraft of interest is detected, classifying it can provide additional information regarding its intentions. It can also help improve visual detection and tracking performance since classification can help change expectations of where and how the aircraft may continue to travel. Most existing visual classification works rely on features visible on the aircraft itself or its silhouette shape. This work discusses an approach to classification by characterizing visually perceived motion of an aircraft as it flies through the air. The study is not limited to just drones, but includes full scale aircraft in the simulated dataset used. Video of an airplane is used to extract motion from each frame. This motion is condensed to and expressed as a single time signal, that is then classified using a neural network trained to recognize audio samples using a time-frequency representation called a spectrogram. This transfer learning approach with Resnet based spectrogram classification is able to achieve 90.9% precision on the simulated test set used.

This thesis consists of studies of two important problems in transfer learning: binary classification under covariate-shift transfer, and off-policy evaluation in reinforcement learning. First, the problem of binary classification under covariate shift is considered, for which the first efficient procedure for optimal pruning of a dyadic classification tree is presented, where optimality is derived with respect to a notion of 𝒂𝒗𝒆𝒓𝒂𝒈𝒆 𝒅𝒊𝒔𝒄𝒓𝒆𝒑𝒂𝒏𝒄𝒚 between the shifted marginal distributions of source and target. Further, it is demonstrated that the procedure is adaptive to the discrepancy between marginal distributions in a neighbourhood of the decision boundary. It is shown how this notion of average discrepancy can be viewed as a measure of 𝒓𝒆𝒍𝒂𝒕𝒊𝒗𝒆 𝒅𝒊𝒎𝒆𝒏𝒔𝒊𝒐𝒏 between distributions, as it relates to existing notions of information such as the Minkowski and Renyi dimensions. Experiments are carried out on real data to verify the efficacy of the pruning procedure as compared to other baseline methods for pruning under transfer. The problem of off-policy evaluation for reinforcement learning is then considered, where two minimax lower bounds for the mean-square error of off-policy evaluation under Markov decision processes are derived. The first of these gives a non-asymptotic lower bound for OPE in finite state and action spaces over a model in which the mean reward is perturbed arbitrarily (up to a given magnitude) that depends on an average weighted chi-square divergence between the behaviour and target policies. The second provides an asymptotic lower bound for OPE in continuous state-space when the mean reward and policy ratio functions lie in a certain smoothness class. Finally, the results of a study that purported to have derived a policy for sepsis treatment in ICUs are replicated and shown to suffer from excessive variance and therefore to be unreliable; our lower bound is computed and used as evidence that reliable off-policy estimation from this data would have required a great deal more samples than were available.