Neural networks have long been known as universal function approximators and have more recently been shown to be powerful and versatile in practice. But it can be extremely challenging to find the right set of parameters and hyperparameters. Model training is both expensive and difficult due to the large number of parameters and sensitivity to hyperparameters such as learning rate and architecture. Hyperparameter searches are notorious for requiring tremendous amounts of processing power and human resources. This thesis provides an analytic approach to estimating the optimal value of one of the key hyperparameters in neural networks, the learning rate. Where possible, the analysis is computed exactly, and where necessary, approximations and assumptions are used and justified. The result is a method that estimates the optimal learning rate for a certain type of network, a fully connected CReLU network.

In recent years, deep learning based computer vision technology has developed rapidly. This is not only due to the improvement of computing power, but also due to the emergence of high-quality datasets. The combination of object detectors and drones has great potential in the field of rescue and disaster relief. We created an image dataset specifically for vision applications on drone platforms. The dataset contains 5000 images, and each image is carefully labeled according to the PASCAL VOC standard. This specific dataset will be very important for developing deep learning algorithms for drone applications. In object detection models, loss function plays a vital role. Considering the uneven distribution of large and small objects in the dataset, we propose adjustment coefficients based on the frequencies of objects of different sizes to adjust the loss function, and finally improve the accuracy of the model.

No description available.

In the most recent years, many deep learning models have been used to identify and classify diseases of plant leaves by inputting plant leaf images as input to the model. However, there is still a gap in research on how to improve the accuracy of the deep learning models of plant leaf diseases. This thesis is about investigating various techniques for improving the MobileNetV2 model's accuracy for plant disease detection in leaves and classification. These techniques involved adjusting the learning rate, adding additional layers, and various data-augmented operations. The results of this thesis have shown that these techniques can significantly improve the accuracy of the model, and the best results can be achieved by using random rotation and crop data augmentation. After adding random rotation and crop data augmentation to the model, it achieved an accuracy of 94%, a precision of 91%, a recall of 96%, and an F1-score of 95%. This shows that the proposed techniques can be used to improve the accuracy of plant leaf disease detection and classification models, which can help farmers identify and treat plant diseases.

Understanding the neural processes underlying motor learning in humans is important to facilitate the acquisition of new motor skills and to aid the relearning of skills lost after neurologic injury. Although it is known that the learning of a new movement is guided by the error feedback received after each repeated attempt to produce the movement, how the central nervous system (CNS) processes individual errors and how it modulates its learning rate in response to the history of errors experienced are issues that remain to be elucidated. To address these issues we studied the generalization of learning and learning decay – the transfer of what has been learned, or unlearned, in a particular movement condition to new movement conditions. Generalization offers a window into the process of error credit assignment during motor learning, since it allows us to measure which actions benefit the most in terms of learning after experiencing an error. We found that the distributions that describe generalization after learning are unimodal and biased towards the motion directions experienced during training, a finding that suggests that the credit for the learning experienced after a particular trial is assigned to the actual motion (motion-referenced learning) and not to the planned motion (plan-referenced learning) as it had previously been assumed in the motor learning literature. In addition, after training the same action along multiple directions, we found that the pattern of learning decay has two distinct components: one that is time-dependent and affects all trained directions, and one that is trial-dependent and affects mostly the direction where decay was induced, generalizing narrowly with a unimodal pattern similar to the one observed for learning generalization. We finally studied the effect that the consistency of the error perturbations in the training environment has on the learning rate adopted by the CNS. We found that learning rate increases when the perturbations experienced in training are consistent, and decreases when these perturbations are inconsistent. Besides increasing our understanding of the mechanisms underlying motor learning, the findings described in the present dissertation will enable the principled design of skill training and rehabilitation protocols that accelerate learning. / Engineering and Applied Sciences

Medical imaging is a key tool used in healthcare to diagnose and prognose patients by aiding the detection of a variety of diseases and conditions. In practice, medical image screening must be performed by clinical practitioners who rely primarily on their expertise and experience for disease diagnosis. The ability of convolutional neural networks (CNNs) to extract hierarchical features and determine classifications directly from raw image data makes CNNs a potentially useful adjunct to the medical image analysis process. A common challenge in successfully implementing CNNs is optimizing hyperparameters for training. In this study, we propose a method which utilizes scheduled hyperparameters and Bayesian optimization to classify cancerous and noncancerous tissues (i.e., segmentation) from head and neck computed tomography (CT) and positron emission tomography (PET) scans. The results of this method are compared using CT imaging with and without PET imaging for 2D and 3D image segmentation models.

Melanoma is a life threatening type of skin cancer with numerous fatal incidences all over the world. The 5-year survival rate is very high for cases that are diagnosed in early stage. So, early detection of melanoma is of vital importance. Except for several techniques that clinicians apply so as to improve the reliability of detecting melanoma, many automated algorithms and mobile applications have been developed for the same purpose.In this paper, deep learning model designed from scratch as well as the pretrained models Inception v3 and VGG-16 are used with the aim of developing a reliable tool that can be used for melanoma detection by clinicians and individual users. Dermatologists who use dermoscopes can take advantage of the algorithms trained on dermoscopical images and acquire a confirmation about their diagnosis. On the other hand, the models trained on clinical images can be used on mobile applications, since a cell phone camera takes images similar to them.The results using Inception v3 model for dermoscopical images achieved accuracy 91.4%, sensitivity 87.8% and specificity 92.3%. For clinical images, the VGG-16 model achieved accuracy 86.3%, sensitivity 84.5% and specificity 88.8%. The results are compared to those of clinicians, which shows that the algorithms can be used reliably for the detection of melanoma.

No description available.

Type-2 fuzzy logic system (FLS) cascaded with neural network, called type-2 fuzzy neural network (T2FNN), is presented in this paper to handle uncertainty with dynamical optimal learning. A T2FNN consists of type-2 fuzzy linguistic process as the antecedent part and the two-layer interval neural network as the consequent part. A general T2FNN is computational intensive due to the complexity of type 2 to type 1 reduction. Therefore the interval T2FNN is adopted in this paper to simplify the computational process. The dynamical optimal training algorithm for the two-layer consequent part of interval T2FNN is first developed. The stable and optimal left and right learning rates for the interval neural network, in the sense of maximum error reduction, can be derived for each iteration in the training process (back propagation). It can also be shown both learning rates can not be both negative. Further, due to variation of the initial MF parameters, i.e. the spread level of uncertain means or deviations of interval Gaussian MFs, the performance of back propagation training process may be affected. To achieve better total performance, a genetic algorithm (GA) is designed to search better-fit spread rate for uncertain means and near optimal learnings for the antecedent part. Several examples are fully illustrated. Excellent results are obtained for the truck backing-up control and the identification of nonlinear system, which yield more improved performance than those using type-1 FNN.

Most story-based games today have manually-scripted non-player characters (NPCs) and the scripts are usually simple and repetitive since it is time-consuming for game developers to script each character individually. ScriptEase, a publicly-available author-oriented developer tool, attempts to solve this problem by generating script code from high-level design patterns, for BioWare Corp.'s role-playing game Neverwinter Nights. The ALeRT algorithm uses reinforcement learning (RL) to automatically generate NPC behaviours that change over time as the NPCs learn from the successes or failures of their own actions. This thesis aims to provide a new learning mechanism to game agents so they are capable of adapting to new behaviours based on the actions of other agents. The new on-line RL algorithm, ALeRT-AM, which includes an agent-modeling mechanism, is applied in a series of combat experiments in Neverwinter Nights and integrated into ScriptEase to produce adaptive behaviour patterns for NPCs.