No description available.

When the novel coronavirus (COVID-19) outbreak began to ring alarm bells worldwide, rapid, efficient diagnosis was critical to the emergency response. The limited ability of medical systems and the increasing number of daily cases pushed researchers to investigate automated models. The use of deep neural networks to help doctors make the correct diagnosis has dramatically reduced the pressure on the healthcare system. Promoting the improvement of diagnosis networks depends not only on the network structure design but also on the activation function performance. To identify an optimal activation function, this study investigates the correlation between the activation function selection and image classification performance in balanced or imbalanced datasets. Our analysis evaluates various network architectures for both commonly used and novel datasets and presents a comprehensive analysis of ten widely used activation functions. The experimental results show that the swish and softplus functions enhance the classification ability of state-of-the-art networks. Finally, this thesis distinguishes the neural networks using ten activation functions, analyzes their pros and cons, and puts forward detailed suggestions on choosing appropriate activation functions in future work. / Master of Science / When the novel coronavirus (COVID-19) outbreak began to ring alarm bells worldwide, the rapid, efficient diagnosis was critical to the emergency response. The manual diagnosis of chest X-rays by radiologists is time and cost-consuming. Compared with traditional diagnostic technology, the artificial intelligence medical system can simultaneously analyze and diagnose hundreds of medical images and speedily obtain high precision and high-efficiency returns. As we all know, machines are brilliant in learning new things and never sleep. Suppose machines can be used to replace human beings in some positions. In that case, it can significantly relieve the pressure on the medical system and buy time for medical practitioners to concentrate more on the research of new technologies. We need to know that the critical decision unit of the intelligent diagnosis system is the activation function. Therefore, this work provides an in-depth evaluation and comparison of the traditional and widely used activation functions with the emerging activation functions, which helps to improve the accuracy of the most advanced diagnostic model on the COVID-19 image dataset. Besides, the results of this study also summarize the cons and pros of using various neural functions and provide many suggestions for future work.

No description available.

Yes / Weight initialization of neural networks has an important influence on the learning process, and the selection of initial weights is related to the activation interval of the activation function. It is proposed that an improved and extended weight initialization method for neural network with asymmetric activation function as an extension of the linear interval tolerance method (LIT), called ‘GLIT’ (generalized LIT), which is more suitable for higher-dimensional inputs. The purpose is to expand the selection range of the activation function so that the input falls in the unsaturated region, so as to improve the performance of the network. Then, a tolerance solution theorem based upon neural network system is given and proved. Furthermore, the algorithm is given about determining the initial weight interval. The validity of the theorem and algorithm is verified by numerical experiments. The input could fall into any preset interval in the sense of probability under the GLIT method. In another sense, the GLIT method could provide a theoretical basis for the further study of neural networks. / The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This paper is partly supported by National Science Foundation of China under Grants (62073226, 61603262), Liaoning Province Natural Science Foundation (2020-KF-11-09, 2021-KF-11-05), Shen-Fu Demonstration Zone Science and Technology Plan Project (2020JH13, 2021JH07), Central Government Guides Local Science and Technology Development Funds of Liaoning Province (2021JH6).

Image recognition and -classification is becoming more important as the need to be able to process large amounts of images is becoming more common. The aim of this thesis is to compare two types of artificial neural networks, FeedForward Network and Convolutional Neural Network, to see how these compare when performing the task of image recognition. Six models of each type of neural network was created that differed in terms of width, depth and which activation function they used in order to learn. This enabled the experiment to also see if these parameters had any effect on the rate which a network learn and how the network design affected the validation accuracy of the models. The models were implemented using the API Keras, and trained and tested using the dataset CIFAR-10. The results showed that within the scope of this experiment the CNN models were always preferable as they achieved a statistically higher validation accuracy compared to their FFN counterparts.

Neuromorphic Engineering (NE) has led to the development of biologically-inspired computer architectures whose long-term goal is to approach the performance of the human brain in terms of energy efficiency and cognitive capabilities. Although there are a number of neuromorphic platforms available for large-scale Spiking Neural Network (SNN) simulations, the problem of programming these brain-like machines to be competent in cognitive applications still remains unsolved. On the other hand, Deep Learning has emerged in Artificial Neural Network (ANN) research to dominate state-of-the-art solutions for cognitive tasks. Thus the main research problem emerges of understanding how to operate and train biologically-plausible SNNs to close the gap in cognitive capabilities between SNNs and ANNs. SNNs can be trained by first training an equivalent ANN and then transferring the tuned weights to the SNN. This method is called ‘off-line’ training, since it does not take place on an SNN directly, but rather on an ANN instead. However, previous work on such off-line training methods has struggled in terms of poor modelling accuracy of the spiking neurons and high computational complexity. In this thesis we propose a simple and novel activation function, Noisy Softplus (NSP), to closely model the response firing activity of biologically-plausible spiking neurons, and introduce a generalised off-line training method using the Parametric Activation Function (PAF) to map the abstract numerical values of the ANN to concrete physical units, such as current and firing rate in the SNN. Based on this generalised training method and its fine tuning, we achieve the state-of-the-art accuracy on the MNIST classification task using spiking neurons, 99.07%, on a deep spiking convolutional neural network (ConvNet). We then take a step forward to ‘on-line’ training methods, where Deep Learning modules are trained purely on SNNs in an event-driven manner. Existing work has failed to provide SNNs with recognition accuracy equivalent to ANNs due to the lack of mathematical analysis. Thus we propose a formalised Spike-based Rate Multiplication (SRM) method which transforms the product of firing rates to the number of coincident spikes of a pair of rate-coded spike trains. Moreover, these coincident spikes can be captured by the Spike-Time-Dependent Plasticity (STDP) rule to update the weights between the neurons in an on-line, event-based, and biologically-plausible manner. Furthermore, we put forward solutions to reduce correlations between spike trains; thereby addressing the result of performance drop in on-line SNN training. The promising results of spiking Autoencoders (AEs) and Restricted Boltzmann Machines (SRBMs) exhibit equivalent, sometimes even superior, classification and reconstruction capabilities compared to their non-spiking counterparts. To provide meaningful comparisons between these proposed SNN models and other existing methods within this rapidly advancing field of NE, we propose a large dataset of spike-based visual stimuli and a corresponding evaluation methodology to estimate the overall performance of SNN models and their hardware implementations.

This thesis presents the use of a new sigmoid activation function in backpropagation artificial neural networks (ANNs). ANNs using conventional activation functions may generalize poorly when trained on a set which includes quirky, mislabeled, unbalanced, or otherwise complicated data. This new activation function is an attempt to improve generalization and reduce overtraining on mislabeled or irrelevant data by restricting training when inputs to the hidden neurons are sufficiently small. This activation function includes a flattened, low-training region which grows or shrinks during back-propagation to ensure a desired proportion of inputs inside the low-training region. With a desired low-training proportion of 0, this activation function reduces to a standard sigmoidal curve. A network with the new activation function implemented in the hidden layer is trained on benchmark data sets and compared with the standard activation function in an attempt to improve area under the curve for the receiver operating characteristic in biological and other classification tasks.

The Von-Neumann bottleneck, a major challenge in computer architecture, results from significant data transfer delays between the processor and main memory. Crossbar arrays utilizing spin-based devices like Magnetoresistive Random Access Memory (MRAM) aim to overcome this bottleneck by offering advantages in area and performance, particularly for tasks requiring linear transformations. These arrays enable single-cycle and in-memory vector-matrix multiplication, reducing overheads, which is crucial for energy and area-constrained Internet of Things (IoT) sensors and embedded devices. This dissertation focuses on designing, implementing, and evaluating reconfigurable computation platforms that leverage MRAM-based crossbar arrays and analog computation to support deep learning and error resilience implementations. One key contribution is the investigation of Spin Torque Transfer MRAM (STT-MRAM) technology scaling trends, considering power dissipation, area, and process variation (PV) across different technology nodes. A predictive model for power estimation in hybrid CMOS/MTJ technology has been developed and validated, along with new metrics considering the Internet of Things (IoT) energy profile of various applications. The dissertation introduces the Spintronically Configurable Analog Processing in-memory Environment (SCAPE), integrating analog arithmetic, runtime reconfigurability, and non-volatile devices within a selectable 2-D topology of hybrid spin/CMOS devices. Simulation results show improvements in error rates, power consumption, and power-error-product metric for real-world applications like machine learning and compressive sensing, while assessing process variation impact. Additionally, it explores transportable approaches to more robust SCAPE implementations, including applying redundancy techniques for artificial neural network (ANN)-based digit recognition applications. Generic redundancy techniques are developed and applied to hybrid spin/CMOS-based ANNs, showcasing improved/comparable accuracy with smaller-sized networks. Furthermore, the dissertation examines hardware security considerations for emerging memristive device-based applications, discussing mitigation approaches against malicious manufacturing interventions. It also discusses reconfigurable computing for AI/ML applications based on state-of-the-art FPGAs, along with future directions in adaptive computing architectures for AI/ML at the edge of the network.

Detection of a human emotion from human speech is always a challenging task. Factors like intonation, pitch, and loudness of signal vary from different human voice. So, it's important to know the exact pitch, intonation and loudness of a speech for making it a challenging task for detection. Some voices exhibit high background noise which will affect the amplitude or pitch of the signal. So, knowing the detailed properties of a speech to detect emotion is mandatory. Detection of emotion in humans from speech signals is a recent research field. One of the scenarios where this field has been applied is in situations where the human integrity and security are at risk In this project we are proposing a set of features based on the decomposition signals from discrete wavelet transform to characterize different types of negative emotions such as anger, happy, sad, and desperation. The features are measured in three different conditions: (1) the original speech signals, (2) the signals that are contaminated with noise or are affected by the presence of a phone channel, and (3) the signals that are obtained after processing using an algorithm for Speech Enhancement Transform. According to the results, when the speech enhancement is applied, the detection of emotion in speech is increased and compared to results obtained when the speech signal is highly contaminated with noise. Our objective is to use Artificial neural network because the brain is the most efficient and best machine to recognize speech. The brain is built with some neural network. At the same time, Artificial neural networks are clearly advanced with respect to several features, such as their nonlinearity and high classification capability. If we use Artificial neural networks to evolve the machine or computer that it can detect the emotion. Here we are using feedforward neural network which is suitable for classification process and using sigmoid function as activation function. The detection of human emotion from speech is achieved by training the neural network with features extracted from the speech. To achieve this, we need proper features from the speech. So, we must remove background noise in the speech. We can remove background noise by using filters. wavelet transform is the filtering technique used to remove the background noise and enhance the required features in the speech.

Les estrogènes jouent un rôle primordial dans le développement et le fonctionnement des tissus reproducteurs par leurs interactions avec les récepteurs des estrogènes ERα et ERβ. Ces récepteurs nucléaires agissent comme facteurs de transcription et contrôlent l’expression des gènes de façon hormono-dépendante et indépendante grâce à leurs deux domaines d’activation (AF-1 et AF-2). Une dérégulation de leur activité transcriptionnelle est souvent à l’origine de pathologies telles que le cancer du sein, de l’endomètre et des ovaires. Alors que ERα est utilisé comme facteur pronostic pour l’utilisation d’agents thérapeutiques, l’importance de la valeur clinique de ERβ est encore controversée. Toutefois, des évidences récentes lui associent un pouvoir anti-tumorigénique en démontrant que sa présence favorise l’inhibition de la progression de ces cancers ainsi que l’efficacité des traitements. En combinaisons avec d’autres études, ces observations démontrent que bien que les deux isoformes partagent une certaine similitude d’action, les ERs sont en mesure d’exercer des fonctions distinctes. Ces différences sont fortement attribuables au faible degré d’homologie observé entre certains domaines structuraux des ERs, comme le domaine AF-1, ce qui fait en sorte que les différents sites de modifications post-traductionnelles (MPTs) présents sur les ERs sont très peu conservés entre les isoformes. Or, l’activité transcriptionnelle ligand-dépendante et indépendante des ERs est hautement régulée par les MPTs. Elles sont impliquées à tous les niveaux de l’activation des ERs incluant la liaison et la sensibilité au ligand, la localisation cellulaire, la dimérisation, l’interaction avec l’ADN, le recrutement de corégulateurs transcriptionnels, la stabilité et l’arrêt de la transcription. Ainsi, de par leur dissimilitude, les ERs seront différemment régulés par la signalisation cellulaire. Comme un débalancement de plusieurs voies de signalisation ont été associées à la progression de tumeurs ER-positives ainsi qu’au développement d’une résistance, une meilleure compréhension de l’impact des MPTs sur la régulation spécifique des ERs s’avère essentielle en vue de proposer et/ou développer des traitements adéquats pour les cancers gynécologiques. Les résultats présentés dans cette thèse ont pour objectif de mieux comprendre les rôles des MPTs sur l’activité transcriptionnelle de ERβ qui sont, contrairement à ERα, très peu connus. Nous démontrons une régulation dynamique de ERβ par la phosphorylation, l’ubiquitination et la sumoylation. De plus, toutes les MPTs nouvellement découvertes par mes recherches se situent dans l’AF-1 de ERβ et permettent de mieux comprendre le rôle capital joué par ce domaine dans la régulation de l’activité ligand-dépendante et indépendante du récepteur. Dans la première étude, nous observons qu’en réponse aux MAPK, l’AF-1 de ERβ est phosphorylé au niveau de sérines spécifiques et qu’elles jouent un rôle important dans la régulation de l’activité ligand-indépendante de ERβ par la voie ubiquitine-protéasome. En effet, la phosphorylation de ces sérines régule le cycle d’activation-dégradation de ERβ en modulant son ubiquitination, sa mobilité nucléaire et sa stabilité en favorisant le recrutement de l’ubiquitine ligase E6-AP. De plus, ce mécanisme d’action semble être derrière la régulation différentielle de l’activité de ERα et ERβ observée lors de l’inhibition du protéasome. Dans le second papier, nous démontrons que l’activité et la stabilité de ERβ en présence d’estrogène sont étroitement régulées par la sumoylation phosphorylation-dépendante de l’AF-1, processus hautement favorisé par l’action de la kinase GSK-3. La sumoylation de ERβ par SUMO-1 prévient la dégradation du récepteur en entrant en compétition avec l’ubiquitination au niveau du même site accepteur. De plus, contrairement à ERα, SUMO-1 réprime l’activité de ERβ en altérant son interaction avec l’ADN et l’expression de ses gènes cibles dans les cellules de cancers du sein. Également, ces recherches ont permis d’identifier un motif de sumoylation dépendant de la phosphorylation (pSuM) jusqu’à lors inconnu de la communauté scientifique, offrant ainsi un outil supplémentaire à la prédiction de nouveau substrat de la sumoylation. En plus de permettre une meilleure compréhension du rôle des signaux intracellulaires dans la régulation de l’activité transcriptionnelle de ERβ, nos résultats soulignent l’importance des MPTs dans l’induction des différences fonctionnelles observées entre ERα et ERβ et apportent des pistes supplémentaires à la compréhension de leurs rôles physiopathologiques respectifs. / Estrogens play a pivotal role in reproductive physiology through direct interaction with the estrogen receptors ERα and ERβ, which belong to the nuclear hormone receptor family of ligand-activated transcription factors. Harbouring two activation domains (AF-1 and AF-2), gene expression can be controlled by ERs either in a hormone-dependent and/or independent manner. Disruption of ER transcriptional regulation is associated with pathological events such as breast and endometrial cancers. While ERα is considered a strong predictive factor in endocrine therapy of reproductive cancers, the clinical value of ERβ is still debated, although greater expression of ERβ has been associated with a favourable outcome since recent evidence has associated ERβ with anti-tumorigenic properties and a better response to anti-estrogenic compounds. Along with others studies, those individual outcomes indicate that even though the two receptors can exert similar roles by sharing resemblances in terms of structure and general response to hormone, they can also carry out distinct functions. These variations can be attributed to the fact that most of the structural domains shared by ERs exhibit a low level of homology, especially at the AF-1 domain. Consequently, the majority of the post-translational modifications sites (PTMs) on ERs are not shared between both isoforms. In fact, ligand-induced and ligand-independent activities of ERs are critically influenced by PTMs. PTMs controls the multiple aspects of ER-dependent activation by modulating ERs ligand binding, specificity, cellular localization, dimerization, interaction with their cognate DNA response element, combinatory recruitment of transcriptional coregulators, stability and transcriptional arrest. Hence, by their discrepancies, ERs will be differently influenced by the cellular environment. Furthermore, as the deregulation of different signalling pathway in cancers is associated with ER-dependant tumour progression and in the acquisition of a therapeutic resistant phenotype, it is crucial to understand the how PTMs affect ERs transactivation in order to eventually propose and/or develop adequate treatment. The results presented in this thesis were carried out with the objective of gaining a better understanding of PTM’s roles on ERβ transcriptional control which, as opposed to ERα, remain unclear. We demonstrate here a dynamic regulation of ERβ by phosphorylation, ubiquitination and sumoylation. Furthermore, as all the newly identified PTM are located within de AF-1 domain of ERβ, our results highlight the key role of this domain in the regulation of ligand-dependent and independent transcriptional properties of this receptor. The first study shows that in response to MAPK, specific serine residues in the AF-1 of ERβ are phosphorylated and play an important role in the regulation of ERβ ligand-independent activity by the ubiquitin-proteasome pathway. In fact, the activation-degradation cycle of ERβ induced by MAPK is regulated upon phosphorylation of these serines coordinating ERβ ubiquitination, subnuclear mobility and stability by promoting the recruitment of the ubiquitin ligase E6-AP. Moreover, this molecular process plays part in the differential regulation of ERα and ERβ activity upon proteasome inhibition. In the second paper, we demonstrate that ERβ activity and stability in presence of estrogen is closely regulated by the phosphorylation-dependent sumoylation of the AF-1 domain, amplified by GSK-3 action. SUMO-1 attachment prevents ERβ degradation by competing with ubiquitin at the same acceptor site and dictates ERβ transcriptional inhibition, as opposed to ERα, by altering estrogen-responsive target promoter occupancy and gene expression in breast cancer cells. Furthermore, these findings uncover a novel phosphorylated sumoylation motif (pSuM) and offer a valuable tool to predict novel SUMO substrates under protein kinase regulation. In combination to our better understanding on how intracellular signals controls ERβ transcriptional activity, our results highlight the significant role of PTMs in ERs isoforms discrepancies and allows supplementary comprehension of their respective physiopathologicals roles.