Global ETD Search
Search theses and dissertations gathered from participating repositories worldwide. Every result links back to the library that holds it. No account is needed.
Results
Showing 1 to 20 of 209 for “"Activation functions"”.
-
Activation functions in deep neural networks
… well understood. In particular, the influence of activation functions on the learning process has received limited theoretical attention, even though phenomena such as vanishing gradients, node saturation and sparsity have been identified as possible contributors when comparing different activation …
-
Network Parameterisation and Activation Functions in Deep Learning
… One of the key components of a network are its activation functions. The activations have a big impact on the overall mathematical form of the network. The \textit{first paper} studies generalisation of neural networks with rectified linear activations units (“ReLUs”). Such networks partition …
-
Parameterizing and Aggregating Activation Functions in Deep Neural Networks
The nonlinear activation functions applied by each neuron in a neural network are essential for making neural networks powerful representational models. If these are omitted, even deep neural networks reduce to simple linear regression due to the fact that a linear combination of linear …
-
Effective Activation Functions for Homomorphic Evaluation of Deep Neural Networks
… networks (CNNs) are primarily composed of linear functions which can be homomorphically evaluated, layers such as the activation layer are non-linear and cannot be homomorphically evaluated. One of the most commonly used alternatives is approximating these non-linear functions using low-degree …
-
Interaction graphs derived from activation functions and their application to gene regulation
… First we develop a natural transformation of activation functions and their derived interaction graphs, called conjugation, that is related to a natural transformation of signed digraphs called switching isomorphism. This is a useful tool for the analysis of interaction graphs used throughout …
-
A Neural Network Test Methodology for Networks with Programmable Low-Gain Activation Functions
… to specific types of ANN architectures. The Low Activation Gain Fault Detection (LAGFD) test algorithm presented in this work assumes no particular network architecture. The only requirements of the algorithm is that the ANN interconnection matrix be programmable and fully connectable, and that …
-
A Comparison of Image Classification with Different Activation Functions in Balanced and Unbalanced Datasets
… on the network structure design but also on the activation function performance. To identify an optimal activation function, this study investigates the correlation between the activation function selection and image classification performance in balanced or imbalanced datasets. Our analysis …
-
Detailed Analysis of Transcriptional Regulation by Activation Functions-1 and -2 of the Human Estrogen Receptor
… be divided into separable domains with specific functions, such as ligand binding, dimerization, DNA binding, and transactivation. The ER contains two distinct activation domains that are involved in transcription activation. Activation function-1 (AF-1) is located in the amino-terminal A/B …
-
Benchmarking Activation Functions of Deep Neural Networks for Protein Secondary Structure Prediction Using the Stuttgart Neural Network Simulator
… αλλά διαφορετικών συναρτήσεων ενεργοποίησης (activation functions). Για το λόγο αυτό, έλαβε χώρα η απαιτούμενη επέκταση του SNNS προκειμένου να εμπεριέχει τις προς μελέτη συναρτήσεις. Οι συναρτήσεις ενεργοποίησης που χρησιμοποιήθηκαν ήταν η Logistic, η Rectified Linear Unit (ReLU) και η Leaky …
-
Neural Network Verification as Piecewise Linear Optimization: Formulations for the Composition of Staircase Functions
… each neuron in a network using piecewise linear activation functions. Additionally, as in general, these formulations may require an exponential number of inequalities, we also derive a separation procedure that runs in super-linear time in the input dimension. We first introduce and develop our …
-
Development of intelligent computational methods for solving time-varying problems
… network (RNN) architectures, introducing new activation functions and discrete-time formulations to address time-varying problems in numerical linear algebra. While nonlinear activation functions have been shown to enhance the convergence speed of Zeroing Neural Network (ZNN) models, existing …
-
Vision-based dynamical systems and learning
… such that guarantee optimality of the designed activation function are provided. Finally, some examples are provided to illustrate the methodology, where some state-of-the-art activation functions are derived analytically they rely solely on the visual information, that is, times-to-collision …
-
Δίκτυα αραιής ενεργοποίησης: μια νέα μέθοδος αποσύνθεσης και συμπίεσης δεδομένων
… representations. We then present and define two activation functions (Identity, ReLU) as base of reference and three sparse activation functions (top-k absolutes, Extrema-Pool indices, Extrema) as candidate structures that minimize the previously defined metric φ. We lastly present Sparsely …
-
Expected Complexity and Gradients of Deep Maxout Neural Networks and Implications to Parameter Initialization
… depends on the particular parametrization of the functions represented by the network, that is, the assignment of parameters to functions. It also depends on the identity of the functions, which get assigned typical parameters at initialization, and, later, the parameters that arise during …
-
TeLU Activation Function for Fast and Stable Deep Learning
… Linear Unit (TeLU), a neural network hidden activation function defined as $TeLU(x)=x \cdot tanh(e^x)$. TeLU’s design is grounded in the core principles of key activation functions, achieving strong convergence by closely approximating the identity function in its active region while …
-
Medical image tomography: A statistically tailored neural network approach
… backpropagation ANN with statistically derived activation functions has been developed to improve the trainability and generalization ability of a network to produce accurate reconstructions. The tailored activation functions are derived from the estimated probability density functions (p.d.f.s) …
-
EFFECTIVE LEARNING IN MAX-MIN NEURAL NETWORKS
… As such, neural networks that employ max-min activation functions have been a subject of interest in recent years. Since max-min functions are not strictly differentiable, some ad hoc learning methods for such max-min neural networks have been proposed in the literature. In this thesis, we …
-
A Theory of Neural Computation with Clifford Algebras
… Clifford Multilayer Perceptrons with real-valued activation functions ((S)CMLPs) are studied. A generic Backpropagation algorithm for CMLPs is derived. Also, universal approximation theorems for (S)CMLPs are presented. Finally, CMLPs with Clifford-valued activation functions are studied.
-
Privacy Preserving Machine Learning as a Service
… we design methods for approximation of the activation functions commonly used in CNNs (i.e. ReLU, Sigmoid, and Tanh) with low degree polynomials which is essential for efficient homomorphic encryption schemes. Then, we train neural networks with the approximation polynomials instead of …
Page 1 of 11