Return to search

A Study of Adaptive Random Features Models in Machine Learning based on Metropolis Sampling / En studie av anpassningsbara slumpmässiga funktioner i maskininlärning baserat på Metropolis-sampling

Artificial neural network (ANN) is a machine learning approach where parameters, i.e., frequency parameters and amplitude parameters, are learnt during the training process. Random features model is a special case of ANN that the structure of random features model is as same as ANN’s but the parameters’ learning processes are different. For random features model, the amplitude parameters are learnt during the training process but the frequency parameters are sampled from some distributions. If the frequency distribution of the random features model is well-chosen, both models can approximate data well. Adaptive random Fourier features with Metropolis sampling is an enhanced random Fourier features model which can select appropriate frequency distribution adaptively. This thesis studies Rectified Linear Unit and sigmoid features and combines them with the adaptive idea to generate another two adaptive random features models. The results show that using the particular set of hyper-parameters, adaptive random Rectified Linear Unit features model can also approximate the data relatively well, though the adaptive random Fourier features model performs slightly better. / I artificiella neurala nätverk (ANN), som används inom maskininlärning, behöver parametrar, kallade frekvensparametrar och amplitudparametrar, hittasgenom en så kallad träningsprocess. Random feature-modeller är ett specialfall av ANN där träningen sker på ett annat sätt. I dessa modeller tränasamplitudparametrarna medan frekvensparametrarna samplas från någon sannolikhetstäthet. Om denna sannolikhetstäthet valts med omsorg kommer båda träningsmodellerna att ge god approximation av givna data. Metoden Adaptiv random Fourier feature[1] uppdaterar frekvensfördelningen adaptivt. Denna uppsats studerar aktiveringsfunktionerna ReLU och sigmoid och kombinerar dem med den adaptiva iden i [1] för att generera två ytterligare Random feature-modeller. Resultaten visar att om samma hyperparametrar som i [1] används så kan den adaptiva ReLU features-modellen approximera data relativt väl, även om Fourier features-modellen ger något bättre resultat.

Identiferoai:union.ndltd.org:UPSALLA1/oai:DiVA.org:kth-293323
Date January 2021
CreatorsBai, Bing
PublisherKTH, Numerisk analys, NA
Source SetsDiVA Archive at Upsalla University
LanguageEnglish
Detected LanguageSwedish
TypeStudent thesis, info:eu-repo/semantics/bachelorThesis, text
Formatapplication/pdf
Rightsinfo:eu-repo/semantics/openAccess
RelationTRITA-SCI-GRU ; 2021:029

Page generated in 0.002 seconds