No description available.

No description available.

Generative models are broadly used in many subfields of DL. DNNs have recently developed a core approach to solving data-centric problems in image classification, translation, etc. The latest developments in parameterizing these models using DNNs and stochastic optimization algorithms have allowed scalable modeling of complex, high-dimensional data, including speech, text, and image. This dissertation proposal presents our state-the-art probabilistic bases and DL algorithms for generative models, including VAEs, GANs, and RNN-based encoder-decoder. The proposal also discusses application areas that may benefit from deep generative models in both NLP and computer vision. In NLP, we proposed an Arabic poetry generation model with extended phonetic and semantic embeddings (Phonetic CNN_subword embeddings). Extensive quantitative experiments using BLEU scores and Hamming distance show notable enhancements over strong baselines. Additionally, a comprehensive human evaluation confirms that the poems generated by our model outperform the base models in criteria including meaning, coherence, fluency, and poeticness. We proposed a generative video model using a hybrid VAE-GAN model in computer vision. Besides, we integrate two attentional mechanisms with GAN to get the essential regions of interest in a video, focused on enhancing the visual implementation of the human motion in the generated output. We have considered quantitative and qualitative experiments, including comparisons with other state-of-the-arts for evaluation. Our results indicate that our model enhances performance compared with other models and performs favorably under different quantitive metrics PSNR, SSIM, LPIPS, and FVD.Recently, mimicking biologically inspired learning in generative models based on SNNs has been shown their effectiveness in different applications. SNNs are the third generation of neural networks, in which neurons communicate through binary signals known as spikes. Since SNNs are more energy-efficient than DNNs. Moreover, DNN models have been vulnerable to small adversarial perturbations that cause misclassification of legitimate images. This dissertation shows the proposed ``VAE-Sleep'' that combines ideas from VAE and the sleep mechanism leveraging the advantages of deep and spiking neural networks (DNN--SNN).On top of that, we present ``Defense–VAE–Sleep'' that extended work of ``VAE-Sleep'' model used to purge adversarial perturbations from contaminated images. We demonstrate the benefit of sleep in improving the generalization performance of the traditional VAE when the testing data differ in specific ways even by a small amount from the training data. We conduct extensive experiments, including comparisons with the state–of–the–art on different datasets.

Current approaches to generating rhyming English poetry with a neural network involve constraining output to enforce the condition of rhyme. We investigate whether this approach is necessary, or if recurrent neural networks can learn rhyme patterns on their own. We compile a new dataset of amateur poetry which allows rhyme to be learned without external constraints because of the dataset’s size and high frequency of rhymes. We then evaluate models trained on the new dataset using a novel framework that automatically measures the system’s knowledge of poetic form and generalizability. We find that our trained model is able to generalize the pattern of rhyme, generate rhymes unseen in the training data, and also that the learned word embeddings for rhyming sets of words are linearly separable. Our model generates a couplet which rhymes 68.15% of the time; this is the first time that a recurrent neural network has been shown to generate rhyming poetry a high percentage of the time. Additionally, we show that crowd-source workers can only distinguish between our generated couplets and couplets from our dataset 63.3% of the time, indicating that our model generates poetry with coherency, semantic meaning, and fluency comparable to couplets written by humans. / Graduate

Cryptocurrencies are a cryptography based technology, that has increased massively in popularity in recent years. These currencies are traded on markets that specialize in cryptocurrency trade. There, you can trade one cryptocurrency for another, or buy one with real world money. These markets are quite volatile, meaning that the price of most cryptocurrencies swing up and down a lot. The largest cryptocurrency is Bitcoin, but there is also more than 1500 smaller ones, that goes by the name alternative coins, or altcoins. This thesis will try to find out if it is possible to make accurate predictions about the future price of the altcoin Ethereum, and also see if Bitcoin may have some influence over the price of the selected altcoin. The predictions were made with the use of an artificial neural network, an LSTM network, that was trained on labeled data from 2017. The predictions were then made in intervals of one hour ahead, six hours ahead, and one day ahead through early 2018. The predictions showed that it is possible to make somewhat accurate predictions about the future. The predictions that were made one hour ahead were more accurate than both the six hours ahead predictions and the full day ahead predictions. By comparing the loss rates of the neural networks that were only trained on Ethereum, with the loss rates of the networks that trained on both Bitcoin and Ethereum, is was made clear that training on both cryptocurrencies did not improve the prediction accuracies.

Tato práce se zabývá automatickým generovaním popisů obrázků s využitím několika druhů neuronových sítí. Práce je založena na článcích z MS COCO Captioning Challenge 2015 a znakových jazykových modelech, popularizovaných A. Karpathym. Navržený model je kombinací konvoluční a rekurentní neuronové sítě s architekturou kodér--dekodér. Vektor reprezentující zakódovaný obrázek je předáván jazykovému modelu jako hodnoty paměti LSTM vrstev v síti. Práce zkoumá, na jaké úrovni je model s takto jednoduchou architekturou schopen popisovat obrázky a jak si stojí v porovnání s ostatními současnými modely. Jedním ze závěrů práce je, že navržená architektura není dostatečná pro jakýkoli popis obrázků.

The thesis investigates a publicly available dataset which covers morethan three million events in football matches. The aim of the study isto train machine learning models capable of modeling the relationshipbetween a shot event and its outcome. That is, to predict if a footballshot will result in a goal or not. By representing the shot indifferent ways, the aim is to draw conclusion regarding what elementsof a shot allows for a good prediction of its outcome. The shotrepresentation was varied both by including different numbers of eventspreceding the shot and by varying the set of features describing eachevent.The study shows that the performance of the machine learning modelsbenefit from including events preceding the shot. The highestpredictive performance was achieved by a long short-term memory neuralnetwork trained on the shot event and six events preceding the shot.The features which were found to have the largest positive impact onthe shot events were the precision of the event, the position on thefield and how the player was in contact with the ball. The size of thedataset was also evaluated and the results suggest that it issufficiently large for the size of the networks evaluated.

E-commerce has seen a rapid growth the last two decades, making it easy for customers to shop wherever they are. The growth has also led to new kinds of fraudulent activities affecting the customers. To make customers feel safe while shopping online, companies like Resurs Bank are implementing different kinds of fraud filters to freeze transactions that are thought to be fraudulent. The latest type of fraud filter is based on machine learning. While this seems to be a promising technology, data and algorithms need to be tuned properly to the task at hand. This thesis project gives a proof of concept of realizing a machine learning based fraud filter for Resurs Bank. Based on a literature study, available data and explainability requirements, this work opts for a supervised learning approach based on Random Forests with a sliding window to overcome concept drift. The inherent class imbalance of the setting makes the area-under-the-receiver operating-curve a suitable metric. This approach provided promising results that a machine learning based fraud filter can add value to companies like Resurs Bank. An alternative approach on how to incorporate non-numerical features by using recurrent neural networks (RNN) was implemented and compared. The non-numerical feature was transformed by a pre-trained RNN-model to a numerical representation that reflects the features suspiciousness. This new numerical feature was then included in the Random Forest model and the result demonstrated that this approach can add valuable insight to the fraud detection field.

The industry of electronic commerce (e-commerce) constitutes a great part of the yearly retail consumption in Sweden. Looking at recent years, it has been seen that a rapidly growing sector within the mentioned field is the clothing industry for clothes and accessories for children and newborns. To get an overview of the items and help customers to find what they are looking for, many web stores have a system called a Recommendation System. The mechanics behind this service can look rather different depending on the method used. However, their unified goal is to provide a list of recommended items of interest to the customer.  A branch within this field is the Session Based Recommendation System (SBRS). These are models which are designed to work with the trace of products, called a session, that a user currently has visited on the web store. Based on that information they then formulate an output of recommended items. The SBRS models have been especially popularized since the majority of customers browse in an anonymous behavior, which means that they due to time efficiency often neglect the possibility of creating or logging into any personal web store account. This however limits the accessible information that a system can make use of to shape its item list.  It can be seen that the number of articles exploring SBRS within the fashion branch of clothing and accessories for children is very limited. This thesis is made to fill that gap. After a thorough literature study, three models were found to be of certain interest, the Short-Term Attention/Memory Priority (STAMP) model, Long Short-Term Memory (LSTM) model, and Gated Recurrent Unit (GRU) model. Further, the LSTM model is included as it is the collaborative company, BabyShop Group AB's current used method.  The results of this thesis show that the GRU model is a promising method, managing to predict the next item for a customer more consistently than any other of the evaluated models. Furthermore, it can also be seen that what embeddings the models use to represent the products plays a significant role in the learning and evaluation of the used data set.  Moreover, a benchmark model included in this thesis also shows the importance of filtering the data set of sessions. It can be seen that a majority of customers visit already-seen products, logged happenings most likely due to refreshing web pages or similar actions. This causes the session data set to be characterized by repeated items. For future work, it would therefore indeed be interesting to see how this data set can be filtered in a different way. To see how that affects the outcome of the used metrics in this thesis. / Industrin för elektronisk handel (e-handel) utgör en stor del av den årliga konsumtionen av återförsäljning i Sverige. Bara genom att följa de senaste åren har det kunnat ses att en snabbt växande sektor inom det nämnda området är den som berör kläder och accessoarer för barn.  För att kunna ge en överblick och hjälpa kunder att finna vad de söker använder många webbutiker ett system som kallas rekommendationssystem. Hur dessa system faktiskt fungerar kan se väldigt olika ut. Men deras gemensamma mål är att i slutändan kunna ge en lista av rekommenderade produkter till kunden. En gren inom detta område är sessionsbaserade rekommendationssystem. Detta är modeller som är designade för att arbeta med själva spåret av besökta produkter, de som en kund har varit inne på under sin nuvarande vistelse på webbutiken. Baserat på denna information formuleras sedan en lista av rekommenderade produkter till besökaren. Dessa typer av modeller har blivit särskilt populära då många kunder gillar att shoppa anonymt. Vilket i denna kontext betyder att de gärna slipper att behöva logga in på något personligt konto på webbutiken, där särskild information kan sparas. Men detta betyder också att mängden tillgängliga data minskas för rekommendationssystemet.  Antalet forskningsartiklar som utforskar sessionsbaserade rekommendationssystem för e-handel inom barnmode är väldigt begränsad. Denna avhandling är därför gjord med syftet att försöka fylla detta tomrum. En genomgående litteraturstudie visade att tre modeller var av särskilt intresse, nämligen Short-Term Attention/Memory Priority (STAMP), Gated Recurrent Unit (GRU) och Long Short Term Memory (LSTM) modellen. Den sistnämnda är inkluderad då detta är den nuvarande modellen som används av företaget som denna avhandling har gjorts i samarbete med, BabyShop Group AB.  Resultaten i denna avhandling kan påvisa att GRU är en mycket lovande modell som lyckades förutbestämma nästkommande produkt i en sessionskedja bäst. Utöver detta kan det också ses att embedding-vektorerna som används för att representera produkterna för modellerna spelar en avgörande roll. Speciellt för deras lärande och evaluering av data.  Förutom det påvisade en av riktvärdesmodellerna som användes i denna avhandling den viktiga innebörden av att filtrera sessionsdata. Det kan nämligen urskiljas i den data som erhölls från företaget att många kunder återbesöker en stor del av redan besökta produkter. Detta åstadkommas troligen av att kunderna uppdaterar sidan de är på, eller utför någon annan liknande handling. Det här gör att en stor del av den sessionsdata som används i denna avhandling innehåller många upprepade produkter i de givna sessionskedjorna. Som framtida arbete vore det därför intressant att utforska olika filtreringsmetoder som kan appliceras på den givna datamängden. Detta för att se hur en mera filtrerad datamängd påverkar slutresultatet av de använda mätmetoderna i denna avhandling.

The introduction of open banking has made it possible for companies to build the next generation of applications based on transactional data. Enabling economic forecasts which private individuals can use to make responsible financial decisions. This project investigated forecasting account balances using supervised learning. 7 different regression models were run on transactional data from 377 anonymised checking accounts split into subgroups. The results concluded that multivariate XGBoost optimised with feature selection was the best performing forecasting model and the subgroup with recurring income transactions was easiest to forecast. Based on the result from this project it can be concluded that a viable option to forecast account balances is to split the transactional data into subgroups and forecast them separately. Minimising the errors given by certain random, infrequent and large types of transactions.