A process that historically has been hard to automate is interior design, mainly due to its subjective nature and lack of obvious guidelines. Scientifically, there is interest to examine if subjective processes can be automated using black box algorithms such as neural networks, as well as corporate interest in this subject to increase efficiency and create systems for automated floor plan design. This work focuses on finding the limitations of such a project, mainly in establishing the threshold of data points needed for an algorithm in this area to generate relevant results as well as an investigation into requirements to make systems of this kind incorporated in a production pipeline. In this work floor plans with functional area division were set out to be generated using a conditional, generative, adversarial network, cGAN. The system is applied on a use-case provided by NORNORM, a company providing a circular, subscription-based furnishing service for office spaces, also providing data in the form of floor plans. The algorithm is inspired by Yang et al.’s stateof-the-art model from 2019 and the network is tested with three data sets of different sizes, consisting of 100, 500 and 1000 floor plans respectively. This work includes a quantitative evaluation inspired by by Di and Yu, using the average intersection over union metric. Additionally, this work proposes a qualitative evaluation. The qualitative evaluation is carried out using interior designers, posed with a subjective, two-alternative, forced-choice (2AFC) approval or disapproval of the design as a first draft to a customer. This evaluation was not conducted due to insufficient results. The generated results suggests that the threshold for data lies above 1000 data points and, compared to the work by Yang et al., below 4000 data points, the quantitative evaluation concurred with this statement. This interval could be narrowed in future work. In relation as to whether or not the system could be production ready there are a few requirements unachieved, for instance automated data collection and preprocessing. Future work could include conducting the qualitative evaluation on a future implementation of this system. / Historiskt sett har subjektiva processer så som inredningsdesign varit svåra att automatisera, på grund av avsaknad av formella processer och riktlinjer. Det finns ett vetenskapligt intresse i att undersöka om processer som inredningsdesign kan automatiseras med hjälp av algoritmer eller neurala nätverk, men även ett industriellt intresse för att minimera resurser som läggs på detta. Det här projektet fokuserar på att hitta begränsningar av sådana system genom att fokusera på att hitta lägsta möjliga mängd data för att generera relevanta resultat samt undersöka vad som krävs för att använda systemet i produktion. I samarbete med företaget NORNORM skapades ett villkorligt generativt motståndsnätverk i syfte att generera planlösningar med funktionsområdesindelning. NORNORM är ett cirkulärt företag som erbjuder en abonnemangsbaserad tjänst för inredning av kontorslokaler och bidrog med data i form av färdiginredda planlösningar. Algoritmen som används är inspirerad av Yang et al.’s modell från 2019 som ligger i den vetenskapliga framkanten. Nätverket testades med tre olika dataset, som bestod av 100, 500 och 1000 datapunkter. Nätverket utvärderades kvantitativt med en utvärdering inspirerad av Di and Yu, som använder genomsnittlig skärningspunkt över union. Utöver detta föreslås en kvalitativ utvärdering i detta arbete, som utförs med utbildade interiördesigners. Varje desginer utsätts för ett subjektiv val att godkänna eller underkänna en design som ett första utkast till en kund. Utvärderingsmodellen användes inte i det här projektet på grund av bristande resultat. Den genererade resultaten visar att gränser för antalet datapunkter som krävs för att generera relevanta resultat ligger mellan 1000 och 4000 datapunkter, vilket styrks av den kvatitativa utvärderingen. Detta intervall kan förslagsvis avsmalna i framtida projekt. Vidare kräver systemet ett antal premisser för att kunna vara redo för produktion, till exempel automatiserad datainhämtning och behandlande. Framtida projekt kan utföra den föreslagna kvalitativa utvärderingen på liknande system.

The deep learning-based automatic segmentation methods have developed rapidly in recent years to give a promising performance in the medical image segmentation tasks, which provide clinical medicine with an accurate and fast computer-aided diagnosis method. Generative adversarial networks and their extended frameworks have achieved encouraging results on image-to-image translation problems. In this report, the proposed hybrid network combined cycle-consistent adversarial networks, which transformed contrast-enhanced images from computed tomography angiography to the conventional low-contrast CT scans, with the segmentation network and trained them simultaneously in an end-to-end manner. The trained segmentation network was tested on the non-contrast-enhanced CT images. The synthetic process and the segmentation process were also implemented in a two-stage manner. The two-stage process achieved a higher Dice similarity coefficient than the baseline U-Net did on test data, but the proposed hybrid network did not outperform the baseline due to the field of view difference between the two training data sets.

Point cloud data can express the 3D features of objects, and is an important data type in the field of 3D object detection. Since point cloud data is more difficult to collect than image data and the scale of existing datasets is smaller, point cloud data augmentation is introduced to allow more features to be discovered on existing data. In this thesis, we propose a novel method to enhance the point cloud scene, based on the generative adversarial network (GAN) to realize the augmentation of the objects and then integrate them into the existing scenes. A good fidelity and coverage are achieved between the fake sample and the real sample, with JSD equal to 0.027, MMD equal to 0.00064, and coverage equal to 0.376. In addition, we investigated the functional data annotation tools and completed the data labeling task. The 3D object detection task is carried out on the point cloud data, and we have achieved a relatively good detection results in a short processing of around 22ms. Quantitative and qualitative analysis is carried out on different models. / Punktmolndata kan uttrycka 3D-egenskaperna hos objekt och är en viktig datatyp inom området för 3D-objektdetektering. Eftersom punktmolndata är svarare att samla in än bilddata och omfattningen av befintlig data är mindre, introduceras punktmolndataförstärkning för att tillåta att fler funktioner kan upptäckas på befintlig data. I det här dokumentet föreslår vi en metod för att förbättra punktmolnsscenen, baserad på det generativa motstridiga nätverket (GAN) för att realisera förstärkningen av objekten och sedan integrera dem i de befintliga scenerna. En god trohet och tackning uppnås mellan det falska provet och det verkliga provet, med JSD lika med 0,027, MMD lika med 0,00064 och täckning lika med 0,376. Dessutom undersökte vi de funktionella verktygen för dataanteckningar och slutförde uppgiften for datamärkning. 3D- objektdetekteringsuppgiften utförs på punktmolnsdata och vi har uppnått ett relativt bra detekteringsresultat på en kort bearbetningstid runt 22ms. Kvantitativ och kvalitativ analys utförs på olika modeller.

Fast detection and identification of unknown substances is an area of interest for many parties. Raman spectroscopy is a laser-based method allowing for long range no contact investigation of substances. A Coded Aperture Snapshot Spectral Imaging (CASSI) system allows for fast and efficient measurements of hyperspectral images of a scene, containing a mixture of the spatial and spectral data. To analyze the scene and the unknown substances within it, it is required that the spectra in each spatial position are known. Utilizing the theory of compressed sensing allows for reconstruction of hyperspectral images of a scene given their CASSI measurements by assuming a sparsity prior. These reconstructions can then be utilized by a human operator to deduce and classify the unknown substances and their spatial locations in the scene. Such classifications are then applicable as decision support in various areas, for example in the judicial system. Reconstruction of hyperspectral images given CASSI-measurements is an ill-posed inverse problem typically solved by utilizing regularization techniques such as total variation (TV). These TV-based reconstruction methods are time consuming relative to the time needed to acquire the CASSI measurements, which is in the order of seconds. This leads to a reduced number of areas where the technology is applicable. In this thesis, a Generative Adversarial Network (GAN) based reconstruction method is proposed. A GAN is trained using simulated training data consisting of hyperspectral images and their respective CASSI measurements. The GAN provides a learned prior, and is used in an iterative optimization algorithm seeking to find an optimal set of latent variables such that the reconstruction error is minimized. The results of the developed GAN based reconstruction method are compared with a traditional TV method and a different machine learning based reconstruction method.  The results show that the reconstruction method developed in this thesis performs better than the compared methods in terms of reconstruction quality in short time spans.

Cieľom tejto diplomovej práce je vygenerovanie datasetu syntetických snímkov odtlačkov prstov, ktoré vykazujú známky kožných ochorení. Práca sa zaoberá poškodením spôsobeným kožnými ochoreniami v odtlačkoch prstov a generovaním syntetických odtlačkov prstov. Odtlačky prstov s prejavom kožných ochorení boli generované s využitím modelu založeného na Wasserstein GAN s penalizáciou gradientu. Na trénovanie GAN modelu bola použitá unikátna databáza odtlačkov prstov s prejavom kožných ochorení vytvorená na FIT VUT. Daný model bol trénovaný na troch typoch kožných ochorení: atopický ekzém, psoriáza a dyshidrotický ekzém. Sieť generátoru z natrénovaného WGAN-GP modelu bola použitá na vygenerovanie datasetov syntetických odtlačkov prstov. Tieto syntetické odtlačky boli porovnané s reálnymi odtlačkami s využitím NFIQ a FiQiVi nástrojov na určenie kvality spoločne s porovnaním rozložení lokácií a orientácii markantov v snímkoch odtlačkov prstov.

Language-based fashion image editing has promising applications in design, sustainability, and art. However, it is considered a challenging problem in computer vision and graphics. The diversity of human poses and the complexity of clothing shapes and textures make the editing problem difficult. Inspired by recent progress in editing face images through manipulating latent representations, such as StyleCLIP and HairCLIP, we apply those methods in editing the images of full-body humans in fashion datasets and evaluate their effectiveness. First, we assess different methodologies to find a latent representation of an image via Generative Adversarial Network (GAN) inversion; then, we apply three image manipulation schemes. Thus, a pre-trained e4e encoder is initially utilized for the inversion process, while the results are compared to a more accurate method, Pivotal Tuning Inversion (PTI). Next, we employ an optimization scheme that uses the Contrastive Language Image Pre-training (CLIP) model to guide the latent representation of an image in the direction of attributes described in the input text. We address the problem of the accuracy and speed of the process by incorporating a mapper network. Finally, we propose an optimized mapper called Text-Driven Garment Editing Mapper (TD-GEM) to achieve high-quality image editing in a disentangled way. Our empirical results show that the proposed method can edit fashion items for changing color and sleeve length. / Språkbaserad bildredigering inom mode har lovande tillämpningar inom design, hållbarhet och konst. Det betraktas dock som ett utmanande problem inom datorseende och grafik. Mångfalden och variationen av mänskliga poser och komplexiteten i klädform och texturer gör redigeringsproblemet svårt. Inspirerade av den senaste utvecklingen inom redigering av ansiktsbilder genom manipulation av latenta representationer, såsom StyleCLIP och HairCLIP, tillämpar vi dessa metoder för att redigera bilderna av fullständiga mänskliga kroppar i mode-dataset och utvärderar deras effektivitet. Först jämför vi olika metoder för att hitta en latent representation av en bild via så kallade Generative Adversarial Network (GAN) inversion; sedan tillämpar vi tre bildmanipulationsscheman. En förtränad (eng: pre-trained) e4e-encoder model används först för inversionsprocessen, medan resultaten jämförs med en mer exakt metod, Pivotal Tuning Inversion (PTI). Därefter använder vi en optimeringmetod som använder Contrastive Language Image Pre-training (CLIP) -modell för att vägleda den latenta representationen av en bild i riktning mot attribut som beskrivs i inmatningstexten. Vi tar upp problemet med noggrannhet och hastigheten i processen genom att integrera en mapper-nätverk. Slutligen föreslår vi en optimerad mapper som kallas TD-GEM för att uppnå högkvalitativ bildredigering på ett lösgjort sätt. Våra empiriska resultat visar att den föreslagna metoden kan redigera modeobjekt för att ändra färg och ärmens längd.

Both computed tomography (CT) and magnetic resonance imaging (MRI) have a purpose for radiotherapy. But having two imaging sessions brings uncertainty which makes it beneficial to create synthetic CT (sCT) images from MR images. In this work a Generative Adversarial Network (GAN) was designed and implemented for sCT generation. The purpose of the work was to broaden the understanding of how variation in training data affects a model’s performance on generating sCTs. This was done by increasing the training sets with patients with artifacts, female patients and synthetic MR contrasts. Eight different machine learning models with varying training data were trained and evaluated. Four models were trained using T2-weighted data only while the other four used both real T2-weighted images and synthetic T1 (sT1) images in their training sets. The models were evaluated on the pixel value difference between the CT and the resulting sCTs using a mean absolute error (MAE) evaluation. Afterwards, dose calculations were made with patients’ treatment plans on both their CTs and their corresponding sCTs and compared doses to some of the structures. Finally the models were compared based on their performance on synthetic MR contrasts. This means I used a contrast transfer model to change the contrast from a T1-weighted image to a synthetic T2 or from a T2-weighted image to a synthetic T1 and then generated sCT images from the synthetic contrasts. These experiments showed that when I increased the models’ training sets with relevant patients MAE decreased between the CT and a generated sCT. Importantly, this was also true for our models trained on sT1s when evaluating on T1 weighted images. Increasing the size of the datasets also increased the performance in a treatment planning purpose and it also decreased the difference when evaluating the models on original MR images and synthetic MR images. In conclusion an improved performance was shown for models evaluated on images with artifacts, female patients and other MR contrast when including more images from those image types in the training dataset.

Large datasets are a crucial requirement to achieve high performance, accuracy, and generalisation for any machine learning task, such as prediction or anomaly detection, However, it is not uncommon for datasets to be small or imbalanced since gathering data can be difficult, time-consuming, and expensive. In the task of collecting vehicle sensor time series data, in particular when the vehicle has an abnormal behaviour, these struggles are present and may hinder the automotive industry in its development. Synthetic data generation has become a growing interest among researchers in several fields to handle the struggles with data gathering. Among the methods explored for generating data, generative adversarial networks (GANs) have become a popular approach due to their wide application domain and successful performance. This thesis focuses on generating multivariate time series data that are similar to vehicle sensor readings from the air pressures in the brake system of vehicles with an abnormal behaviour, meaning there is a leakage somewhere in the system. A novel GAN architecture called TimeGAN was trained to generate such data and was then evaluated using both qualitative and quantitative evaluation metrics. Two versions of this model were tested and compared. The results obtained proved that both models learnt the distribution and the underlying information within the features of the real data. The goal of the thesis was achieved and can become a foundation for future work in this field. / När man applicerar en modell för att utföra en maskininlärningsuppgift, till exempel att förutsäga utfall eller upptäcka avvikelser, är det viktigt med stora dataset för att uppnå hög prestanda, noggrannhet och generalisering. Det är dock inte ovanligt att dataset är små eller obalanserade eftersom insamling av data kan vara svårt, tidskrävande och dyrt. När man vill samla tidsserier från sensorer på fordon är dessa problem närvarande och de kan hindra bilindustrin i dess utveckling. Generering av syntetisk data har blivit ett växande intresse bland forskare inom flera områden som ett sätt att hantera problemen med datainsamling. Bland de metoder som undersökts för att generera data har generative adversarial networks (GANs) blivit ett populärt tillvägagångssätt i forskningsvärlden på grund av dess breda applikationsdomän och dess framgångsrika resultat. Denna avhandling fokuserar på att generera flerdimensionell tidsseriedata som liknar fordonssensoravläsningar av lufttryck i bromssystemet av fordon med onormalt beteende, vilket innebär att det finns ett läckage i systemet. En ny GAN modell kallad TimeGAN tränades för att genera sådan data och utvärderades sedan både kvalitativt och kvantitativt. Två versioner av denna modell testades och jämfördes. De erhållna resultaten visade att båda modellerna lärde sig distributionen och den underliggande informationen inom de olika signalerna i den verkliga datan. Målet med denna avhandling uppnåddes och kan lägga grunden för framtida arbete inom detta område.

Backdoor (Trojan) attacks are a major threat to the security of deep neural network (DNN) models. They are created by an attacker who adds a certain pattern to a portion of given training dataset, causing the DNN model to misclassify any inputs that contain the pattern. These infected classifiers are called Trojan models and the added pattern is referred to as the trigger. In image domain, a trigger can be a patch of pixel values added to the images and in text domain, it can be a set of words. In this thesis, we propose Trojan-Miner (T-Miner), a defense scheme against such backdoor attacks on text classification deep learning models. The goal of T-Miner is to detect whether a given classifier is a Trojan model or not. To create T-Miner , our approach is based on a sequence-to-sequence text generation model. T-Miner uses feedback from the suspicious (test) classifier to perturb input sentences such that their resulting class label is changed. These perturbations can be different for each of the inputs. T-Miner thus extracts the perturbations to determine whether they include any backdoor trigger and correspondingly flag the suspicious classifier as a Trojan model. We evaluate T-Miner on three text classification datasets: Yelp Restaurant Reviews, Twitter Hate Speech, and Rotten Tomatoes Movie Reviews. To illustrate the effectiveness of T-Miner, we evaluate it on attack models over text classifiers. Hence, we build a set of clean classifiers with no trigger in their training datasets and also using several trigger phrases, we create a set of Trojan models. Then, we compute how many of these models are correctly marked by T-Miner. We show that our system is able to detect trojan and clean models with 97% overall accuracy over 400 classifiers. Finally, we discuss the robustness of T-Miner in the case that the attacker knows T-Miner framework and wants to use this knowledge to weaken T-Miner performance. To this end, we propose four different scenarios for the attacker and report the performance of T-Miner under these new attack methods. / M.S. / Backdoor (Trojan) attacks are a major threat to the security of predictive models that make use of deep neural networks. The idea behind these attacks is as follows: an attacker adds a certain pattern to a portion of given training dataset and in the next step, trains a predictive model over this dataset. As a result, the predictive model misclassifies any inputs that contain the pattern. In image domain this pattern that is called trigger, can be a patch of pixel values added to the images and in text domain, it can be a set of words. In this thesis, we propose Trojan-Miner (T-Miner), a defense scheme against such backdoor attacks on text classification deep learning models. The goal of T-Miner is to detect whether a given classifier is a Trojan model or not. T-Miner is based on a sequence-to-sequence text generation model that is connected to the given predictive model and determine if the predictive model is being backdoor attacked. When T-Miner is connected to the predictive model, it generates a set of words, called perturbations, and analyses these perturbations to determine whether they include any backdoor trigger. Hence if any part of the trigger is present in the perturbations, the predictive model is flagged as a Trojan model. We evaluate T-Miner on three text classification datasets: Yelp Restaurant Reviews, Twitter Hate Speech, and Rotten Tomatoes Movie Reviews. To illustrate the effectiveness of T-Miner, we evaluate it on attack models over text classifiers. Hence, we build a set of clean classifiers with no trigger in their training datasets and also using several trigger phrases, we create a set of Trojan models. Then, we compute how many of these models are correctly marked by T-Miner. We show that our system is able to detect Trojan models with 97% overall accuracy over 400 predictive models.

<p dir="ltr">This research involves numerous network intrusion techniques through novel applications of graph representation learning and image representation learning. The methods are tested on multiple publicly available network flow datasets.</p>