This work is dedicated to robot path planning with using principles of dynamic programing in discrete state space. Theoretical part is dedicated to actual situation in this field and to principle of applying Markov decission process to path planning. Practical part is dedicated to implementation of two algorithms based on MDP principles.

En omfattande digitalisering har under den senaste tiden pågått i samhället där många företag väljer att till viss del eller helt flytta sin verksamhet till internet. För vissa branscher har det varit mer optimalt att skifta försäljningskanal, medan andra har stött på hinder. Livsmedelsbranschen är den bransch som dominerar totalmarknaden men den halkar efter i digitaliseringen. Denna studie har som syfte att ta reda på vilka faktorer som påverkar konsumenter att handla livsmedel på nätet. För att tillgodose studiens syfte användes en enkätstudie. Detta skapade möjlighet att få en djupare förståelse för konsumenternas beteenden. Enkätstudien bidrog också till att få en så bred bild om ämnet som möjligt inom den givna tidsramen för uppsatsskrivandet. Det empiriska materialet användes sedan som diskussionsunderlag till att jämföra med vad tidigare forskning har kunnat dra för slutsatser kring ämnet. Slutsatsen av studien är att det fortfarande är många konsumenter som inte handlar livsmedel på nätet. Respondenterna i denna studie ansåg att den främsta faktorn som påverkar att de hellre köper livsmedel i fysisk butik är att de vill kunna se och känna på varorna först. Priset på varor och fraktkostnad är även de faktorer som hade stor påverkan på varför konsumenter väljer bort att handla livsmedel på nätet. De faktorer som visade sig ha störst påverkan på hur ofta respondenterna handlade livsmedel på nätet var ålder, hushållsstorlek, varupris, fraktkostnad, krånglig retur/reklamation av varor samt trygga betalningsalternativ. / In recent times, an extensive digitalization has taken place in society where many companies choose to move their business to the Internet to some extent or completely. For some industries, it has been more optimal to change sales channels, while others have encountered obstacles. The food industry is the industry that dominates the total market, but it has fallen behind in the digitalization. The purpose of this study is to find out which factors that influence consumers to buy groceries online. In order to meet the purpose of the study, a survey study was used. This created the opportunity to gain a deeper understanding of consumer behaviour. The survey also helped to get as broad a picture of the topic as possible within the given time frame for the writing of the essay. The empirical material was then used as a discussion basis to compare with what previous research has concluded about the topic. The conclusion of the study is that there are still a lot of consumers who do not buy groceries online. The respondents in this study considered that the main factor that influences the fact that they prefer to buy groceries in a physical store is that they want to be able to see and feel the products first. The price of the products and shipping costs are also factors that had a major impact on consumers opinions. The factors that were found to have the greatest impact on how often the respondents buy groceries online were age, size of household, product price, shipping cost, difficult return/reclaim of products and secure payment options.

Nous proposons une nouvelle approche pour l'apprentissage de représentation parcimonieuse, où le but est de limiter le nombre de caractéristiques sélectionnées \textbf{par donnée}, résultant en un modèle que nous appellerons \textit{Modèle de parcimonie locale pour la classification} --- \textit{Datum-Wise Sparse Classification} (DWSC) en anglais. Notre approche autorise le fait que les caractéristiques utilisées lors de la classification peuvent être différentes d'une donnée à une autre: une donnée facile à classifier le sera ainsi en ne considérant que quelques caractéristiques, tandis que plus de caractéristiques seront utilisées pour les données plus complexes. Au contraire des approches traditionnelles de régularisation qui essaient de trouver un équilibre entre performance et parcimonie au niveau de l'ensemble du jeu de données, notre motivation est de trouver cet équilibre au niveau des données individuelles, autorisant une parcimonie moyenne plus élevée, pour une performance équivalente. Ce type de parcimonie est intéressant pour plusieurs raisons~: premièrement, nous partons du principe que les explications les plus simples sont toujours préférables~; deuxièmement, pour la compréhension des données, une représentation parcimonieuse par donnée fournit une information par rapport à la structure sous-jacente de celles-ci~: typiquement, si un jeu de données provient de deux distributions disjointes, DWSC autorise le modèle à choisir automatiquement de ne prendre en compte que les caractéristiques de la distribution génératrice de chaque donnée considérée. / This thesis introduces a body of work on sequential models for classification. These models allow for a more flexible and general approach to classification tasks. Many tasks ultimately require the classification of some object, but cannot be handled with a single atomic classification step. This is the case for tasks where information is either not immediately available upfront, or where the act of accessing different aspects of the object being classified may present various costs (due to time, computational power, monetary cost, etc.). The goal of this thesis is to introduce a new method, which we call datum-wise classification, that is able to handle these more complex classifications tasks by modelling them as sequential processes.

L'apprentissage automatique (ML) a connu d'énormes succès ces dernières années et repose sur un nombre toujours croissant d'applications réelles. Cependant, la conception d'algorithmes prometteurs pour un problème spécifique nécessite toujours un effort humain considérable. L'apprentissage automatique (AutoML) a pour objectif de sortir l'homme de la boucle. AutoML est généralement traité comme un problème de sélection d’algorithme / hyper-paramètre. Les approches existantes incluent l’optimisation Bayésienne, les algorithmes évolutionnistes et l’apprentissage par renforcement. Parmi eux, auto-sklearn, qui intègre des techniques de meta-learning à l'initialisation de la recherche, occupe toujours une place de choix dans les challenges AutoML. Cette observation a orienté mes recherches vers le domaine du meta-learning. Cette orientation m'a amené à développer un nouveau cadre basé sur les processus de décision Markovien (MDP) et l'apprentissage par renforcement (RL). Après une introduction générale (chapitre 1), mon travail de thèse commence par une analyse approfondie des résultats du Challenge AutoML (chapitre 2). Cette analyse a orienté mon travail vers le meta-learning, menant tout d’abord à proposer une formulation d’AutoML en tant que problème de recommandation, puis à formuler une nouvelle conceptualisation du problème en tant que MDP (chapitre 3). Dans le cadre du MDP, le problème consiste à remplir de manière aussi rapide et efficace que possible une matrice S de meta-learning, dans laquelle les lignes correspondent aux tâches et les colonnes aux algorithmes. Un élément de matrice S (i, j) est la performance de l'algorithme j appliqué à la tâche i. La recherche efficace des meilleures valeurs dans S nous permet d’identifier rapidement les algorithmes les mieux adaptés à des tâches données. Dans le chapitre 4, nous examinons d’abord le cadre classique d’optimisation des hyper-paramètres. Au chapitre 5, une première approche de meta-learning est introduite, qui combine des techniques d'apprentissage actif et de filtrage collaboratif pour prédire les valeurs manquantes dans S. Nos dernières recherches appliquent RL au problème du MDP défini pour apprendre une politique efficace d’exploration de S. Nous appelons cette approche REVEAL et proposons une analogie avec une série de jeux pour permettre de visualiser les stratégies des agents pour révéler progressivement les informations. Cette ligne de recherche est développée au chapitre 6. Les principaux résultats de mon projet de thèse sont : 1) Sélection HP / modèle : j'ai exploré la méthode Freeze-Thaw et optimisé l'algorithme pour entrer dans le premier challenge AutoML, obtenant la 3ème place du tour final (chapitre 3). 2) ActivMetaL : j'ai conçu un nouvel algorithme pour le meta-learning actif (ActivMetaL) et l'ai comparé à d'autres méthodes de base sur des données réelles et artificielles. Cette étude a démontré qu'ActiveMetaL est généralement capable de découvrir le meilleur algorithme plus rapidement que les méthodes de base. 3) REVEAL : j'ai développé une nouvelle conceptualisation du meta-learning en tant que processus de décision Markovien et je l'ai intégrée dans le cadre plus général des jeux REVEAL. Avec un stagiaire en master, j'ai développé des agents qui apprennent (avec l'apprentissage par renforcement) à prédire le meilleur algorithme à essayer. Le travail présenté dans ma thèse est de nature empirique. Plusieurs méta-données du monde réel ont été utilisées dans cette recherche. Des méta-données artificielles et semi-artificielles sont également utilisées dans mon travail. Les résultats indiquent que RL est une approche viable de ce problème, bien qu'il reste encore beaucoup à faire pour optimiser les algorithmes et les faire passer à l’échelle aux problèmes de méta-apprentissage plus vastes. / Machine Learning (ML) has enjoyed huge successes in recent years and an ever- growing number of real-world applications rely on it. However, designing promising algorithms for a specific problem still requires huge human effort. Automated Machine Learning (AutoML) aims at taking the human out of the loop and develop machines that generate / recommend good algorithms for a given ML tasks. AutoML is usually treated as an algorithm / hyper-parameter selection problems, existing approaches include Bayesian optimization, evolutionary algorithms as well as reinforcement learning. Among them, auto-sklearn which incorporates meta-learning techniques in their search initialization, ranks consistently well in AutoML challenges. This observation oriented my research to the Meta-Learning domain. This direction led me to develop a novel framework based on Markov Decision Processes (MDP) and reinforcement learning (RL).After a general introduction (Chapter 1), my thesis work starts with an in-depth analysis of the results of the AutoML challenge (Chapter 2). This analysis oriented my work towards meta-learning, leading me first to propose a formulation of AutoML as a recommendation problem, and ultimately to formulate a novel conceptualisation of the problem as a MDP (Chapter 3). In the MDP setting, the problem is brought back to filling up, as quickly and efficiently as possible, a meta-learning matrix S, in which lines correspond to ML tasks and columns to ML algorithms. A matrix element S(i, j) is the performance of algorithm j applied to task i. Searching efficiently for the best values in S allows us to identify quickly algorithms best suited to given tasks. In Chapter 4 the classical hyper-parameter optimization framework (HyperOpt) is first reviewed. In Chapter 5 a first meta-learning approach is introduced along the lines of our paper ActivMetaL that combines active learning and collaborative filtering techniques to predict the missing values in S. Our latest research applies RL to the MDP problem we defined to learn an efficient policy to explore S. We call this approach REVEAL and propose an analogy with a series of toy games to help visualize agents’ strategies to reveal information progressively, e.g. masked areas of images to be classified, or ship positions in a battleship game. This line of research is developed in Chapter 6. The main results of my PhD project are: 1) HP / model selection: I have explored the Freeze-Thaw method and optimized the algorithm to enter the first AutoML challenge, achieving 3rd place in the final round (Chapter 3). 2) ActivMetaL: I have designed a new algorithm for active meta-learning (ActivMetaL) and compared it with other baseline methods on real-world and artificial data. This study demonstrated that ActiveMetaL is generally able to discover the best algorithm faster than baseline methods. 3) REVEAL: I developed a new conceptualization of meta-learning as a Markov Decision Process and put it into the more general framework of REVEAL games. With a master student intern, I developed agents that learns (with reinforcement learning) to predict the next best algorithm to be tried. To develop this agent, we used surrogate toy tasks of REVEAL games. We then applied our methods to AutoML problems. The work presented in my thesis is empirical in nature. Several real world meta-datasets were used in this research. Artificial and semi-artificial meta-datasets are also used in my work. The results indicate that RL is a viable approach to this problem, although much work remains to be done to optimize algorithms to make them scale to larger meta-learning problems.

Indiana University-Purdue University Indianapolis (IUPUI) / The main contributions in this thesis include the selectively decentralized method in solving multi-agent reinforcement learning problems and the discretized Markov-decision-process (MDP) algorithm to compute the sub-optimal learning policy in completely unknown learning and control problems. These contributions tackle several challenges in multi-agent reinforcement learning: the unknown and dynamic nature of the learning environment, the difficulty in computing the closed-form solution of the learning problem, the slow learning performance in large-scale systems, and the questions of how/when/to whom the learning agents should communicate among themselves. Through this thesis, the selectively decentralized method, which evaluates all of the possible communicative strategies, not only increases the learning speed, achieves better learning goals but also could learn the communicative policy for each learning agent. Compared to the other state-of-the-art approaches, this thesis’s contributions offer two advantages. First, the selectively decentralized method could incorporate a wide range of well-known algorithms, including the discretized MDP, in single-agent reinforcement learning; meanwhile, the state-of-the-art approaches usually could be applied for one class of algorithms. Second, the discretized MDP algorithm could compute the sub-optimal learning policy when the environment is described in general nonlinear format; meanwhile, the other state-of-the-art approaches often assume that the environment is in limited format, particularly in feedback-linearization form. This thesis also discusses several alternative approaches for multi-agent learning, including Multidisciplinary Optimization. In addition, this thesis shows how the selectively decentralized method could successfully solve several real-worlds problems, particularly in mechanical and biological systems.

No description available.

A large class of practical planning problems that require reasoning about uncertain outcomes, as well as tradeoffs among competing goals, can be modeled as Markov decision processes (MDPs). This model has been studied for over 60 years, and has many applications that range from stochastic inventory control and supply-chain planning, to probabilistic model checking and robotic control. Standard dynamic programming algorithms solve these problems for the entire state space. A more efficient heuristic search approach focuses computation on solving these problems for the relevant part of the state space only, given a start state, and using heuristics to identify irrelevant parts of the state space that can be safely ignored. This dissertation considers the heuristic search approach to this class of problems, and makes three contributions that advance this approach. The first contribution is a novel algorithm for solving MDPs that integrates the standard value iteration algorithm with branch-and-bound search. Called branch-and-bound value iteration, the new algorithm has several advantages over existing algorithms. The second contribution is the integration of recently-developed suboptimality bounds in heuristic search algorithm for MDPs, making it possible for iterative algorithms for solving these planning problems to detect convergence to a bounded-suboptimal solution. The third contribution is the evaluation and analysis of some techniques that are widely-used by state-of-the-art planning algorithms, the identification of some weaknesses of these techniques, and the development of a more efficient implementation of one of these techniques -- a solved-labeling procedure that speeds converge by leveraging a decomposition of the state-space graph of a planning problem into strongly-connected components. The new algorithms and techniques introduced in this dissertation are experimentally evaluated on a range of widely-used planning benchmarks.

Background: Social media influencers (SMI’s) can influence the consumers’ decisions by sending out social signals through social functions which social media platforms enable. These social functions SMI’s as well as consumers use, could be for example through sharing content; having a conversation; developing relationships with like-minded individuals; or it could also be used for sending out social signals, such as identity, presence or reputation. SMI’s can play a significant role when it comes to consumers' buying decision process, thus makes it important to research on how the consumers’ buyer decision process potentially could be influenced and therefore shaped, as the staticity of the buyer decision process, hence the five stages, might change depending upon what it is combined with and in regards to what consumer audience. Purpose: The purpose of this paper is to explore how the consumers’ buyer decision process is influenced by social media influencers from the consumers’ perspective. Methodology: This research has been taking advantage of the qualitative research approach in an explanatory nature. As the empirical material had to become thick, subjective and from a consumer's point of view, this research included unstructured interviews allowing the interviews to become conversational-like, thus giving the respondents the chance to control and steer the topics within the scope. Aide-mémoaire’s were used if the interviews developed a complete wrong direction, steering the conversation back. The unstructured interviews included 6 females between the ages of 20-30 years old, this selection were made with purposive sampling method. Findings: This bachelor thesis identified four different factors that were influencing the consumer's buyer decision process, namely, characteristics, trust, risk and drivers. Conclusion: Concluding the thesis, four different factors were found having an influence upon the consumers buyer decision process while being exposed to an SMI. These four factors further played different roles in different stages of the buyer decision process, which is important for companies, brands, marketers and SMIs to observe. Keywords: Buyer Decision Process, SMI’s, Social Media Influencers, Influence

Existing approaches to continuous optimization are essentially mechanisms for deciding which locations should be sampled in order to obtain information about a target function's global optimum. These methods, while often effective in particular domains, generally base their decisions on heuristics developed in consideration of ill-defined desiderata rather than on explicitly defined goals or models of the available information that may be used to achieve them. The problem of numerical optimization is essentially one of deciding what information to gather, then using that information to infer the location of the global optimum. That being the case, it makes sense to model the problem using the language of decision theory and Bayesian inference. The contribution of this work is precisely such a model of the optimization problem, a model that explicitly describes information relationships, admits clear expression of the target function class as dictated by No Free Lunch, and makes rational and mathematically principled use of utility and cost. The result is an algorithm that displays surprisingly sophisticated behavior when supplied with simple and straightforward declarations of the function class and the utilities and costs of sampling. In short, this work intimates that continuous optimization is equivalent to statistical inference and decision theory, and the result of viewing the problem in this way has concrete theoretical and practical benefits.

Flera undersökningar har under de senaste åren visat på hur företag använder sig av miljömässiga budskap i sin marknadsföring för att framhäva sina produkter som mer miljövänliga än vad de i själva verket är, så kallad greenwashing. Genom det enorma utbudet som finns tillgängligt för nutidens konsumenter och de olika influenser de får till sig genom reklam kan det vara svårt för den enskilda konsumenten att avgöra om produkten är trovärdig eller inte. Tidigare forskning inriktat på ämnet har i första hand utgått från en yngre åldersgrupp. Uppsatsens forskning har utformats för att undersöka den brist på forskning som rör åldersgapet genom att studera vilka attribut på dagligvaror som konsumenter mellan 35–60 år upplever som hållbara utifrån ett klimatperspektiv. Undersökningen syftar till att bidra med ökad förståelse för hur attribut på produkter i dagligvaruhandeln påverkar konsumenter inom åldersintervallet att tro att de väljer hållbara produkter. Företag kan framgent dra nytta av resultatet från studien i marknadsföringen av hållbara produkter, samt hur de undviker greenwashing. Andra uppseendeväckande aspekter av ämnet tas upp som hypoteser och testas genom chitvå. I studien undersöks konsumenternas kännedom och attityd till vilseledande marknadsföring och miljömässigt hållbara val i dagligvaruhandeln genom en enkätundersökning. Resultatet analyseras sedan med hjälp av den valda teorin, kognitiv dissonans och köpbeslutsprocessens sista delar, vilka är köpbeslut och efterköpsbeteende. Undersökningen visade att det finns flera attribut som övertygar konsumenten att produkter i dagligvaruhandeln är hållbara. 85 % av respondenterna uppgav att de helst handlade varor som var paketerade av papper och kartong utifrån ett miljöperspektiv. 29 % uppgav att hållbarhetsmärkningar är det som är mest övertygande på produkter att de är hållbara tätt följt av 20 % som uppgav att innehållsförteckningen var det som övertygade mest. Svaren från enkäterna pekar på att det finns många faktorer som påverkar konsumenters gröna köpbeslut i dagligvaruhandeln och att de visuella attribut som en produkt har spelar roll, men att även andra, mer subtila faktorer också är med och påverkar det slutliga valet av att köpa produkten. Ett steg för företag att minska risken för att stämplas för greenwashing är att vara ärliga och transparenta och att det också speglar sig på produktens attribut. Studien är skriven på svenska. / In recent years, several studies have shown how companies use environmental messages in their marketing to highlight their products as more environmentally friendly than they are, so-called greenwashing. Due to the huge range available to today's consumers and the various influences they get through advertising, it can be difficult for the individual consumer to determine if the product is credible or not. Previous research in this field has primarily been based on a younger age group. This study has been designed to investigate the lack of research concerning the age gap by studying which attributes of groceries that consumers between the ages of 35-60 perceive as sustainable from a climate perspective. The survey aims to contribute to an increased understanding of how attributes on food products affect consumers in this age range to believe that they choose sustainable products. In the future, companies can benefit from the results of the study in the marketing of sustainable products, as well as how they avoid greenwashing. Other startling aspects of the subject are taken up as hypotheses and tested by chi square. The study examines consumers' knowledge and attitudes to misleading marketing and environmentally sustainable choices in the grocery trade through a survey. The result is then analyzed with the help of the chosen theories, cognitive dissonance and the last parts of the purchase decision process which are purchase decision and after-purchase behavior. The survey showed that there are several attributes that convince the consumer that products in the grocery trade are sustainable. 85% of the respondents stated that they preferred to buy goods that were packaged from paper and cardboard from an environmental perspective. 29% stated that sustainability labels are the most convincing on products that they are durable, closely followed by 20% who stated that the table of contents was the most convincing. The answers from the surveys indicate that there are many factors that influence consumers' green purchasing decisions in the grocery trade and that the visual attributes that a product has play a role, but that other, more subtle factors also play a role in influencing the final choice to buy the product. One step for companies to reduce the risk of being stamped for greenwashing is to be honest and transparent and that this is also reflected in the product's attributes. This study is written in Swedish.