京都大学 / 0048 / 新制・論文博士 / 博士(理学) / 乙第13323号 / 論理博第1570号 / 新制||理||1663(附属図書館) / 京都大学理学研究科生物科学専攻 / (主査)教授 伊谷 原一, 教授 平田 聡, 教授 幸島 司郎 / 学位規則第4条第2項該当 / Doctor of Science / Kyoto University / DGAM

Recommendation systems serve to users of e-commerce applications for individual recommendations to certain products or services based on their preferences. The aim of this thesis is to create a module of recommender system. The work includes analysis of recommendation systems and the methods used in these systems, including a description of the calculations. This work also solves the cold start problem, which is a problem when generation of some good recommendations for the new user is needed, but the recommendation system has no or little information about this user. Based on analysis is in this thesis designed module for recommender system, which is applicable e.g. internet for e-commerce or other internet-based application. Part of this module is the realization of a platform Apache Mahout, which some parts are built on a distributed computing platform Apache Hadoop project. Furthermore, in this work, on the aforementioned platform Mahout, selected methods of calculating the similarity using selected criteria (e.g. the average time for a recommendation, and the number of users for who have not been able to generate recommendations) are tested.

abstract: As the size and scope of valuable datasets has exploded across many industries and fields of research in recent years, an increasingly diverse audience has sought out effective tools for their large-scale data analytics needs. Over this period, machine learning researchers have also been very prolific in designing improved algorithms which are capable of finding the hidden structure within these datasets. As consumers of popular Big Data frameworks have sought to apply and benefit from these improved learning algorithms, the problems encountered with the frameworks have motivated a new generation of Big Data tools to address the shortcomings of the previous generation. One important example of this is the improved performance in the newer tools with the large class of machine learning algorithms which are highly iterative in nature. In this thesis project, I set about to implement a low-rank matrix completion algorithm (as an example of a highly iterative algorithm) within a popular Big Data framework, and to evaluate its performance processing the Netflix Prize dataset. I begin by describing several approaches which I attempted, but which did not perform adequately. These include an implementation of the Singular Value Thresholding (SVT) algorithm within the Apache Mahout framework, which runs on top of the Apache Hadoop MapReduce engine. I then describe an approach which uses the Divide-Factor-Combine (DFC) algorithmic framework to parallelize the state-of-the-art low-rank completion algorithm Orthogoal Rank-One Matrix Pursuit (OR1MP) within the Apache Spark engine. I describe the results of a series of tests running this implementation with the Netflix dataset on clusters of various sizes, with various degrees of parallelism. For these experiments, I utilized the Amazon Elastic Compute Cloud (EC2) web service. In the final analysis, I conclude that the Spark DFC + OR1MP implementation does indeed produce competitive results, in both accuracy and performance. In particular, the Spark implementation performs nearly as well as the MATLAB implementation of OR1MP without any parallelism, and improves performance to a significant degree as the parallelism increases. In addition, the experience demonstrates how Spark's flexible programming model makes it straightforward to implement this parallel and iterative machine learning algorithm. / Dissertation/Thesis / M.S. Computer Science 2014

This paper explores how known machine learning techniques can be applied in unique ways to simplify software and therefore dramatically increase its usability. As software has increased in popularity, its complexity has increased in lockstep, to a point where it has become burdensome. By shifting the focus from the software to the user, great advances can be achieved by way of simplification. The example problem used in this report is well known: suggest local dining choices tailored to a specific person based on known habits and those of similar people. By analyzing past choices and applying likely probabilities, assumptions can be made to reduce user interaction, allowing the user to realize the benefits of the software faster and more frequently. This is accomplished with Java Servlets, Apache Mahout machine learning libraries, and various third party resources to gather dimensions on each recommendation. / text

In recent years, web has experienced a tremendous growth concerning users and content. As a result information overload problem has always been always one of the main discussion topics. The aim has always been to find the most desired solution in order to help users when they find it increasingly difficult to locate the accurate information at the right time. Recommender systems developed to address this need by helping users to find relevant information among huge amounts of data and they have now become a ubiquitous attribute to many websites. A recommender system guides users in their decisions by predicting their preferences while they are searching, shopping or generally surfing, based on their preferences collected from past as well as the preferences of other users. Until now, recommender systems has been vastly used in almost all professional e-commerce websites, selling or offering different variety of items from movies and music to clothes and foods. This thesis will present and explore different recommender system algorithms such as User-User Collaborative and Item-Item Collaborative filtering using open source library Apache mahout. Algorithms will be developed in order to evaluate the performance of these collaborative filtering algorithms. They will be compared and their performance will be measured in detail by using evaluation metrics such as RMSE and MAE and similarity algorithms such as Pearson and Loglikelihood.