Machine Learning (ML) is nowadays being offered as a service by several cloud providers. Consumers require metrics to be able to evaluate and compare between multiple ML cloud services. There aren’t many established metrics that can be used specifically for these types of services. In this paper, the Goal-QuestionMetric paradigm is used to define a set of metrics applicable for ML cloud services. The metrics are created based on goals expressed by professionals who use or are interested in using these services. At the end, a questionnaire is used to evaluate the metrics based on two criteria: relevance and ease of use.

Machine learning is a growing area of artificial intelligence that is widely used in our world today. Training machine learning models requires knowledge and computing power. Machine Learning as a Service (MLaaS) tries to solve these issues. By storing the datasets and using virtual computing instances in the cloud, one can create machine learning models without writing a single line of code. When selecting an MLaaS platform to use, the natural question of which one to use arises. This thesis conducts controlled experiments to compare the image classification capabilities of Microsoft Azure ML, Amazon Web Services SageMaker, and Google Cloud Platform Vertex AI. The prediction accuracy, training time, and cost will be measured with three different datasets. Some subjective comments about the user experience while conducting these experiments will also be provided. The results of these experiments will be used to make recommendations as to which MLaaS platform to use depending on which metric is most suitable. This thesis found that Microsoft Azure ML performed best in terms of prediction accuracy, and training cost, across all datasets. Amazon Web Services SageMaker had the shortest time to train but performed the worst in terms of accuracy and had trouble with two of the three datasets. Google Cloud Platform Vertex AI did achieve the second-bestprediction accuracy but was the most expensive platform by far as it had the largest time to train. It did, however, provide the smoothest user experience.Overall, Azure ML would be the platform of choice for image classification tasks after weighing together the results of the experiments as well as the subjective user experience.

Video is a large part of today’s society where surveillance cameras represent the biggest source of big data, and real-time entertainment is the largest network traffic category. There is currently a large interest in analysing the contents of video where video analysis is mainly conducted by people. This increase in video has for instance made it difficult for professional editors to analyse movies and series in a scalable way, and alternative solutions are needed. The media technology company June, want to explore scalable alternatives for extracting metadata from video. With recent advances in Machine Learning and the rise of machine-learning-asa-service platforms, June wished more specifically to explore how these Machine Learning services can be utilised for extracting metadata from videos, and from it construct a summary regarding its contents. This work examined Machine Learning as an option for scalable video summarisation which resulted in developing and evaluating an application that utilised transcription, summarisation, and translation services to produce a text based summarisation of video. Furthermore to examine the services current state of affairs, multiple services from different providers were tested, evaluated and compared to each other. Lastly, in order to evaluate the summarisation services an evaluation model was developed. The test results showed that the translation services were the only service that produced good results. Transcription and summarisation performed poorly in the tests which renders the suggested solution of combining the three services for video summarisation as impractical. / Video är en stor del av dagens samhälle där bland annat övervakningskameror är den största källan av data och underhållning i realtid är den kategori som står för mest nätverkstrafik. Det finns i dagsläget ett stort intresse i att analysera innehållet av video, denna videoanalys utförs även främst av människor. Ökningen av video har gjort det svårt för exempelvis professionella redaktörer att hinna analysera filmer och serier och mer skalbara alternativ behövs. Mediaföretaget June vill utforska alternativ för att extrahera metadata från video på ett skalbart sätt. Med de senaste framstegen inom maskininlärning och framväxten av machine-learningas-a-service plattformar, önskar June mer specifikt att utforska hur maskininlärning kan nyttjas för att extrahera metadata från video och med det konstruera en sammanfattning av innehållet. Det utförda arbetet undersökte maskininlärning som skalbart alternativ för att kunna sammanfatta videos innehåll. Arbetet resulterade i utvecklandet samt utvärderingen av en applikation som nyttjade maskininlärningstjänster för transkribering, sammanfattning samt översättning för att producera en textbaserad sammanfattning av videos innehåll. För att utvärdera tjänsternas nuvarande tillstånd så testades samt utvärderades tjänster från olika leverantörer för att sedan jämföras mot varandra. Slutligen framtogs en egenutvecklad modell för att kunna utvärdera tjänsterna för sammanfattning. Testresultaten visade att tjänsterna för översättning var de enda tjänsterna som gav bra resultat. Tjänsterna för transkribering och sammanfattning gav dåliga resultat vilket gör den föreslagna lösningen av att kombinera de tre tjänsterna för att sammanfatta videoinnehåll som opraktisk.