Similarity plays a central role in language understanding process. However, it is always difficult to precisely define on which type of data and what similarity metrics we can apply in order to assess the similarity of two texts. According to this spirit, the task Semantic Textual Similarity (STS) was introduced as a pilot task at the Semantic Evaluation (SemEval) workshop in year 2012. This thesis seeks to investigate the variances of performance of STS systems with respect to the heterogeneous data sources, and find solutions to alleviate these variances to improve the system performance. We carry a series of works focusing on addressing different aspects of measuring semantic similarity for texts under the umbrella of the Semantic Textual Similarity task. Firstly, we analyze the variance of system performance on dierent corpora with preliminary experiments and propose the hypothesis that system performance depends heavily on the type of train and test corpora coming from heterogeneous sources. We analyze a standard textual similarity model built on vectorial representation and we derive a couple of modalities which help significantly alleviating the negative in influence of vectorial mapping model. In particular, we study how structural information and the most advanced word alignment models in Machine Translation improve the accuracy of systems. Our analysis also leads us to carry out, for the first time, an analysis between Semantic Relatedness and Textual Entailment, then we propose a co-learning model to improve the accuracy on both tasks by exploiting their mutual relationship. As a result, all these steps lead to a consistent improvement over the standard model which is manifested across corpora. The evaluation shows that our system systematically achieves and goes beyond the former state of the art, whereas it also reduces the variation of the accuracy on various types of corpora.

Wireless-enabled electronic devices are becoming cheaper, more powerful and thus more popular. They include sensors, actuators, smartphones, tablets, wearable devices, and other complex devices. They can carry out complex tasks, cooperating with their ``neighbors''. However, it is difficult to develop mobile applications to exploit the full power of available resources because the computational capabilities on devices are not homogeneous, and their connectivity changes with physical movement. We propose a mobile environment model to describe the connected devices and study the structural and behavioral characteristics of the environments. Based on the model, we design the routing protocols and a language to support the composition of environments. We propose a framework to provide a unified, flexible and scalable service for task/process deployment and execution on top of the heterogeneous and dynamic mobile environments. We compare different architectures, and discuss the optimization of resources discovery and routing algorithm. A proof-of-concept framework is implemented and shows the feasibility of our Environment-as-a-Service approach. Finally, we explore the theoretical principles and practical techniques for performance optimization, including a data prefetching technique and a dynamic process/task allocation algorithm.

Music making is among the activities that best fulfil a person’s full potential, but it is also one of the most complex and exclusive: successful music making requires study and dedication, combined with a natural aptitude that only gifted individuals possess. This thesis proposes new design solutions to reproduce the human ability to make music. It offers insights to provide the general public with novel experiences of music making by exploring a different interactive metaphor. Emotions are proposed as a mediator of musical meanings: an algorithmic composer is developed to generate new music, and the player can interact with the composition, controlling the desired levels of the composition’s emotional character. The adequacy of this metaphor is tested with the case study of The Music Room, an interactive installation that allows visitors to influence the emotional aspect of an original classical style musical composition by means of body movements. This thesis addresses research questions and performs exploratory studies that are grounded in and contribute to different fields of research, including musical interface design, algorithmic composition, and psychology of music. The thesis presents MINUET, a conceptual framework for the design of musical interfaces, and the Music Room, an example of interactive installation based on the emotional metaphor. The Music Room was the result of a two-year iteration of design and evaluation cycles that informed an operational definition of the concept of engagement with interactive art. New methods for evaluating visitors’ experience based on the integration of evidences from different user-research techniques are also presented. As regards the field of algorithmic composition, the thesis presents Robin, a rule-based algorithmic affective composer, and a study to test its validity in eliciting different emotions in listeners (N=33). Valence (positive vs. negative) and arousal (high vs. low) were manipulated in a 2*2 within-subjects design. Results showed that Robin correctly elicited valence and arousal in converging conditions (high valence, high arousal and low valence, low arousal). However, in cases of diverging conditions (high valence, low arousal and low valence, high arousal), valence received neutral values. As regards the psychology of music, this thesis contributes new evidence to the on-going debate about the innate or learned nature of musical competence, defined as the ability to recognise emotion in music. Results of an experimental study framed within Russell’s two-dimensional theory of emotion suggest that musical competence is not affected by training when listeners are required to evaluate arousal (dictated by variations of tempo). The evaluation of valence (dictated by the combination of tempo and mode), however, was found to be more complicated, highlighting a difference in the evaluation of musical excerpts when tempo and mode conveyed diverging emotional information. In this debate, Robin is proposed as a suitable tool for future experimental research as it allows the manipulation of individual musical factors.

Network representation learning (NRL) and cascade representation learn- ing (CRL) are fundamental backbones of different kinds of network analysis problems. They are usually carried out in settings where the structure of the network under consideration is known. Motivated by real-world prob- lems, this study presents several algorithms for scenarios where the network structure is partially or completely unknown. The objective of network representation learning is to identify a mapping function that projects sparse and high-dimensional network graphs into a dense latent representation, which preserves the original information about nodes and their neighborhoods. The notion of neighborhood, however, be- comes illusive when the network structure is partially or completely hidden. Inspired by previous results, in our thesis work we have developed novel algorithms that are resilient to such lack of knowledge. These results estab- lish a correlation between the properties of the network and different kind of node activities performed over it, information which is generally more available and can be easily observed. In particular, we focus on diffusion events – also called cascades – such as shares, retweets and hashtags. In the first of our contributions, we have developed a novel NRL algorithm called Mineral, a simple technique that combines the observed cascades with the partially accessible network structure by sampling artificial cas- cades. Node representation is then learned from the observed and sampled cascades by using the SkipGram model that is widely used for word representation learning in natural language documents. In our second contribution, called NetTensor, we assume that the network structure is completely hidden and we propose novel techniques that are capable to estimate both the hidden neighborhood (proximity) and the similarity of nodes. Such estimated values are then used to learn a unified embedding of nodes using a scalable truncated singular value decomposition and deep autoencoders. In addition to the NRL algorithms, we have also proposed a novel CRL algorithm called cas2vec for virality (popularity) prediction. Again, we pursue a network-agnostic approach following the above assumption that the network structure is completely unknown. Unlike prior studies that rely on manual feature extraction, cas2vec automatically learns cascade representations based on convolutional neural networks, that are effective in predicting virality of cascades. We have carried out extensive experiments using several real-world datasets for all of our methods and compared them against strong baselines from the state-of-the-art, achieving significantly better results than many of them.

As the explosive growth of the visual data, it is particularly important to develop intelligent visual understanding techniques for dealing with a large amount of data. Many efforts have been made in recent years to build highly effective and large-scale visual processing algorithms and systems. One of the core aspects in the research line is how to learn robust representations to better describe the data. In this thesis we study the problem of visual image and video understanding and specifically, we address the problem via designing and implementing novel multi-modal and structured representation learning approaches, both of which are fundamental research hot-spots in machine learning. Multi-modal representation learning involves relating information from multiple input sources, and the structured representation learning works on exploring rich structural information hidden in the data for robust feature learning. We investigate both the shallow representation learning frameworks such as dictionary learning and the deep representation learning frameworks such as deep neural networks, and present different modules devised in our works, consisting of cross-paced representation learning, cross-modal feature learning and transferring, multi-scale structured prediction and fusion, multi-modal prediction and distillation. These techniques are further applied in various visual understanding topics, i.e. sketch-based-image retrieval (SBIR), video pedestrian detection, monocular depth estimation and scene parsing, showing superior performance.

Poor diet and physical inactivity are two of the biggest healthcare challenges we are facing, and are related to individuals lifestyle. In fact, a poor lifestyle is strongly correlated to chronic diseases, the leading causes for morbidity and mortality. Adhering to a healthy diet and following an active lifestyle are thus necessary to promote the overall health. However, maintaining a healthy diet and physically active lifestyle is hard. This is due to poor health literacy, lack of awareness, motivation and effective intervention support. Recent years have seen a blast of mHealth apps for health promotion, targeting in particular dietary behavior change. However, reviews showed difficulties in effective adoption and use of these applications in long-term health promotion. Contemporary approaches have focused on tracking user condition and few have analyzed aspects of user interaction with the system. To promote individuals health, users can benefit from some form of tailored guidance or coaching. That said, to ensure adequate users support, personalized care with a human agent in the loop can enhance the care delivery. Due to the increasing demand for continuous care by users and the shortage of caregiver resources, current health services are inefficient relative to user support and decreasing caregivers workload. Digital health devices can act as a key player in providing interactive health activities (via mobile and telemedicine systems), enhancing self-monitoring (through wearable tracker) and tailored coaching (using either automated or manual coaching systems). However, they’re ineffective in providing continuous health services and creating a balance between users support and caregivers workload. In addition, even with the technology existence, there is low motivation to maintain a healthy diet or exercise routines. Individuals use messaging applications as part of their regular daily routines. We harness the power of messaging chatbot systems to provide behavior change interventions for healthy lifestyle promotion. We particularly introduce the role of chatbot in task automation and adhering users to a health plan. Thus, in this thesis we present the concept of "Conversational User Interface in Health Coaching Interventions" that consists of a just-in-time health services to users and caregivers. We discuss ways to integrate the chatbot to assist caregivers with their tasks and support users with their condition. We get users to cue themselves to action by attaching the chatbot with users’ daily messaging routines. The service will eliminate the technology barrier and impairment for the users i.e., elderly. The chatbot accesses reliable user compliance data, sets adherence reminders by condition, and reports daily individuals adherence. The chatbot alerts the coach through a web application in critical cases. The approach facilitates adherence to health interventions by investigating a human-virtual agent mediated coaching approach on user motivation to adhere to the health promotion plan. The approach was validated with different experimentation phases. Using multiple research methods, this dissertation has made several contributions to the understanding of user motivation and the role of a semi-automated system with a human and virtual agent in tracking individuals with poor lifestyle. We will discuss the main contributions and experimentation results throughout the thesis.

The use of Deep Neural Networks with their increased representational power has allowed for great progress in core areas of computer vision, and in their applications to our day-to-day life. Unfortunately the performance of these systems rests on the "big data" assumption, where large quantities of annotated data are freely and legally available for use. This assumption may not hold due to a variety of factors: legal restrictions, difficulty in gathering samples, expense of annotations, hindering the broad applicability of deep learning methods. This thesis studies and provides solutions for different types of data scarcity: (i) the annotation task is prohibitively expensive, (ii) the gathered data is in a long tail distribution, (iii) data storage is restricted. For the first case, specifically for use in video understanding tasks, we have developed a class agnostic, unsupervised spatio-temporal proposal system learned in a transductive manner, and a more precise pixel-level unsupervised graph based video segmentation method. At the same time, we have developed a cycled, generative, unsupervised depth estimation system that can be further used in image understanding tasks, avoiding the use of expensive depth map annotations. Further, for use in cases where the gathered data is scarce we have developed two few-shot image classification systems: a method that makes use of category-specific 3D models to generate novel samples, and one that increases novel sample diversity by making use of textual data. Finally, data collection and annotation can be legally restricted, significantly impacting the function of lifelong learning systems. To overcome catastrophic forgetting exacerbated by data storage limitations, we have developed a deep generative memory network that functions in a strictly class incremental setup.

Human face and behavior analysis are very important research topics in the field of computer vision and they have broad applications in our everyday life. For instance, face alignment, face aging, face expression analysis and action recognition have been well studied and applied for security and entertainment. With these face analyzing techniques (e.g., face aging), we could enhance the performance of cross-age face verification system which now has been used for banks and electronic devices to recognize their clients. With the help of action recognition system, we could better summarize the user uploaded videos or generate logs for surveillance videos. This could help us retrieve the videos more accurately and easily. The dictionary learning and neural networks are powerful machine learning models for these research tasks. Initially, we focus on the multi-view action recognition task. First, a class-wise dictionary is pre-trained which encourages the sparse representations of the between-class videos from different views to lie close by. Next, we integrate the classifiers and the dictionary learning model into a unified model to learn the dictionary and classifiers jointly. For face alignment, we frame the standard cascaded face alignment problem as a recurrent process by using a recurrent neural network. Importantly, by combining a convolutional neural network with a recurrent one we alleviate hand-crafted features to learn task-specific features. For human face aging task, it takes as input a single image and automatically outputs a series of aged faces. Since human face aging is a smooth progression, it is more appropriate to age the face by going through smooth transitional states. In this way, the intermediate aged faces between the age groups can be generated. Towards this target, we employ a recurrent neural network. The hidden units in the RFA are connected autoregressively allowing the framework to age the person by referring to the previous aged faces. For smile video generation, one person may smile in different ways (e.g., closing/opening the eyes or mouth). This is a one-to-many image-to-video generation problem, and we introduce a deep neural architecture named conditional multi-mode network (CMM-Net) to approach it. A multi-mode recurrent generator is trained to induce diversity and generate K different sequences of video frames.

The Industry 4.0 challenge is to exploit the synergy of different technologies in order to achieve the results required by its specifications. This chapter presents: (a) the state of the art in Augmented Reality applied to industrial engineering and manufacturing machines, (b) insights on the implementation of optimal feed-rate interpolation for computer numerical control machine tools, (c) an application of knowledge-based techniques such as computer algebra systems in the implementation of solvers for optimal control problems, and (d) challenges in the application of artificial neural networks to the massive amount of unlabeled data available in the industrial practice. It is shown how these topics, wich may appear as distant one from each other, play a central and correlated role in the Industry 4.0.

Free and Open Source Software (FOSS) components are ubiquitous in both proprietary and open source applications. In this dissertation we discuss challenges that large software vendors face when they must integrate and maintain FOSS components into their software supply chain. Each time a vulnerability is disclosed in a FOSS component, a software vendor must decide whether to update the component, patch the application itself, or just do nothing as the vulnerability is not applicable to the deployed version that may be old enough to be not vulnerable. This is particularly challenging for enterprise software vendors that consume thousands of FOSS components, and offer more than a decade of support and security fixes for applications that include these components. First, we design a framework for performing security vulnerability experimentations. In particular, for testing known exploits for publicly disclosed vulnerabilities against different versions and software configurations. Second, we provide an automatic screening test for quickly identifying the versions of FOSS components likely affected by newly disclosed vulnerabilities: a novel method that scans across the entire repository of a FOSS component in a matter of minutes. We show that our screening test scales to large open source projects. Finally, for facilitating the global security maintenance of a large portfolio of FOSS components, we discuss various characteristics of FOSS components and their potential impact on the security maintenance effort, and empirically identify the key drivers.