While classifications are heavily used to categorize web content, the evolution of the web foresees a more formal structure -- ontology - which can serve this purpose. Ontologies are core artifacts of the Semantic Web which enable machines to use inference rules to conduct automated reasoning on data. Lightweight ontologies bridge the gap between classifications and ontologies. A lightweight ontology (LO) is an ontology representing a backbone taxonomy where the concept of the child node is more specific than the concept of the parent node. Formal lightweight ontologies can be generated from their informal ones. The key applications of formal lightweight ontologies are document classification, semantic search, and data integration. However, these applications suffer from the following problems: the disambiguation accuracy of the state of the art NLP tools used in generating formal lightweight ontologies from their informal ones; the lack of background knowledge needed for the formal lightweight ontologies; and the limitation of ontology reuse. In this dissertation, we propose a novel solution to these problems in formal lightweight ontologies; namely, faceted lightweight ontology (FLO). FLO is a lightweight ontology in which terms, present in each node label, and their concepts, are available in the background knowledge (BK), which is organized as a set of facets. A facet can be defined as a distinctive property of the groups of concepts that can help in differentiating one group from another. Background knowledge can be defined as a subset of a knowledge base, such as WordNet, and often represents a specific domain.

It is a well known fact that high quality Automatic Speech Recognition is still difficult to guarantee under conditions in which the speaker is distant from the microphone due to the distortions caused by acoustic phenomena, such as noise and reverberation. Among the different research directions pursued around this problem, the adoption of multi-channel approaches is of great interest to the community given the potential of taking advantage of information diversity. In this thesis we elaborate on approaches that exploit different instances of a sound source, captured by various largely spaced microphones, in order to extract a Distant Speech Recognition hypothesis. Two original solutions are presented, based on information fusion approaches at different levels of the recognition system, one at front-end stage and one at post-decoding stage, namely for the problems of channel selection (CS) and hypothesis combination. First, a new CS framework is proposed. Cepstral distance (CD), which is effectively applied in other acoustic processing fields, is the basis of the CS method developed. Experimental results confirmed the advantages of a CD-based selection schema under different scenarios. The second contribution concerns the combination of information extracted from the individual decoding processes performed over the multiple captured signals. It is shown how temporal cues can be identified in the hypothesis space, and be beneficial for the elaboration of a multi-microphone confusion network, from which the final speech transcription is derived. The proposed methods are applicable in a setting equipped with synchronized distributed microphones, independently of the proximity between the sensors. Analysis of the novel concepts were performed over synthetic and real-captured data. Both approaches achieved positive results at the different assessment tasks they were exposed to.

Network Representation Learning (NRL) aims at learning a low-dimensional latent representation of nodes in a graph while preserving the graph information. The learned representation enables to easily and efficiently perform various machine learning tasks. Graphs are often associated with diverse and rich information such as attributes that play an important role in the formation of the network. Thus, it is imperative to exploit this information to complement the structure information and learn a better representation. This requires designing effective models which jointly leverage structure and attribute information. In case of a heterogeneous network, NRL methods should preserve the different relation types. Towards this goal, this thesis proposes two models to learn a representation of attributed graphs and one model for learning representation in a heterogeneous network. In general, our approach is based on appropriately modeling the relation between graphs and attributes on one hand, between heterogeneous nodes on the other, executing a large collection of random walks over such graphs, and then applying off-the-shelf learning techniques to the data obtained from the walks. All our contributions are evaluated against a large number of state-of-the-art algorithms, on several well-known datasets, obtaining better results.

In distributed, open evironments, possibly heterogeneous computational entities need to engage in complex interactions in order to complete tasks and often have to face sudden changes; it therefore becomes essential for modern information systems to adopt coordination technologies which support dynamic and flexible interactions among processes, whether reactive (e.g., web services) or proactive (e.g., autonomous agents). Substantial efforts are being put forward to devise suitable mechanisms for process coordination. In the past few years, interaction-oriented frameworks have been proposed, which enable distributed and heterogeneus agents to engage in coordination activities by sharing interaction models specified in executable protocol languages. Software systems have started to be developed to apply such frameworks to concrete use. In particular, the OpenKnowledge framework has been proposed as such an interaction-oriented framework, and the OpenKnowledge (OK) system has been developed for its realization. Such system provides a distributed infrastructure which allows a-priori unknown peers to gather together and coordinate with each other by publishing, discovering and executing interaction models specified in the Lightweight Coordination Calculus (LCC) protocol language. Although the realization of the OpenKnowledge approach is promising, its application in realistic, complex scenarios is not fully exploited. This thesis aims at applying the OpenKnowledge framework to realistic contexts such as emergency response (e-Response). Its main contribution is in the design and simulation of emergency response scenarios which are expressed in terms of LCC specifications, and are enacted by means of a simulation environment fully integrated with the OK system. Such environment is developed to: (1) informally validate the e-Response scenarios; (2) test the capability of the OK system to support such scenarios, and (3) provide a preliminary evaluation of the efficacy of different information gathering strategies (i.e., centralized or distributed) in emergency response settings. The results obtained show that the OK system is able to support complex coordination tasks; however, some limitations have emerged in relation to the discovery mechanism. Furthermore, simulations have shown to adhere with realistic scenarios, and that - under ideal conditions - centralized and decentralized information-gathering strategies are comparable.

Crowdsourcing is the outsourcing of a unit of work to a crowd of people via an open call for contributions. While there are various forms of crowdsourcing, such as open innovation, civic engagement and crowdfunding in this work we specifically focus on microtasking. Microtasking is a branch of crowdsourcing, where a work is presented as a set of identical microtasks, each requiring contributors only several minutes to complete usually in exchange for a reward of less than 1 USD. Labeling images, transcribing documents, analyzing sentiments of short sentences and cleaning datasets are popular examples of work which could be solved as microtasks. Available up to date microtask crowdsourcing platforms, such as CrowdFlower and Amazon Mechanical Turk, allow thousands of microtasks to be solved in parallel by hundreds of contributors available online. To tackle the problem of quality in microtask crowdsourcing, it is necessary to study different quality attributes, to investigate what causes low quality of results and slow task execution in microtask crowdsourcing, to identify effective methods to both assess and assure that these quality attributes are of high level. We conducted the most extensive literature review analysis of quality attributes, assessment and assurance techniques ever done in the area of microtasking and crowdsourcing in general. We further advanced the state of the art in three research tracks: i) Improving accuracy and execution speed (the major track), where we monitor in-page user activity of each individual worker, automatically predict abandoned assignments causing delays and assignments with low quality of results, and relaunch them to other workers using our tool ReLauncher; ii) Crowdsourcing complex processes, where we introduce BPMN-extensions to design business processes of both crowd and machine tasks, and the crowdsourcing platform Crowd Computer to deploy these tasks; and iii) Improving workers user experience, where we identify problems workers face searching for tasks to work on, address these problems in our prototype of the task listing interface and introduce a new mobile crowdsourcing platform, CrowdCafe, designed in a way to optimize task searching time and to motivate workers with tangible rewards, such as a coffee.

Web applications have become integral part of everyday life, as they are used by a huge number of customers on regular basis, for daily operations in business, leisure, government or academia, and so correctness of these applications is fundamental. In particular, security is a crucial concern especially for these applications that are constantly exposed to potentially malicious environments. Cross-site scripting (XSS for short) is considered one of the major threats to the security of web applications. Missing input validation can be exploited by attackers to inject malicious code into the application under attack. Static analysis supports manual security review in mitigating the impact of XSS-related issues, by suggesting a set of potential problems, expressed in terms of candidate vulnerabilities. A security problem spotted by static analysis, however, only consists of a list of (possibly complicated) conditions that should be satisfied to concretely exploit a vulnerability. Static analysis does not provide examples of what input values must be used to make the application execute the sometimes complex execution path that causes a XSS vulnerability. Executable test cases, on the contrary, consist of a runnable and reproducible evidence of the vulnerability mechanics. Then, test cases represent a valuable support for developers who should concretely understand security problems in detail before fixing them. The urge for reliable and secure web applications motivates the development of automatic, inexpensive, thus effective security testing methods, whose aim is to verify the presence of security-related defects. Security tests consist of two major parts, input values that need to be generated to run the application in the hope of exposing the vulnerabilities, and the decision if the obtained output actually exposes the vulnerabilities, the latter is known as the “oracle”. However, current approaches to either generate security tests and to define security oracles have limitations. To address the shortcomings of approaches for input value generation for security, this dissertation proposes a structured approach, inspired by software testing, based on the combination of genetic algorithms and concrete symbolic execution. This combined strategy is compared with genetic algorithms and with concrete symbolic execution in their atomic forms, in terms of coverage and efficiency on four case study web applications, showing to be effective for security testing. In fact, genetic algorithms resulted to be able to generate input values only for few and simple vulnerabilities when not combined with other approaches. However, their contribution is fundamental to improve the coverage of those input values generated by concrete symbolic execution. The dissertation also explores the possibility to define oracle components that can be integrated with input generation strategies to perform security testing of web applications, so to expose security-related faults. A security oracle can be seen as a classifier able to detect when a vulnerability is exploited by a test case, i.e. verifying if a test case is an instance of a successful attack. This dissertation presents two distinct approaches to define security oracles, either (1) by applying tree kernel methods, and (2) by resorting to a model of the application under analysis when run in harmless situations. In the former approach, the classifier is trained on a set of test cases containing both safe executions and successful attacks, in the aim of learning important structural properties of web pages. In the latter, the learning phase is devoted to analyze web pages generated only in safe conditions, in order to build a “safe” model of their syntactic structure. Then, in the actual testing phase, both oracles are used to classify new output pages either as “safe tests” or as “successful attacks”. Furthermore, the dissertation moves few steps onto the world of applications for smartphone, in the attempt of breaking the barriers of our research and bringing the lesson learned from the experience in the domain of web applications towards a new domain. To motivate our work, we noticed that an important reason behind the popularity of smartphones and tablets is the huge amount of available applications to download, to expand functionalities of the devices with brand new features. Official stores provide a plethora of applications developed by third parties, for entertainment and business, mostly for free. Again, security represents a fundamental requirement: for example, confidential data (e.g., phone contacts, global GPS position, banking data and emails) might be disclosed by vulnerable applications and so, sensitive applications should carefully be tested to avoid security problems. The dissertation proposes a novel approach to perform security testing with respect to the communication among applications on mobile devices with the objective of spotting errors in the routines that validate incoming messages.

The aim of this PhD thesis is to make a step forward towards teaching computers to understand videos in a similar way as humans do. In this work we tackle the video classification and/or action recognition tasks. This thesis was completed in a period of transition, the research community moving from traditional approaches (such as hand-crafted descriptor extraction) to deep learning. Therefore, this thesis captures this transition period, however, unlike image classification, where the state-of-the-art results are dominated by deep learning approaches, for video classification the deep learning approaches are not so dominant. As a matter of fact, most of the current state-of-the-art results in video classification are based on a hybrid approach where the hand-crafted descriptors are combined with deep features to obtain the best performance. This is due to several factors, such as the fact that video is a more complex data as compared to an image, therefore, more difficult to model and also that the video datasets are not large enough to train deep models with effective results. The pipeline for video classification can be broken down into three main steps: feature extraction, encoding and classification. While for the classification part, the existing techniques are more mature, for feature extraction and encoding there is still a significant room for improvement. In addition to these main steps, the framework contains some pre/post processing techniques, such as feature dimensionality reduction, feature decorrelation (for instance using Principal Component Analysis - PCA) and normalization, which can influence considerably the performance of the pipeline. One of the bottlenecks of the video classification pipeline is represented by the feature extraction step, where most of the approaches are extremely computationally demanding, what makes them not suitable for real-time applications. In this thesis, we tackle this issue, propose different speed-ups to improve the computational cost and introduce a new descriptor that can capture motion information from a video without the need of computing optical flow (which is very expensive to compute). Another important component for video classification is represented by the feature encoding step, which builds the final video representation that serves as input to a classifier. During the PhD, we proposed several improvements over the standard approaches for feature encoding. We also propose a new feature encoding approach for deep feature encoding. To summarize, the main contributions of this thesis are as follows3: (1) We propose several speed-ups for descriptor extraction, providing a version for the standard video descriptors that can run in real-time. We also investigate the trade-off between accuracy and computational efficiency; 
(2) We provide a new descriptor for extracting information from a video, which is very efficient to compute, being able to extract motion information without the need of extracting the optical flow; (3) We investigate different improvements over the standard encoding approaches for boosting the performance of the video classification pipeline.;(4) We propose a new feature encoding approach specifically designed for encoding local deep features, providing a more robust video representation.

This research thesis aims to provide a novel approach to Emotion Recognition of Images: based on empirical studies, we employ the state-of-the-art computer vision techniques in order to understand what makes an abstract artwork emotional. We identify and quantify the emotional regions of abstract paintings. We also investigate the contributions of the main aspects present on abstract artworks (i.e., colour, shape and texture) to automatically predict emotional valence of them. By using eye-tracking recordings we investigate the link between the detected emotional content and the way people look at abstract paintings. We apply a bottom-up saliency model to compare with eye-tracking in order to predict the emotional salient regions of abstract paintings. Finally, we use the metadata associated to the paintings (e.g., title, description and/or artist statement) and correlate it with the emotional responses of the paintings. This research opens opportunity to understand why an abstract painting is perceived as emotional from global and local scales. Moreover, this work provides to art historians and art researches with a new perspective on the analysis of abstract paintings.

Deciphering how neurons represent the external world is a fundamental goal in neuroscience. This requires identifying which features in the population response in a single trial are informative about the stimulus. Neurons can code stimuli using both space and time. Individual neurons show differential selectivity to certain stimuli across space at coarse time scales while representing others by modulating their activity at fine time scales. The information content in the population is modified from neural interactions across space and time. While this emphasizes the need to examine population responses across space and time, analyzing a population of hundreds of neurons is challenging when only a limited number of trials are available due to the high dimensionality of the joint spatiotemporal response space. We addressed this by introducing a novel method called space-by-time non-negative matrix factorization. The method describes the population activity with a low dimensional representation consisting of spatial modules, groups of neurons that are coactivated, and temporal modules, patterns that describe how these neurons modulate their spiking across time. The population activity in each trial is described by a set of coefficients, that indicate the level of activation of each spatial and temporal module in the trial. We used this method to analyze datasets from auditory, visual and somatosensory modalities. It identified physiologically meaningful spatial and temporal modules that described how each population coded stimuli in space and time. It further indicated the differential contributions of spatial and temporal dimensions for the population code. Particularly, the first spike latency was demonstrated to be informative at the population level. We refined the method to model the sub-Poisson, Poisson and supra-Poisson variability typically observed in spike counts. This refinement demonstrated enhanced capacity in identifying spatial and temporal modules from empirical data and indicated that the activity of a neural population code stimuli using multiple representations. Our findings indicate that our method is scalable to large populations of neurons and has the capacity to efficiently identify biologically meaningful and informative low dimensional representations.

Natural languages that originate from a common ancestor are genetically related, words are the core of any language and cognates are words sharing the same ancestor and etymology. Cognate identification, therefore, represents the foundation upon which the evolutionary history of languages may be discovered, while linguistic phylogenetic inference aims to estimate the genetic relationships that exist between them. In this thesis, using several techniques originally developed for biological sequence analysis, we have designed a data driven orthographic learning system for measuring string similarity and we have successfully applied it to the tasks of cognate identification and phylogenetic inference. Our system has outperformed the best comparable phonetic and orthographic cognate identification models previously reported in the literature, with results statistically significant and remarkably stable, regardless of the variation of the training dataset dimension. When applied to phylogenetic inference of the Indo-European language family, whose higher structure does not yet have consensus, our method has estimated phylogenies which are compatible with the benchmark tree and has reproduced correctly all the established major language groups and subgroups present in the dataset.