Performance management of engineering education emerges from the need to assure proper training of future engineers in order to meet the constantly evolving expectations and challenges for the engineering profession. The process of accreditation ensures that engineering graduates are adequately prepared for their professional careers and responsibilities by ensuring that they possess an expected set of mandatory graduate attributes. Engineering programs are required by accreditation bodies to have systematic performance management of their programs that informs a continuous improvement process. Unfortunately, the vast diversity of engineering disciplines, varieties of information systems, and the large number of actors involved in the process makes this task challenging and complex. We performed a systematic literature review of jurisdictions around the world who are doing accreditation and examined how universities across Canada, US and other countries, have addressed tool support for performance management of engineering education. Our initial systematic approach for tool supported performance management evolved from this, and then we refined it through an iterative process of combined action research and design science research. We developed a prototype, Graduate Attribute Information Analysis (GAIA) in collaboration with the School of Electrical Engineering and Computer Science at the University of Ottawa, to support a systematic approach for accreditation of three engineering programs. This thesis contributes to research on the problem by developing a systematic approach, a tool that supports it, a set of related data transformations, and a tool-assessment checklist. Our systematic approach for tool-supported performance management addresses system architecture, a common continuous improvement process, a common set of key performance indicators, and identifies the performance management forms and reports needed to analyze graduate attribute data. The data transformation and analysis techniques we demonstrate ensure the accurate analysis of statistical and historical trends.

This document specifies a simple signalling mechanism for inter-domain QoS marking using a BGP Extended Community QoS Attribute. Class based packet forwarding for delay and loss critical services is currently performed in an individual AS internal manner. The new QoS marking attribute makes the QoS class setup within the IP prefix advertising AS known to all access and transit ASes. This enables individual (re-)marking and forwarding treatment adaptation to the original QoS class setup of the respective IP prefix. The attribute provides the means to signal QoS markings on different layers, which are linked together in QoS class sets. It provides inter-domain and cross-layer insight into the QoS class mapping of the source AS with minimal signalling traffic.

Diagnostic classification models used in conjunction with diagnostic assessments are to classify individual respondents into masters and nonmasters at the level of attributes. Previous researchers (Madison & Bradshaw, 2015) recommended items on the assessment should measure all patterns of attribute combinations to ensure classification accuracy, but in practice, certain attributes may not be measured by themselves. Moreover, the model estimation requires large sample size, but in reality, there could be unanswered items in the data. Therefore, the current study sought to provide suggestions on selecting between two alternative Q-matrix designs when an attribute cannot be measured in isolation and when using maximum likelihood estimation in the presence of missing responses. The factorial ANOVA results of this simulation study indicate that adding items measuring some attributes instead of all attributes is more optimal and that other missing data treatments should be sought if the percent of missing responses is greater than 5%.

It has been noticed that one of the main challenges for e-commerce sites is providing the users with an efficient search function. It has also been noticed that the search function is one of the things the user is valuing the most when evaluating an e-commerce. The information technology enables the possibility to consume almost anything one could wish for. The challenge is in how to find this specific item. It is hence of interest to examine how to improve the search tool and what effect the updated search tool results in. The objective of this research is twofold. The objective motivated by economic factors is to examine how a user’s ability to find relevant items is affected by being able to refine a search result by selecting relevant attribute values. The other objective has a more technical character and is to examine how the rule based method performs in terms of extracting attribute values for enable the creation of the filtering function. The examinations for this research is conducted at a Swedish online auction company that due to the structure of its e-catalogue provides a suitable setup for the examinations. Because of the examinations being conducted in the environment of the auction company’s system this limits the result to only being representative for data with the same characteristics as the auction company’s. For answering the questions stated in the objective two methods are applied. One for examining the economic part and one for examining the technical part. The economic question is answered after analysing the result of an A/B-test conducted at the auction company. For answering the technical examination an information extraction technique is evaluated. The result of the economical examination is that a significant increase in conversion rate can be concluded for the system version with filtering enabled. The result of the technical examination shows that the rule based method can be used for information extraction in some cases. However, the obtained accuracy will be affected by the characteristics of the data the information extraction is performed on.

There has been an increasing relevance of the cultural sector in the economic and social development of different countries. However, this sector continues without much input from multi-criteria decision-making (MDCM) techniques and sustainability analysis, which are widely used in other sectors. This paper proposes an MCDM model to assess the sustainability of a musical institution’s program. To define the parameters of the proposed model, qualitative interviews with relevant representatives of Catalan cultural institutions and highly recognized professionals in the sector were performed. The content of the 2015–2016 season of the ‘Palau de la Música Catalana’, a relevant Catalan musical institution located in Barcelona, was used as a case study to empirically test the method. The method allows the calculation of a season value index (SVI), which serves to make more sustainable decisions on musical season programs according to the established criteria. The sensitivity analysis carried out for different scenarios shows the robustness of the method. The research suggests that more complex decision settings, such as MCDM methods that are widely used in other sectors, can be easily applied to the sustainable management of any type of cultural institution. To the best of the authors’ knowledge, this method was never applied to a cultural institution and with real data. / Universitat Oberta de Catalunya

Online supervised learning from fast-evolving imbalanced data streams has applications in many areas. That is, the development of techniques that are able to handle highly skewed class distributions (or 'class imbalance') is an important area of research in domains such as manufacturing, the environment, and health. Solutions should be able to analyze large repositories in near real-time and provide accurate models to describe rare classes that may appear infrequently or in bursts while continuously accommodating new instances. Although numerous online learning methods have been proposed to handle binary class imbalance, solutions suitable for multi-class streams with varying degrees of imbalance in evolving streams have received limited attention. To address this knowledge gap, the first contribution of this thesis introduces the Online Learning from Imbalanced Multi-Class Streams through Dynamic Sampling (DynaQ) algorithm for learning in such multi-class imbalanced settings. Our approach utilizes a queue-based learning method that dynamically creates an instance queue for each class. The number of instances is balanced by maintaining a queue threshold and removing older samples during training. In addition, new and rare classes are dynamically added to the training process as they appear. Our experimental results confirm a noticeable improvement in minority-class detection and classification performance. A comparative evaluation shows that the DynaQ algorithm outperforms the state-of-the-art approaches. Our second contribution in this thesis focuses on fairness-aware learning from imbalanced streams. Our work is motivated by the observation that the decisions made by online learning algorithms may negatively impact individuals or communities. Indeed, the development of approaches to handle these concerns is an active area of research in the machine learning community. However, most existing methods process the data in offline settings and are not directly suitable for online learning from evolving data streams. Further, these techniques fail to take the effects of class imbalance, on fairness-aware supervised learning into account. In addition, recent fairness-aware online learning supervised learning approaches focus on one sensitive attribute only, which may lead to subgroup discrimination. In a fair classification, the equality of fairness metrics across multiple overlapping groups must be considered simultaneously. In our second contribution, we thus address the combined problem of fairness-aware online learning from imbalanced evolving streams, while considering multiple sensitive attributes. To this end, we introduce the Multi-Sensitive Queue-based Online Fair Learning (MQ-OFL) algorithm, an online fairness-aware approach, which maintains valid and fair models over evolving streams. MQ-OFL changes the training distribution in an online fashion based on both stream imbalance and discriminatory behavior of the model evaluated over the historical stream. We compare our MQ-OFL method with state-of-art studies on real-world datasets and present comparative insights on the performance. Our final contribution focuses on explainability and interpretability in fairness-aware online learning. This research is guided by the concerns raised due to the black-box nature of models, concealing internal logic from users. This lack of transparency poses practical and ethical challenges, particularly when these algorithms make decisions in finance, healthcare, and marketing domains. These systems may introduce biases and prejudices during the learning phase by utilizing complex machine learning algorithms and sensitive data. Consequently, decision models trained on such data may make unfair decisions and it is important to realize such issues before deploying the models. To address this issue, we introduce techniques for interpreting the outcomes of fairness-aware online learning. Through a case study predicting income based on features such as ethnicity, biological sex, age, and education level, we demonstrate how our fairness-aware learning process (MQ-OFL) maintains a balance between accuracy and discrimination trade-off using global and local surrogate models.

Services relying on internet of things (IoTs) are increasing day by day. IoT makes use of internet services like network connectivity and computing capability to transform everyday objects into smart things that can interact with users, and the environment to achieve a purpose they are designed for. IoT nodes are memory, and energy constrained devices that acquire information from the surrounding environment, those nodes cannot handle complex data processing and heavy security tasks alone, thus, in most cases a framework is required for processing, storing, and securing data. The framework can be cloud-based, a publish/subscribe broker, or edge computing based. As services relying on IoT are increasing enormously nowadays, data security and privacy are becoming concerns. Security concerns arise from the fact that most IoT data are stored unencrypted on untrusted third-party clouds, which results in many issues like data theft, data manipulation, and unauthorized disclosure. While some of the solutions provide frameworks that store data in encrypted forms, coarse-grained encryption provides less specific access policies to the users accessing data. A more secure control method applies fine-grained access control, and is known as attribute-based encryption (ABE). This research aims to enhance the privacy and the security of the data stored in an IoT middleware named network smart objects (NOS) and extend its functionality by proposing a new IoT security architecture using an efficient ABE scheme known as key-policy attribute-based encryption (KP-ABE) along with an efficient key revocation mechanism based on proxy re-encryption (PRE). Design science research (DSR) was used to facilitate the solution. To establish the knowledge base, a previous case study was reviewed to explicate the problem and the requirements to the artefact were elicited from research documents. The artefact was designed and then demonstrated in a practical experiment by means of Ubuntu operating system (OS). Finally, the artefact’s requirements were evaluated by applying a computer simulation on the Ubuntu OS. The result of the research is a model artefact of an IoT security architecture which is based on ABE. The model prescribes the components and the architectural structure of the IoT system. The IoT system consists of four entities: data producers, data consumers, NOS, and the TA. The model prescribes the new components needed to implement KP-ABE and PRE modules. First, data is transferred from data producers to NOS through secure hypertext transfer protocol (HTTPS), then the data is periodically processed and analyzed to obtain a uniform representation and add useful metadata regarding security, privacy, and data-quality. After that, the data is encrypted by KP-ABE using users’ attributes. PRE takes place when a decryption key is compromised, then the ciphertext is re-encrypted to prevent it’s disclosure. The evaluation results show that the proposed model improved the data retrieval time of the previous middleware by 32% and the re-encryption time by 87%. Finally, the author discusses the limitations of the proposed model and highlights directions for future research.

Data streams being transmitted over a network channel with capacity less than the data rate of the data streams is very common when using network channels such as dial-up, low bandwidth wireless links. Not only does this lower capacity creates delays but also causes sequential network problems such as packet losses, network congestion, errors in data packets giving rise to other problems and creating a cycle of problems hard to break out from. In this thesis, we present a new approach for shedding the less informative attribute data from a data stream with a fixed schema to maintain a data rate lesser than the network channels capacity. A scheme for shedding attributes, instead of tuples, becomes imperative in stream data where the data for one of the attributes remains relatively constant or changes less frequently compared to the data for the other attributes. In such a data stream management system, shedding a complete tuple would lead to shedding of some informative-attribute data along with the less informative-attribute data in the tuple, whereas shedding of the less informative-attribute data would cause only the less informative data to be dropped. In this thesis, we deal with two major problems in load shedding: the intra-stream load shedding and the inter-stream load shedding problems. The intra-stream load shedding problem deals with shedding of the less informative attributes when a single data stream with the data rate greater than the channel capacity has to be transmitted to the destination over the channel. The inter-stream load shedding problem refers to shedding of attributes among different streams when more than one stream has to be transferred to the destination over a channel with the channel capacity less than the combined data rate of all the streams to be transmitted. As a solution to the inter-stream or intra-stream load shedding problem, we apply our load shedding schema approach to determine a ranking amongst the attributes on a singe data stream or multiple data streams with the least informative attribute(s) being ranked the highest. The amount of data to be shed to maintain the data rate below the capacity is calculated dynamically, which means that the amount of data to be shed changes with any change in the channel capacity or any change in the data rate. Using these two pieces of information, a load shedding schema describing the attributes to be shed is generated. The load shedding schema is generated dynamically, which means that the load shedding schema is updated with any change in (i) the rankings of attributes that capture the rate of change on the values of each attribute, (ii) channel capacity, and (iii) data rate even after load shedding has been invoked. The load shedding schema is updated using our load shedding schema re-evaluation algorithm, which adapts to the data stream characteristics and follows the attribute data variation curve of the data stream. Since data dropped at the source may be of interest to the user at the destination, we also propose a recovery module which can be invoked to recover attribute data already shed. The recovery module maintains the minimal amount of information about data already shed for recovery purpose. Preliminary experimental results have shown that recovery accuracy ranges from 90% to 99%, which requires only 5% to 33% and 4.88% to 50% of the dropped data to be stored for weather reports and stock exchanges, respectively. Storing of recovery information imposes storage and processing burden on the source site, and our recovery method aims at satisfactory recovery accuracy while imposing minimal burden on the source site. Our load shedding approach, which achieves a high performance in reducing the data stream load, (i) handles wide range of data streams in different application domains (such as weather, stocks, and network performance, etc.), (ii) is dynamic in nature, which means that the load shedding scheme adjusts the amount of data to be shed and which attribute data to be shed according to the current load and network capacity, and (iii) provides a data recovery mechanism that is capable to recover any shedded attribute data with recovery accuracy up to 90% with very low burden on the source site and 99% with a higher burden on some stream data. To the best of our knowledge, the dynamic load shedding scheme we propose is the first one in the literature to shed attributes, instead of tuples, along with providing a recovery mechanism in a data stream management system. Our load shedding approach is unique since it is not a static load shedding schema, which is less appealing in an ever-changing (sensor) network environment, and is not based on queries, but works on the general characteristics of the data stream under consideration instead.

Feature selection is the process of automatically selecting important features from data. It is an essential part of machine learning, artificial intelligence, data mining, and modelling in general. There are many feature selection algorithms available and the appropriate choice can be difficult. The aim of this thesis was to compare feature selection algorithms in order to provide an experimental basis for which algorithm to choose. The first phase involved assessing which algorithms are most common in the scientific community, through a systematic literature study in the two largest reference databases: Scopus and Web of Science. The second phase involved constructing and implementing a benchmark pipeline to compare 31 algorithms’ performance on 50 data sets.The selected features were used to construct classification models and their predictive performances were compared, as well as the runtime of the selection process. The results show a small overall superiority of embedded type algorithms, especially types that involve Decision Trees. However, there is no algorithm that is significantly superior in every case. The pipeline and data from the experiments can be used by practitioners in determining which algorithms to apply to their respective problems. / Variabelselektion är en process där relevanta variabler automatiskt selekteras i data. Det är en essentiell del av maskininlärning, artificiell intelligens, datautvinning och modellering i allmänhet. Den stora mängden variabelselektionsalgoritmer kan göra det svårt att avgöra vilken algoritm som ska användas. Målet med detta examensarbete är att jämföra variabelselektionsalgoritmer för att ge en experimentell bas för valet av algoritm. I första fasen avgjordes vilka algoritmer som är mest förekommande i vetenskapen, via en systematisk litteraturstudie i de två största referensdatabaserna: Scopus och Web of Science. Den andra fasen bestod av att konstruera och implementera en experimentell mjukvara för att jämföra algoritmernas prestanda på 50 data set. De valda variablerna användes för att konstruera klassificeringsmodeller vars prediktiva prestanda, samt selektionsprocessens körningstid, jämfördes. Resultatet visar att inbäddade algoritmer i viss grad är överlägsna, framför allt typer som bygger på beslutsträd. Det finns dock ingen algoritm som är signifikant överlägsen i varje sammanhang. Programmet och datan från experimenten kan användas av utövare för att avgöra vilken algoritm som bör appliceras på deras respektive problem.

An important problem that appears often when analyzing data involves identifying irregular or abnormal data points called outliers. This problem broadly arises under two scenarios: when outliers are to be removed from the data before analysis, and when useful information or knowledge can be extracted by the outliers themselves. Outlier Detection in the context of the second scenario is a research field that has attracted significant attention in a broad range of useful applications. For example, in credit card transaction data, outliers might indicate potential fraud; in network traffic data, outliers might represent potential intrusion attempts. The basis of deciding if a data point is an outlier is often some measure or notion of dissimilarity between the data point under consideration and the rest. Traditional outlier detection methods assume numerical or ordinal data, and compute pair-wise distances between data points. However, the notion of distance or similarity for categorical data is more difficult to define. Moreover, the size of currently available data sets dictates the need for fast and scalable outlier detection methods, thus precluding distance computations. Additionally, these methods must be applicable to data which might be distributed among different locations. In this work, we propose novel strategies to efficiently deal with large distributed data containing mixed-type attributes. Specifically, we first propose a fast and scalable algorithm for categorical data (AVF), and its parallel version based on MapReduce (MR-AVF). We extend AVF and introduce a fast outlier detection algorithm for large distributed data with mixed-type attributes (ODMAD). Finally, we modify ODMAD in order to deal with very high-dimensional categorical data. Experiments with large real-world and synthetic data show that the proposed methods exhibit large performance gains and high scalability compared to the state-of-the-art, while achieving similar accuracy detection rates.