Modern computer software systems are complicated. Developers can change the behavior of the software system through software configurations. The large number of configuration option and their interactions make the task of software tuning, testing, and debugging very challenging. Performance is one of the key aspects of non-functional qualities, where performance bugs can cause significant performance degradation and lead to poor user experience. However, performance bugs are difficult to expose, primarily because detecting them requires specific inputs, as well as specific configurations. While researchers have developed techniques to analyze, quantify, detect, and fix performance bugs, many of these techniques are not effective in highly-configurable systems. To improve the non-functional qualities of configurable software systems, testing engineers need to be able to understand the performance influence of configuration options, adjust the performance of a system under different configurations, and detect configuration-related performance bugs. This research will provide an automated framework that allows engineers to effectively analyze performance-influence configuration options, detect performance bugs in highly-configurable software systems, and adjust configuration options to achieve higher long-term performance gains. To understand real-world performance bugs in highly-configurable software systems, we first perform a performance bug characteristics study from three large-scale opensource projects. Many researchers have studied the characteristics of performance bugs from the bug report but few have reported what the experience is when trying to replicate confirmed performance bugs from the perspective of non-domain experts such as researchers. This study is meant to report the challenges and potential workaround to replicate confirmed performance bugs. We also want to share a performance benchmark to provide real-world performance bugs to evaluate future performance testing techniques. Inspired by our performance bug study, we propose a performance profiling approach that can help developers to understand how configuration options and their interactions can influence the performance of a system. The approach uses a combination of dynamic analysis and machine learning techniques, together with configuration sampling techniques, to profile the program execution, analyze configuration options relevant to performance. Next, the framework leverages natural language processing and information retrieval techniques to automatically generate test inputs and configurations to expose performance bugs. Finally, the framework combines reinforcement learning and dynamic state reduction techniques to guide subject application towards achieving higher long-term performance gains.

Laws and regulation affect software development, as they frequently demand changes in software’ requirements to protect individuals and businesses regarding security, privacy, governance, sustainability and more. Legal requirements can dictate new requirements or constrain existing ones. The problem of software compliance is howto ensure that the software complies with the norms that the legislation imposes. The problem is particularly challenging because it combines difficultsteps: 1)analyze legal documents, 2) extract requirements from those documents, 3) identify conflicting requirements with those already implemented in software and 4) ensure that software remains compliant even with the changes. Compliance is a continuous process: laws, software and the context within which software system operates changes continuously. The works dealing with the compliance problem focus only on one or two subjects: analyze legal documents or extract requirements or identify conflicts or changes. This thesis deals with all the problems at the same time; the idea is to extract requirements from legal text, compare them with the software requirement, resolve the possible conflicts that may arise, continuously leading with the changes on environment, laws and requirements. For this, this work proposes a framework that is composed of a compliance process and continuous monitoring of environmental changes. The framework deals with different types of laws (security, privacy, transparency, health care) that are represented in explicit norms. The compliance process supports the identification, extraction, comparison and conflict resolution to help software compliance, by producing a compliant set of requirements. The compliance process is based on the semantic annotation and goal model. The semantic annotation helps to extract requirements from thelaw, using patterns. The goal model is used to help the comparison between requirement and to represent requirements in a formal and consistent requirement specification. The process is tool supported; some tools were reused (Desiree and NomosT) to further each step. It was necessary to adapt the tools for the context of the compliance process, creating a guideline, patterns, and heuristics. The continuous monitoring is concerned about the changes that affect the software compliance and has 7 the mechanism to ensure that even with those changes the software will regain compliance. The compliance monitor is basedon agents and Non Functional Requirements. The agents are represented using in i*, the idea is to showthe collaboration between the agents to ensure the continuous compliance. The requirement specification of how each agent should behave was also generated using Business Process Modeling Notation and Desiree language. The Non Functional Requirements catalogue is used to help to define operalizations for the software awareness. The framework validation was made in two parts: first, the compliance process and after all the framework proposed. For the compliance process, the effort and correctness were measured comparing the use of the proposed process andan ad-hoc method. For the entire framework, the example of monitoring the changes in the environment when an automated car is crossing the border between Washington and Canada was used. The study shows that context has a strong influence on the software requirements, and nonconformity problems may incur penalties. The contribution of this work is the Eunomia framework that has a process and goal model perspective with emphasis on monitoring that helps to deal with the compliance challenge. The framework equips the requirements engineering team with a systematic method. Eunomia framework is a tool-supported and systematic process which can be reused to reduce the time effort and to improve the quality of the requirement specification that helps to create a compliant software requirement specification that is compliant over the time.

This study explored the feasibility and practicability of designing and developing a Knowledge Assessment System (KAS) for assessing different types of knowledge as defined in the revision of Bloom's Taxonomy (Anderson et al., 2001). The KAS created as a result of this study is based on concept maps and employs two differential weighting approaches. It is a developmental study, which includes the design phase, development phase and evaluation phase. The general software system design model (Sommerville, 2009) was adopted to guide the design of the Knowledge Assessment System based on its procedures, including system requirements analysis, architecture design, component design, interface design, and database design. The assessment criteria in this system are designed to be proposition-based and consist of either a non-weighting approach or a weighting approach, which can help provide instructors with flexible assessing methods as well as help them obtain a whole picture of what kinds of knowledge their students have grasped and to what extent the students have mastered that knowledge, based on the student-created concept maps. The two differential weighting approaches initially compare student-created concept maps with expert maps stored in the system. Because some correct propositions in student concept maps may be not included in the initial expert concept maps, the system is designed to continually refine the assessment criterion by inspecting and evaluating the correctness of the propositions in the student-created concept maps and adding the results to the system's database. The current system is able to assess three types of knowledge: factual, conceptual, and procedural, all of which are defined in the revision of Bloom's Taxonomy (Anderson et al., 2001). The assessment process begins with the assignment of different types of concept map tasks entered and stored in the Knowledge Assessment System. Following by student completion of the tasks and submission of a concept map, the submitted concept map is compared to the criteria stored in the system and a performance report is generated. The research results show that the Knowledge Assessment System based on concept maps and two differential weighting approaches can act as a useful tool for assessing students' factual, conceptual and procedural knowledge based on their concept maps. / Ph. D.

Thesis (MScEng (Electrical and Electronic Engineering))--University of Stellenbosch, 2010. / ENGLISH ABSTRACT: Stellenbosch University and the Katholieke Universiteit Leuven has a joint undertaking to develop a satellite communications payload. The goals of the project are: to undertake research and expand knowledge in the area of dynamically configurable antenna beam forming, to prove the viability of this research for space purposes and to demonstrate the feasibility of the development in a practical application. The practical application is low Earth orbit satellite communication system for applications in remote monitoring. Sensor data will be uploaded to the satellite, stored and forwarded to a central processing ground station as the satellite passes over these ground stations. The system will utilise many low-cost ground sensor stations to collect data and distribute it to high-end ground stations for processing. Applications of remote monitoring systems are maritime- and climate change monitoring- and tracking. Climate change monitoring allows inter alia, for the monitoring of the effects and causes of global warming. The Katholieke Universiteit Leuven is developing a steerable antenna to be mounted on the satellite. Stellenbosch University is developing the communications payload to steer and use the antenna. The development of the communications protocol stack is part of the project. The focus of this work is to implement the application layer protocol, which handles all file level communications and also implements the communications strategy. The application layer protocol is called the Satellite Communications Software System (SCSS). It handles all high level requests from ground stations, including requests to store data, download data, download log files and upload configuration information. The design is based on a client-server model, with a Station Server and Station Handler. The Station Server schedules ground stations for communication and creates a Station Handler for each ground station to handle all ground station requests. During the design, all file formats were defined for efficient ground station-satellite communications and system administration. All valid ground station requests and handler responses were also defined. It was also found that the system may be made more efficient by scheduling ground stations for communications, rather than polling each ground station until one responds. To be able to schedule ground station communications, the times when ground stations will come into view of the satellite have to be predicted. This is done by calculating the positions of the Satellite and ground stations as functions of time. A simple orbit propagator was developed to predict the satellite distance and to ease testing and integration with the communications system. The times when a ground station will be within range of the satellite were then predicted and a scheduling algorithm developed to minimise the number of ground stations not able to communicate. All systems were implemented and tested. The SCSS executing on the Satellite was developed and tested on the satellite on-board computer. Embedded implementations possess strict resource limitations, which were taken into account during the development process. The SCSS is a multi-threaded system that makes use of thread cancellation to improve responsiveness. / AFRIKAANSE OPSOMMING: Die Universiteit van Stellenbosch ontwerp tans ’n satelliet kommunikasieloonvrag in samewerking met die Katolieke Universiteit van Leuven. Die doel van die projek is om navorsing te doen oor die lewensvatbaarheid van dinamies verstelbare antenna bundelvorming vir ruimte toepassings, asook om die haalbaarheid van hierdie navorsing in die praktyk te demonstreer. Die praktiese toepassing is ’n satellietkommunikasiestelsel vir afstandsmonitering, wat in ’n Lae-Aarde wentelbaan verkeer. Soos die satelliet in sy wentelbaan beweeg, sal sensor data na die satelliet toe gestuur, gestoor en weer aangestuur word. Die stelsel gebruik goedkoop sensorgrondstasies om data te versamel en aan te stuur na kragtiger grondstasies vir verwerking. Afstandsmoniteringstelsels kan gebruik word om klimaatsverandering, sowel as die posisie van skepe en voertuie, te monitor. Deur oa. klimaatsveranderinge te dokumenteer, kan gevolge en oorsake van globale verhitting gemonitor word. Die Katholieke Universiteit van Leuven is verantwoordelik vir die ontwerp en vervaardiging van die satelliet antenna, terwyl die Universiteit van Stellenbosch verantwoordelik is vir die ontwerp en bou van die kommunikasie loonvrag. ’n Gedeelte van hierdie ontwikkeling sluit die ontwerp en implementasie van al die protokolle van die kommunikasieprotokolstapel in. Dit fokus op die toepassingsvlak protokol van die protokolstapel, wat alle leêrvlak kommunikasie hanteer en die kommunikasiestrategie implementeer. Die toepassingsvlaksagteware word die Satellietkommunikasie sagtewarestelsel (SKSS) genoem. Die SKSS is daarvoor verantwoordelik om alle navrae vanaf grondstasies te hanteer. Hierdie navrae sluit die oplaai en stoor van data, die aflaai van data, die aflaai van logs en die oplaai van konfigurasie inligting in. Die ontwerp is op die standaard kliënt-bediener model gebasseer, met ’n stasiebediener en ’n stasiehanteerder. Die stasiebediener skeduleer die tye wanneer grondstasies toegelaat sal word om te kommunikeer en skep stasiehanteerders om alle navrae vanaf die stasies te hanteer. Gedurende die ontwerp is alle leêrformate gedefinieer om doeltreffende adminstrasie van die stelsel, asook kommunikasie tussen grondstasies en die satelliet te ondersteun. Alle geldige boodskappe tussen die satelliet en grondstasies is ook gedefnieer. Daar is gevind dat die doeltreffendheid van die stelsel verhoog kan word deur die grondstasies wat wil kommunikeer te skeduleer, eerder as om alle stasies te pols totdat een reageer. Om so ’n skedule op te stel, moet die tye wanneer grondstasies binne bereik van die satelliet gaan wees voorspel word. Hierdie voorspelling is gedoen deur die posisies van die satelliet en die grondstasies as funksies van tyd te voorspel. ’n Eenvoudige satelliet posisievoorspeller is ontwikkel om toetsing en integrasie met die SKSS te vergemaklik. ’n Skeduleringsalgoritme is toe ontwikkel om die hoeveelheid grondstasies wat nie toegelaat word om te kommunikeer nie, te minimeer. Alle stelsels is geimplementeer en getoets. Die SKSS, wat op die satelliet loop, is ontwikkel en getoets op die satelliet se aanboord rekenaar. Die feit dat ingebedde stelsels oor baie min hulpbronne beskik, is in aanmerking geneem gedurende die ontwikkeling en implementasie van die SKSS. Angesien die SKSS ’n multidraadverwerkingsstelsel is, word daar van draadkansellasie gebruik gemaak om die stelsel se reaksietyd te verbeter.

A indústria automotiva vem mudando seu perfil drasticamente na última década, tornando-se mais intensiva em sistemas computacionais, e consequentemente em software, do que em sistemas mecânicos. A maioria das funcionalidades dos veículos modernos é controlada por software distribuído, dentro de muitas Unidades de Controle Eletrônicas (ECUs) interconectadas, de modo que o veículo pode ser considerado como um conjunto de sistemas complexos. Como resultado da integração, os testes para as funções de software distribuído se tornaram um grande desafio. Muitas estratégias têm sido propostas para organizar o multinível de teste de software em sistemas embarcados automotivos, a fim de reduzir custos e melhorar sua eficácia e robustez. Esta investigação pretende estender este conceito de teste de software multinível baseado no modelo em V, projetando uma metodologia sistemática que mapeie cada função dentro do sistema embarcado do veículo, sem a existência de uma especificação detalhada de cada componente e possibilitando a definição de uma estratégia e um planejamento de teste de software antecipadamente. Esta metodologia foi totalmente desenvolvida incorporando as características da aplicação automotiva, incluindo os requisitos de segurança funcional ISO 26262. Desta forma e dentro de rigorosas condições de segurança, a nova metodologia melhora a eficiência dos processos de desenvolvimento de sistemas embarcados, possibilitando a antecipação do planejamento de teste de software. Sendo este, baseado em um mapeamento e codificação da função e das atividades de teste proposto nesta tese, que funciona também como base para futuros desenvolvimento de soluções de rastreabilidade do sistema embarcado. / The automotive industry has been changing its profile drastically in the last decade, becoming more intensive in computer systems, and consequently in software, than in mechanical systems. Most of the functionality of modern vehicles is controlled by distributed software, within many interconnected Electronic Control Units (ECUs), so that the vehicle can be considered as a set of complex systems. As a result of integration, testing for distributed software functions has become a major challenge. Many strategies have been proposed to organize the multilevel software testing in embedded automotive systems in order to reduce costs and improve their effectiveness and robustness. This research intends to extend this concept of multilevel software testing based on the V development model, designing a systematic methodology that maps each function within the embedded system of the vehicle, without the existence of a detailed specification of each component, allowing the definition of a strategy and software testing planning in advance. This methodology was fully developed incorporating the characteristics of the automotive application, including the ISO 26262 functional safety requirements. In this way, the new methodology improves the efficiency of the embedded systems development processes, within strict security conditions, allowing the anticipation of the planning based on a mapping and encoding of the function and test activities proposed in this thesis, which also serves as a basis for future development of embedded system traceability solutions.

A indústria automotiva vem mudando seu perfil drasticamente na última década, tornando-se mais intensiva em sistemas computacionais, e consequentemente em software, do que em sistemas mecânicos. A maioria das funcionalidades dos veículos modernos é controlada por software distribuído, dentro de muitas Unidades de Controle Eletrônicas (ECUs) interconectadas, de modo que o veículo pode ser considerado como um conjunto de sistemas complexos. Como resultado da integração, os testes para as funções de software distribuído se tornaram um grande desafio. Muitas estratégias têm sido propostas para organizar o multinível de teste de software em sistemas embarcados automotivos, a fim de reduzir custos e melhorar sua eficácia e robustez. Esta investigação pretende estender este conceito de teste de software multinível baseado no modelo em V, projetando uma metodologia sistemática que mapeie cada função dentro do sistema embarcado do veículo, sem a existência de uma especificação detalhada de cada componente e possibilitando a definição de uma estratégia e um planejamento de teste de software antecipadamente. Esta metodologia foi totalmente desenvolvida incorporando as características da aplicação automotiva, incluindo os requisitos de segurança funcional ISO 26262. Desta forma e dentro de rigorosas condições de segurança, a nova metodologia melhora a eficiência dos processos de desenvolvimento de sistemas embarcados, possibilitando a antecipação do planejamento de teste de software. Sendo este, baseado em um mapeamento e codificação da função e das atividades de teste proposto nesta tese, que funciona também como base para futuros desenvolvimento de soluções de rastreabilidade do sistema embarcado. / The automotive industry has been changing its profile drastically in the last decade, becoming more intensive in computer systems, and consequently in software, than in mechanical systems. Most of the functionality of modern vehicles is controlled by distributed software, within many interconnected Electronic Control Units (ECUs), so that the vehicle can be considered as a set of complex systems. As a result of integration, testing for distributed software functions has become a major challenge. Many strategies have been proposed to organize the multilevel software testing in embedded automotive systems in order to reduce costs and improve their effectiveness and robustness. This research intends to extend this concept of multilevel software testing based on the V development model, designing a systematic methodology that maps each function within the embedded system of the vehicle, without the existence of a detailed specification of each component, allowing the definition of a strategy and software testing planning in advance. This methodology was fully developed incorporating the characteristics of the automotive application, including the ISO 26262 functional safety requirements. In this way, the new methodology improves the efficiency of the embedded systems development processes, within strict security conditions, allowing the anticipation of the planning based on a mapping and encoding of the function and test activities proposed in this thesis, which also serves as a basis for future development of embedded system traceability solutions.

Contemporary enterprises depend to great extent on software systems. During the past decades the number of systems has been constantly increasing and these systems have become more integrated with one another. This has lead to a growing complexity in managing software systems and their environment. At the same time business environments today need to progress and change rapidly to keep up with evolving markets. As the business processes change, the systems need to be modified in order to continue supporting the processes. The complexity increase and growing demand for rapid change makes the management of enterprise systems a very important issue. In order to achieve effective and efficient management, it is essential to be able to analyze the system modifiability (i.e. estimate the future change cost). This is addressed in the thesis by employing architectural models. The contribution of this thesis is a method for software system modifiability analysis using enterprise architecture models. The contribution includes an enterprise architecture analysis formalism, a modifiability metamodel (i.e. a modeling language), and a method for creating metamodels. The proposed approach allows IT-decision makers to model and analyze change projects. By doing so, high-quality decision support regarding change project costs is received. This thesis is a composite thesis consisting of five papers and an introduction. Paper A evaluatesa number of analysis formalisms and proposes extended influence diagrams to be employed for enterprise architecture analysis. Paper B presents the first version of the modifiability metamodel. InPaper C, a method for creating enterprise architecture metamodels is proposed. This method aims to be general, i.e. can be employed for other IT-related quality analyses such as interoperability, security, and availability. The paper does however use modifiability as a running case. The second version of the modifiability metamodel for change project cost estimation is fully described in Paper D. Finally, Paper E validates the proposed method and metamodel by surveying 110 experts and studying 21 change projects at four large Nordic companies. The validation indicates that the method and metamodel are useful, contain the right set of elements and provide good estimation capabilities. / QC20100716

Softwaresysteme sind ein wichtiger Bestandteil in der heutigen digitalen Welt und müssen oft als fundamentales Investitionsgut von Unternehmen betrachtet werden. Deswegen benötigen solche Systeme eine betriebswirtschaftliche Sichtweise: der Wert als operatives System sowie als langfristige finanzielle Investition bezüglich zukünftiger Anforderungen. Die Grundlage jeden guten Softwaresystems liegt in dessen Architektur, dem Zusammenhang von allen Teilen und deren Beziehungen untereinander, dem inneren Aufbau und der Organisation der Teile. Der Zusammenhang zwischen guter Architektur und resultierender guter Evolutionsfähigkeit und Zukunftsfähigkeit eines Systems ist generell in den Meinungen vertreten. Jedoch liegen dazu nur wenige Literaturstellen vor, welche die Thematik tatsächlich mehr als nur empirisch aufzeigen. Diese Arbeit soll einen möglichen Weg aufzeigen, wie mithilfe eines modellbasierten Ansatzes ein kausaler und quantitiver Zusammenhang zwischen Architektur eines Softwaresystems und dem resultierenden betriebswirtschaftlichen Wert (im Sinne der Evolutionsfähigkeit) hergestellt werden kann.

Softwaresysteme sind ein wichtiger Bestandteil in der heutigen digitalen Welt und müssen oft als fundamentales Investitionsgut von Unternehmen betrachtet werden. Deswegen benötigen solche Systeme eine betriebswirtschaftliche Sichtweise: der Wert als operatives System sowie als langfristige finanzielle Investition bezüglich zukünftiger Anforderungen. Die Grundlage jeden guten Softwaresystems liegt in dessen Architektur, dem Zusammenhang von allen Teilen und deren Beziehungen untereinander, dem inneren Aufbau und der Organisation der Teile. Der Zusammenhang zwischen guter Architektur und resultierender guter Evolutionsfähigkeit und Zukunftsfähigkeit eines Systems ist generell in den Meinungen vertreten. Jedoch liegen dazu nur wenige Literaturstellen vor, welche die Thematik tatsächlich mehr als nur empirisch aufzeigen. Diese Arbeit soll einen möglichen Weg aufzeigen, wie mithilfe eines modellbasierten Ansatzes ein kausaler und quantitiver Zusammenhang zwischen Architektur eines Softwaresystems und dem resultierenden betriebswirtschaftlichen Wert (im Sinne der Evolutionsfähigkeit) hergestellt werden kann.

Large software systems are in need of a construction plan to determine and define every concept and element used in order to not end up in complex, unusable, and cost-intensive systems. Different modeling languages, like UML, support the development of these construction plans and visualize them for the system’s stakeholders. Reference models are a specific kind of construction plan, used as templates for information systems and already capture business domain knowledge for reuse and tailoring. By adaptation, reference models are tailored to enterprise-specific application models, which can be used for software construction and maintenance. However, current adaptation methods suffer from the limitations of pure object-oriented development (e.g., identity issues, large inheritance trees, and inflexibility). In this thesis, the usage of roles as the sole adaptation mechanism is proposed to solve these challenges. With the help of conceptual roles, it is possible to create rich model variations and adaptations from existing (industry standard) reference models, and it is simpler to react to model evolution and changing business logic. Adaptations can be specified with more precision by maintaining or even increasing the model’s expressiveness. As a consequence, the role-enriched final application model can be used to describe software systems in more detail, with different perspectives, and, if available, can be implemented with a role supporting programming language. However, even without this step, the application model itself will provide valuable insights into the overall construction plan of a software system by the combination of structure and behavior and a clear separation of relatively stable domain knowledge from its use case specific adaptation.