The goal of this thesis is to improve visualization dashboards in order to help decision making and Business Intelligence (BI) analysis, and make them accessible to a larger user audience. We identify current trends and use practices of visualization dashboards in the BI domain, and following a user-cantered design approach we provide enhancements and solutions. More specifically we address the following challenges: making dashboard construction and use accessible to visualization novices, enhancing dashboards with advanced annotation capabilities to help BI analysts in the analysis process, and add storytelling functionality to dashboards to help the communication between analysts and decision makers. Our user-centered approach consists of gathering user requirements (through background search, in-depth interviews or prototyping sessions with BI experts), iterative prototype development, and evaluation of our prototypes with representative users. First, our work reports gathered insights regarding novice user practices on Exploration Views (EV), a system that allows novice visualization users to easily build and customize BI information dashboards, but also provides functionality needed by experts. We evaluated EV with both expert and novice visualization users, and found differences in how experts and novice users interact with the interface. Based on those observations we provide new guidelines that augment previous work on designing for visualization novices, in the context of interactive visualization systems in the form of dashboards. BI analysts using dashboards (experts or novices) need support in record-keeping of their analysis, their data discoveries and gathered insights. This support is lacking in most dashboard visualization systems. We conducted in-depth interviews with BI experts that led to new annotation needs for multi-chart visualization systems (such as dashboards), on which we based the design of a dashboard prototype that supports data and context aware annotations, shared across visualizations and datasets. We focused particularly on novel annotation aspects, such as multi-target annotations, annotation transparency across charts, and annotations that can be shared among different data-sources and BI applications. We identified and provided solutions to challenges in using context aware annotations, notably issues arising when the annotation's "context" changes (e.g. annotated data are deleted or changed). Our prototype was evaluated with BI experts that were able to easily perform several tasks that they deemed important in their work. To increase the value of an analysis, recorded insights and knowledge need to be organized and communicated to others, and made available for future analysts to learn from. This communication step is very important, as often the analyst of data and the decision maker are two different people. To achieve this we need an easy insight organization and sharing mechanism to facilitate the transfer of knowledge, experiences, and stories, to decision makers and other analysts. We interviewed BI analysis experts and collected new requirements related to BI visual storytelling. Based on these requirements we designed and implemented a storytelling prototype that is integrated in a dashboard analysis tool, and allows easy transition from analysis to story creation and sharing. Our system was identified by experts as having great potential for training other analysts. Moreover, it can help BI analysis authors organize and communicate their findings faster to decision makers, and they can reach a broader audience since the results of a data analysis presented in the storytelling tool can be read with little to no training.

L’objectif de cette thèse est d’améliorer les tableaux de bord (dashboard) de visualisation afin d’aider à la prise de décision et d’analyse BI, et les rendre accessibles à un public plus large. Nous identifions les tendances actuelles et pratiques d’utilisation des dashboards de visualisation dans le domaine BI, et après une approche de conception centreé, nous proposons des améliorations et des solutions. Plus précisément, nous relevons les défis suivants: faire la construction du dashboard et d’utiliser accessible aux novices de visualisation, l’amélioration des dashboards avec des capacités d’annotation pour aider les analystes BI dans le processus d’analyse, et ajouter des fonctionnalités du storytelling pour aider à la communication entre les analystes et les décideurs. Notre approche centrée-utilisateur consiste à recueillir les besoins des utilisateurs (grâce à la recherche de fond, des entrevues en profondeur ou des séances de prototypage avec des experts BI), le développement itératif des prototypes, et l’évaluation des prototypes issus avec des utilisateurs représentatifs. Dans la première étape, notre travail rapporte idées recueillies au sujet des pratiques des utilisateurs novices sur Exploation Views (EV), un système qui permet aux utilisateurs novices de visualisation à créer facilement des dashboards et les personnaliser, mais fournit également des fonctionnalités nécessaires aux les experts. Nous avons évalué EV à la fois d’experts et des utilisateurs novices de visualisation, et constaté des différences dans la façon dont les experts et les utilisateurs novices interagissent avec l’interface. En se basant sur ces observations nous proposons de nouvelles lignes directrices, qui augmentent les travaux antérieurs, sur la conception pour les novices, dans le contexte des systèmes de visualisation interactifs sous forme de dashboard. Les analystes BI utilisant des dashboards (experts ou novices) ont besoin d’aide dans la tenue des dossiers de leur analyse, de leurs découvertes et des idées recueillies des données. Ce soutien fait défaut dans la plupart des systèmes de visualisation de dashboard. Nous avons effectué des entretiens approfondis avec des experts en analyse BI, qui ont révélé de nouveaux besoins d’annotation pour les systèmes de visualisation multi-graphiques (tels que des dashboard), sur lesquels nous avons basé la conception d’un prototype de tableau de bord qui gère les données et les annotations contextuels, qui peuvent être partagées à travers visualisations et des ensembles duel données. Nous nous sommes concentrés en particulier sur les aspects d’annotation innovantes, tels que les annotations multi-cibles, la transparence des annotations à travers des graphiques, ainsi que les annotations qui peuvent être partagées entre les différentes sources de données et des applications de BI. Nous avons identifié et fourni des solutions aux problèmes en utilisant les annotations contextuels, notamment les questions qui se posent lorsque le contexte de l’annotation change (par exemple les données annotées sont supprimés ou modifiés). Notre prototype a été évaluée avec des experts, utilisateurs BI, qui ont réussi d’effectuer facilement plusieurs tâches qu’ils estiment importants dans leur travail. / The goal of this thesis is to improve visualization dashboards in order to help decision making and Business Intelligence (BI) analysis, and make them accessible to a larger user audience. We identify current trends and use practices of visualization dashboards in the BI domain, and following a user-cantered design approach we provide enhancements and solutions. More specifically we address the following challenges: making dashboard construction and use accessible to visualization novices, enhancing dashboards with advanced annotation capabilities to help BI analysts in the analysis process, and add storytelling functionality to dashboards to help the communication between analysts and decision makers. Our user-centered approach consists of gathering user requirements (through background search, in-depth interviews or prototyping sessions with BI experts), iterative prototype development, and evaluation of our prototypes with representative users. First, our work reports gathered insights regarding novice user practices on Exploration Views (EV), a system that allows novice visualization users to easily build and customize BI information dashboards, but also provides functionality needed by experts. We evaluated EV with both expert and novice visualization users, and found differences in how experts and novice users interact with the interface. Based on those observations we provide new guidelines that augment previous work on designing for visualization novices, in the context of interactive visualization systems in the form of dashboards. BI analysts using dashboards (experts or novices) need support in record-keeping of their analysis, their data discoveries and gathered insights. This support is lacking in most dashboard visualization systems. We conducted in-depth interviews with BI experts that led to new annotation needs for multi-chart visualization systems (such as dashboards), on which we based the design of a dashboard prototype that supports data and context aware annotations, shared across visualizations and datasets. We focused particularly on novel annotation aspects, such as multi-target annotations, annotation transparency across charts, and annotations that can be shared among different data-sources and BI applications. We identified and provided solutions to challenges in using context aware annotations, notably issues arising when the annotation’s ”context” changes (e.g. annotated data are deleted or changed). Our prototype was evaluated with BI experts that were able to easily perform several tasks that they deemed important in their work. To increase the value of an analysis, recorded insights and knowledge need to be organized and communicated to others, and made available for future analysts to learn from. This communication step is very important, as often the analyst of data and the decision maker are two different people. To achieve this we need an easy insight organization and sharing mechanism to facilitate the transfer of knowledge, experiences, and stories, to decision makers and other analysts. We interviewed BI analysis experts and collected new requirements related to BI visual storytelling. Based on these requirements we designed and implemented a storytelling prototype that is integrated in a dashboard analysis tool, and allows easy transition from analysis to story creation and sharing. Our system was identified by experts as having great potential for training other analysts. Moreover, it can help BI analysis authors organize and communicate their findings faster to decision makers, and they can reach a broader audience since the results of a data analysis presented in the storytelling tool can be read with little to no training.

With the increasing pace of development, the process of interpreting software test results data has become more challenging and time-consuming. While the test results provide valuable insights into the software product, the increasing complexity of software systems and the growing volume of test data pose challenges in effectively analyzing this data to ensure quality. To address these challenges, organizations are adopting various tools. Visualization dashboards are a common approach used to streamline the analysis process. By aggregating and visualizing test results data, these dashboards enable easier identification of patterns and trends, facilitating informed decision-making. This study proposes a management dashboard with visualizations of test results data as a decision support system. A case study was conducted involving eleven quality assurance experts with a number of various roles, including managers, directors, testers, and project managers. User interviews were conducted to evaluate the need for a dashboard and identify relevant test results data to visualize. The participants expressed the need for a dashboard, which would benefit both newcomers and experienced employees. A low-fidelity prototype of the dashboard was created and A/B testing was performed through a survey to prioritize features and choose the preferred version of the prototype. The results of the user interviews highlighted pass-rate, executed test cases, and failed test cases as the most important features. However, different professions showed interest in different test result metrics, leading to the creation of multiple views in the prototype to accommodate varying needs. A high-fidelity prototype was implemented based on feedback and underwent user testing, leading to iterative improvements. Despite the numerous advantages of a dashboard, integrating it into an organization can pose challenges due to variations in testing processes and guidelines across companies and teams. Hence, the dashboards require customization. The main contribution of this study is twofold. Firstly, it provides recommendations for relevant test result metrics and suitable visualizations to effectively communicate test results. Secondly, it offers insights into the visualization preferences of different professions within a quality assurance team that were missing in previous studies. / Med den ökande utvecklingstakten har processen att tolka testresultatdata för programvara blivit mer utmanande och tidskrävande. Även om testresultaten ger värdefulla insikter i mjukvaruprodukten, innebär den ökande komplexiteten hos mjukvarusystemen och den växande volymen testdata utmaningar när det gäller att effektivt analysera dessa data för att säkerställa kvalitet. För att möta dessa utmaningar använder organisationer olika verktyg. Visualiseringspaneler är ett vanligt tillvägagångssätt som används för att effektivisera analysprocessen. Genom att aggregera och visualisera testresultatdata möjliggör dessa instrumentpaneler enklare identifiering av mönster och trender, vilket underlättar välgrundat beslutsfattande. Den här studien föreslår en management-panel med visualiseringar av testresultatdata som ett beslutsstödssystem. En fallstudie genomfördes med elva experter inom kvalitetssäkring med olika roller, inklusive chefer, direktörer, testare och projektledare. Användarintervjuer genomfördes för att utvärdera behovet av en panel och identifiera relevanta testresultatdata att visualisera. Deltagarna uttryckte behovet av en visualiseringspanel, som skulle gynna både nyanställda och erfarna medarbetare. En prototyp av panelen med låg detaljnivå skapades och A/B-testning genomfördes genom en enkät för att prioritera funktioner och välja den föredragna versionen av prototypen. Resultaten av användarintervjuerna lyfte fram andel av godkända testresultat, exekverade testfall och misslyckade testfall som de viktigaste egenskaperna. Men olika yrkesgrupper visade intresse för olika testresultatmått, vilket ledde till skapandet av flera vyer i prototypen för att tillgodose olika behov. En prototyp med hög detaljnivå implementerades baserat på feedback och genomgick användartestning, vilket ledde till iterativa förbättringar. Trots de många fördelarna med en instrumentpanel kan det innebära utmaningar att integrera den i en organisation på grund av variationer i testprocesser och riktlinjer mellan företag och team. Därför kräver paneler anpassning. Det huvudsakliga bidraget från denna studie är dubbelt. För det första ger den rekommendationer för relevanta testresultatmått och lämpliga visualiseringar för att effektivt kommunicera testresultat. För det andra ger den insikter i visualiseringspreferenser för olika yrken inom ett kvalitetssäkringsteam vilket saknats i tidigare studier.