Learning from text data often involves a loop of tasks that iterate between foraging for information and synthesizing it in incremental hypotheses. Past research has shown the advantages of using spatial workspaces as a means for synthesizing information through externalizing hypotheses and creating spatial schemas. However, spatializing the entirety of datasets becomes prohibitive as the number of documents available to the analysts grows, particularly when only a small subset are relevant to the tasks at hand. To address this issue, we developed the multi-model semantic interaction (MSI) technique, which leverages user interactions to aid in the display layout (as was seen in previous semantic interaction work), forage for new, relevant documents as implied by the interactions, and then place them in context of the user's existing spatial layout. This results in the ability for the user to conduct both implicit queries and traditional explicit searches. A comparative user study of StarSPIRE discovered that while adding implicit querying did not impact the quality of the foraging, it enabled users to 1) synthesize more information than users with only explicit querying, 2) externalize more hypotheses, 3) complete more synthesis-related semantic interactions. Also, 18% of relevant documents were found by implicitly generated queries when given the option. StarSPIRE has also been integrated with web-based search engines, allowing users to work across vastly different levels of data scale to complete exploratory data analysis tasks (e.g. literature review, investigative journalism). The core contribution of this work is multi-model semantic interaction (MSI) for usable big data analytics. This work has expanded the understanding of how user interactions can be interpreted and mapped to underlying models to steer multiple algorithms simultaneously and at varying levels of data scale. This is represented in an extendable multi-model semantic interaction pipeline. The lessons learned from this dissertation work can be applied to other visual analytics systems, promoting direct manipulation of the data in context of the visualization rather than tweaking algorithmic parameters and creating usable and intuitive interfaces for big data analytics. / Ph. D.

The current proliferation of connected displays and mobile devices from smart phones and tablets to wall-sized displays presents a number of exciting opportunities for information visualization and visual analytics. When a user employs heterogeneous displays collaboratively to achieve a goal, they form what is known as a display ecology. The display ecology enables multiple displays to function in concert within a broader technological environment to accomplish tasks and goals. However, since information and tasks are scattered and disconnected among separate displays, one of the inherent challenges associated with visual analysis in display ecologies is enabling users to seamlessly coordinate and subsequently connect and integrate information across displays. This research primarily addresses these challenges through the creation of interaction and visualization techniques and systems for display ecologies in order to support sensemaking with visual analysis. This dissertation explores essential visual analysis activities and design considerations for visual analysis in order to inform the new design of display ecologies for visual analysis. Based on identified design considerations, we then designed and developed two visual analysis systems. First, VisPorter supports intuitive gesture interactions for sharing and integrating information in a display ecology. Second, the Spatially Aware Visual Links (SAViL) presents a cross-display visual link technique capable of guiding the user's attention to relevant information across displays. It also enables the user to visually connect related information over displays in order to facilitate synthesizing information scattered over separate displays and devices. The various aspects associated with the techniques described herein help users to transform and empower the multiple displays in a display ecology for enhanced visual analysis and sensemaking. / Ph. D.

When exploring large, high-dimensional datasets, analysts often utilize two techniques for reducing the data to make exploration more tractable. The first technique, dimension reduction, reduces the high-dimensional dataset into a low-dimensional space while preserving high-dimensional structures. The second, clustering, groups similar observations while simultaneously separating dissimilar observations. Existing work presents a number of systems and approaches that utilize these techniques; however, these techniques can cooperate or conflict in unexpected ways. The core contribution of this work is the systematic examination of the design space at the intersection of dimension reduction and clustering when building intelligent, interactive tools in visual analytics. I survey existing techniques for dimension reduction and clustering algorithms in visual analytics tools, and I explore the design space for creating projections and interactions that include dimension reduction and clustering algorithms in the same visual interface. Further, I implement and evaluate three prototype tools that implement specific points within this design space. Finally, I run a cognitive study to understand how analysts perform dimension reduction (spatialization) and clustering (grouping) operations. Contributions of this work include surveys of existing techniques, three interactive tools and usage cases demonstrating their utility, design decisions for implementing future tools, and a presentation of complex human organizational behaviors. / Doctor of Philosophy / When an analyst is exploring a dataset, they seek to gain insight from the data. With data sets growing larger, analysts require techniques to help them reduce the size of the data while still maintaining its meaning. Two commonly-utilized techniques are dimension reduction and clustering. Dimension reduction seeks to eliminate unnecessary features from the data, reducing the number of columns to a smaller number. Clustering seeks to group similar objects together, reducing the number of rows to a smaller number. The contribution of this work is to explore how dimension reduction and clustering are currently being used in interactive visual analytics systems, as well as to explore how they could be used to address challenges faced by analysts in the future. To do so, I survey existing techniques and explore the design space for creating visualizations that incorporate both types of computations. I look at methods by which an analyst could interact with those projections in other to communicate their interests to the system, thereby producing visualizations that better match the needs of the analyst. I develop and evaluate three tools that incorporate both dimension reduction and clustering in separate computational pipelines. Finally, I conduct a cognitive study to better understand how users think about these operations, in order to create guidelines for better systems in the future.

With the dramatic increase and continued growth of digital information, developing Visual Analytic systems that support human cognition and insight generation are more necessary than ever before, but there is currently no content-agnostic method for measuring or com- paring how well a system facilitates analysis. Researchers in industry and academia are developing advanced tools that offer automated data analysis combined with support for human sense-making; tools for a wide variety of sense-making tasks are freely available. Now, the pressing question is: which tool works best, and for what? We show that using Shannon's entropy and self-information measures will provide a measure of the complexity reduction that results from an analyst's actions while sorting the information. Further, we demonstrate that reduced complexity can be linked to the knowledge gained. This is important, because a metric for objectively evaluating the success of current systems in generating insights would establish a standard that future tools could build on. This work could help guide researchers and developers in making the next generation of analytic tools, and in the age of big data the effect of such tools could potentially impact everyone. / Master of Science

Narratives are fundamental to our understanding of the world, and they are pervasive in all activities that involve representing events in time. Narrative analysis has a series of applications in computational journalism, intelligence analysis, and misinformation modeling. In particular, narratives are a key element of the sensemaking process of analysts. In this work, we propose a narrative model and visualization method to aid analysts with this process. In particular, we propose the narrative maps framework—an event-based representation that uses a directed acyclic graph to represent the narrative structure—and a series of empirically defined design guidelines for map construction obtained from a user study. Furthermore, our narrative extraction pipeline is based on maximizing coherence—modeled as a function of surface text similarity and topical similarity—subject to coverage—modeled through topical clusters—and structural constraints through the use of linear programming optimization. For the purposes of our evaluation, we focus on the news narrative domain and showcase the capabilities of our model through several case studies and user evaluations. Moreover, we augment the narrative maps framework with interactive AI techniques—using semantic interaction and explainable AI—to create an interactive narrative model that is capable of learning from user interactions to customize the narrative model based on the user's needs and providing explanations for each core component of the narrative model. Throughout this process, we propose a general framework for interactive AI that can handle similar models to narrative maps—that is, models that mix continuous low-level representations (e.g., dimensionality reduction) with more abstract high-level discrete structures (e.g., graphs). Finally, we evaluate our proposed framework through an insight-based user study. In particular, we perform a quantitative and qualitative assessment of the behavior of users and explore their cognitive strategies, including how they use the explainable AI and semantic interaction capabilities of our system. Our evaluation shows that our proposed interactive AI framework for narrative maps is capable of aiding users in finding more insights from data when compared to the baseline. / Doctor of Philosophy / Narratives are essential to how we understand the world. They help us make sense of events that happen over time. This research focuses on developing a method to assist people, like journalists and analysts, in understanding complex information. To do this, we introduce a new approach called narrative maps. This model allows us to extract and visualize stories from text data. To improve our model, we use interactive artificial intelligence techniques. These techniques allow our model to learn from user feedback and be customized to fit different needs. We also use these methods to explain how the model works, so users can understand it better. We evaluate our approach by studying how users interact with it when doing a task with news stories. We consider how useful the system is in helping users gain insights. Our results show that our method aids users in finding important insights compared to traditional methods.

With todays flood of data coming from many types of sources, Machine Learning becomes increasingly important. Though, many times the use of Machine Learning is not enough to make sense of all this data. This makes visualization a very useful tool for Machine Learning practitioners and data analysts alike. Interactive visualization techniques can be very helpful by giving insight on the meaning of the output from classification tasks. In this work, the aim is to explore, implement and evaluate different visualization techniques with the explicit goal of directly relating these visualization to the Machine Learning process. The proposed approach is the development of visualization techniques for a posteriori analysis that combines data exploration and classification evaluation. Results include a modified version of the Radial Visualization technique, called Dual RadViz, and also the use of interactive multiclass Partial Dependence Plots as means of finding counterfactual explanations about Machine Learning classification. An account of some of the many ways Machine Learning and visualization are used together is also given. / Hoje em dia, com o enorme fluxo de dados provenientes de muitos tipos de fontes, Aprendizado de Máquina se torna cada vez mais importante. No entanto, muitas vezes o uso de Aprendizado de Máquina não é o suficiente para que seja possível enxergar o valor e o significado de todos estes dados. Isso faz com que visualização seja uma valiosa ferramenta tanto para analistas de dados quanto para aqueles que praticam tarefas relacionadas à Aprendizado de Máquina. Técnicas de visualização interativa podem ser de grande utilidade por possibilitarem insights sobre o significado do resultado de tarefas de classificação. Neste trabalho, o objetivo é explorar, implementar e avaliar diferentes técnicas de visualização, explicitamente focando em suas relações com o processo de Aprendizado de Máquina. A abordagem proposta se trata do desenvolvimento de técnicas de visualização para análise a posteriori dos resultados de tarefas de classificação, combinando avaliação da classificação e exploração visual de dados. Os resultados incluem uma versão modificada da técnica de Visualização Radial, chamada Dual RadViz, e também o uso de Gráficos de Dependência Parcial multiclasse interativos como meio de se chegar à explicações contrafatuais sobre resultados de classificação. É dado também um relato de algumas das muitas maneiras onde Aprendizado de Máquina e visualização são usados conjuntamente.

Military acquisition programs have long been criticized for the exponential growth in program costs required to generate modest improvements in capability. One of the most promising reform efforts to address this trend is the open system architecture initiative, which uses modular design principles and commercial interface standards as a means to reduce the cost and complexity of upgrading systems over time. While conceptually simple, this effort has proven to be exceptionally difficult to implement in practice. This difficulty stems, in large part, from the fact that open systems trade additional cost and risk in the early phases of development for the option to infuse technology at a later date, but the benefits provided by this option are inherently uncertain. Practical implementation therefore requires a decision support framework to determine when these uncertain, future benefits are worth the cost and risk assumed in the present. The objective of this research is to address this gap by developing a method to measure the expected costs, benefits and risks associated with open systems. This work is predicated on three assumptions: (1) the purpose of future technology infusions is to keep pace with the uncertain evolution of operational requirements, (2) successful designs must justify how future upgrades will be used to satisfy these requirements, and (3) program managers retain the flexibility to adapt prior decisions as new information is made available over time. The analytical method developed in this work is then applied to an example scenario for an aerial Intelligence, Surveillance, and Reconnaissance platform with the potential to upgrade its sensor suite in future increments. Final results demonstrate that the relative advantages and drawbacks between open and integrated system architectures can be presented in the context of a cost-effectiveness framework that is currently used by acquisition professionals to manage complex design decisions.

Técnicas de visualização de informaçoes têm sido usadas para a representação de ontologias visando permitir a compreensão de conceitos e propriedades em domínios específicos. A visualização de ontologias deve ser baseada em representaccões gráficas efetivas e téquinas de interação que auxiliem tarefas de usuários relacionadas a diferentes entidades e aspectos. Ontologias podem ser complexas devido tanto à grande quantidade de níveis da hierarquia de classes como também aos diferentes atributos. Neste trabalho, propo˜e-se uma abordagem baseada no uso de múltiplas e coordenadas visualizações para explorar ambos os níceis intensional e extensional de uma ontologia. Para tanto, são empregadas estruturas visuais baseadas em árvores que capturam a característica hierárquiva de partes da ontologia enquanto preservam as diferentes categorias de classes. Além desta contribuição, propõe-se um inovador emprego do conceito "Degree of Interest" de modo a reduzir a complexidade da representação da ontologia ao mesmo tempo que procura direcionar a atenção do usuádio para os principais conceitos de uma determinada tarefa. Através da análise automáfica dos diferentes aspectos da ontologia, o principal conceito é colocado em foco, distinguindo-o, assim, da informação desnecessária e facilitando a análise e o entendimento de dados correlatos. De modo a sincronizar as visualizações propostas, que se adaptam facilmente às tarefas de usuários, e implementar esta nova proposta de c´calculo baseado em "Degree of Interest", foi desenvolvida uma ferramenta de visualização de ontologias interativa chamada OntoViewer, cujo desenvolvimento seguiu um ciclo interativo baseado na coleta de requisitos e avaliações junto a usuários em potencial. Por fim, uma última contribuição deste trabalho é a proposta de um conjunto de "guidelines"visando auxiliar no projeto e na avaliação de téncimas de visualização para os níceis intensional e extensional de ontologias. / Visualization techniques have been used for the representation of ontologies to allow the comprehension of concepts and properties in specific domains. Techniques for visualizing ontologies should be based on effective graphical representations and interaction techniques that support users tasks related to different entities and aspects. Ontologies can be very large and complex due to many levels of classes’ hierarchy as well as diverse attributes. In this work we propose a multiple, coordinated views approach for exploring the intensional and extensional levels of an ontology. We use linked tree structures that capture the hierarchical feature of parts of the ontology while preserving the different categories of classes. We also present a novel use of the Degree of Interest notion in order to reduce the complexity of the representation itself while drawing the user attention to the main concepts for a given task. Through an automatic analysis of ontology aspects, we place the main concept in focus, distinguishing it from the unnecessary information and facilitating the analysis and understanding of correlated data. In order to synchronize the proposed views, which can be easily adapted to different user tasks, and implement this new Degree of Interest calculation, we developed an interactive ontology visualization tool called OntoViewer. OntoViewer was developed following an iterative cycle of refining designs and getting user feedback, and the final version was again evaluated by ten experts. As another contribution, we devised a set of guidelines to help the design and evaluation of visualization techniques for both the intensional and extensional levels of ontologies.

As more and more data is created each day, researchers from different science domains are trying to make sense of it. A lot of this data, for example our connections to friends on different social networking websites, can be modeled as graphs, where the nodes are actors and the edges are relationships between them. Researchers analyze this data to find new forms of communication, to explore different social groups or subgroups, to detect illegal activities or to seek for different communication patterns that could help companies in their marketing campaigns. Another example are huge networks in system biology. Their visualization is crucial for the understanding of living beings. The topological structure of a network on its own could give insight into the existence or distribution of interesting actors in the network. However, this is often not enough to understand complex network systems in real-world applications. The reason for this is that all the network elements (nodes or edges) are not simple one-dimensional data. For instance in biology, experiments can be performed on biological networks. These experiments and network analysis approaches produce additional data that are often important to be analyzed with respect to the underlying network structure. Therefore, it is crucial to visualize the additional attributes of the network while preserving the network structure as much as possible. The problem is not trivial as these so-called multivariate networks could have a high number of attributes that are related to their nodes, edges, different groups, or clusters of nodes and/or edges. The aim of this thesis is to contribute to the development of different visualization and interaction techniques for the visual analysis of multivariate networks. Two research goals are defined in this thesis: first, a deeper understanding of existing approaches for visualizing multivariate networks should be acquired in order to classify them into categories and to identify disadvantages or unsolved visualization challenges. The second goal is to develop visualization and interaction techniques that will overcome various issues of these approaches. Initially, a brief survey on techniques to visualize multivariate networks is presented in this thesis. Afterwards, a small task-based user study investigating the usefulness of two main approaches for multivariate network visualization is discussed. Then, various visualization and interaction techniques for multivariate network visualization are presented. Three different software tools were implemented to demonstrate our research efforts. All features of our systems are highlighted, including a description of visualization and interaction techniques as well as disadvantages and scalability issues if present.

El gran volumen de datos que las técnicas de minería Web generan sobre espacios Web puede llegar a ser muy difícil de entender, provocando la necesidad de desarrollar nuevas técnicas que permitan generar conocimiento sobre esos datos con el fin de facilitar la toma de decisiones. Esta tesis explora la utilización de técnicas de InfoVis/VA para ayudar en la exploración de espacios Web. Más concretamente, presentamos el desarrollo de un prototipo muy flexible que ha sido utilizado para analizar tres tipos distintos de espacios Web con distintas metas informacionales: el análisis de la usabilidad de páginas Web, la evaluación del comportamiento de los estudiantes en entornos virtuales de aprendizaje y la exploración de la estructura de grandes conversaciones asíncronas existentes en foros online. Esta tesis pretende aceptar el reto propuesto por la comunidad de InfoVis/VA de llevar a cabo investigaciones en condiciones más reales, introduciendo los problemas relacionados con el análisis de los espacios Web ya mencionados, y explorando las ventajas de utilizar las visualizaciones proporcionadas por nuestra herramienta con usuarios reales. / The vast amount of data that Web mining techniques generate from Web spaces is difficult to understand, suggesting the need to develop new techniques to gather insight into them in order to assist in decision making processes. This dissertation explores the usage of InfoVis/VA techniques to assist in the exploration of Web spaces. More specifically, we present the development of a customisable prototype that has been used to analyse three different types of Web spaces with different information goals: the analysis of the usability of a website, the assessment of the students in virtual learning environments, and the exploration of the structure of large asynchronous conversations existing in online forums. Echoing the call of the Infovis/VA community for the need for more research into realistic circumstances, we introduce the problems of the analysis of such Web spaces, and further explore the benefits of using the visualisations provided by our system with real users. / El gran volum de dades que les tècniques de mineria Web proporcionen sobre els espais Web és generalment molt difícil dʼentendre, provocant la necessitat de desenvolupar noves tècniques que permetin generar coneixement sobre les dades de manera que facilitin la presa de decissions. Aquesta tesi explora la utilizació de tècniques dʼInfovis/VA per ajudar en lʼexploració dʼespais Web. Més concretament, presentem el desenvolupament dʼun prototipus molt flexible que hem utilitzat per analitzar tres tipus diferents dʼespais Web amb diferents objectius informacionals: lʼanèlisi de la usabilitat de pàgines Web, lʼavaluació del comportament dels estudiants en entorns virtuals dʼaprenentatge i lʼexploració de lʼestructura de grans converses asíncrones existents en fòrums online. Aquesta tesi pretén acceptar el repte proposat per la comunitat dʼInfoVis/VA de fer recerca en condicions més reals, introduint els problemes relacionats en lʼanàlisi dels espais Web ja esmentats, i explorant els avantatges dʼutilizar les visualitzacions proporcionades per la nostra eina amb usuaris reals.