With the proliferation of sensor-enabled smartphones, significant attention has been attracted to develop sensing-driven mobile systems. Current research on sensing-driven mobile systems can be classified into two categories, based on the purpose of sensing. In the first category, smartphones are used to sense personal context information, such as locations, activities, and daily habits to enable applications such as location-aware systems and virtual reality systems. In the second category, smartphones are exploited to collect sensing data of the physical world and enable applications including traffic monitoring, environmental monitoring, and others. As smartphones become blossomed in popularity and ubiquity, new problems have emerged in both categories of mobile sensing systems. In this thesis, we investigate three core challenges by answering the following fundamental questions: first, how can we utilize user context to improve the operating system performance? Second, how can we process sensing data, especially images, with high accuracy? Third, how can we enable distributed sensing while satisfy resource constraints of smartphones? The first part of this thesis studies how to exploit user context to improve the responsiveness of mobile operating systems. We propose a context-aware application-preloading engine named FALCON. The core of FALCON is a decision engine that learns application usage patterns of mobile users and preloads applications ahead of time to improve the responsiveness of mobile OS. Compared with other approaches such as caching schemes like Least Recently Used (LRU), Falcon improves the application responsiveness by two times. The second part of this thesis focuses on image search for mobile phones. We first explore how to improve image search accuracy on centralized servers, and propose an image search engine named CrowdSearch. The core idea of CrowdSearch is to incorporate crowdsourced human validation into the system for removing erroneous results from automated image search engines, while still provide realtime response for mobile users. Compared with existing automated image search engines, CrowdSearch achieves over 95% accuracy consistently across multiple categories of images with response time in a minute. We then extend image search to distributed mobile phones, and emphasis resource constraint problems, especially on energy and bandwidth. We propose a distributed image search system named SenSearch, which turns smartphones into micro image search engines. Images are collected, indexed, and transmitted using compact features that are two magnitudes smaller than their raw format. SenSearch improves the energy and bandwidth cost by five times compared with centralized image search engines.

No description available.

For decades, the relationship between journalists and PR representatives has been found to be a difficult one, due to a lot of economic and professional pressure on journalists, and PR agents trying to push their agenda. Digitalization has brought many new tools into the game and one of them is turning this relationship around: Media Catching. Such services put the power back into the hands of journalists and other content creators, by giving them the opportunity of crowdsourcing sources for their specific stories by sending out a request to a big pool of sources including PR agents, entrepreneurs, experts etc. While such services are growing in the US, they are not established in Sweden yet. Hence, this study looks at two questions: (1) How should a media catching service in Sweden be designed, in order to meet the requirements of the Swedish media and PR landscape? (2) For which different target groups within the Media and PR landscape could this service be beneficial? To find answers, this research conducted qualitative interviews with three representatives of the Swedish media and five PR agents and/or entrepreneurs. Literature research, previous studies and a benchmark analysis of media catching services delivered further insights. In terms of designing a media catching service, the results showed that it should allow journalists to choose whether they want to be contacted via mail, phone or social media; it should send out requests via mail; different Facebook integrations are discussed due to its big popularity; it should include an option for users to gather the contacts they made through the service; the question whether it should offer an online community should further be investigated; Users should be given personalization and targeting options so that sources can choose how often and which media requests they want to receive, and so that content creators can send their requests to their specific target group. At least four different user groups were found that could benefit of a media catching service in Sweden. Due to their different demands, knowledge and target groups the media catching service should either focus only on two of these groups or offer enough targeting and personalization options to make it successful for all of them. Finally, it is important to ensure quality on a media catching platform which can be achieved through different steps, like offering trainings for the users or implementing different control functions. Hence, when establishing a media catching service in Sweden, all of the above mentioned findings can serve as a guideline for designing it and for understanding which groups could be targeted with it to create a user base. / Sedan årtionden tillbaka har förhållandet mellan journalister och PR-representanter visat sig vara problematiskt, på grund av en både större ekonomisk och professionell press på journalister samt att PR-agenter trycker på att få igenom sin egen agenda. Digitaliseringen har medfört många nya verktyg och förhållningssätt, och speciellt ett kan skifta den här balansen: Media Catching. Sådana tjänster ger tillbaka makten till journalister och andra innehållsskapare, genom att ge dem möjligheten att använda? crowdsourcing källor för deras specifika berättelser genom att skicka ut en förfrågan till en stor grupp av källor inklusive kommunikatörer, entreprenörer, experter mm. Även om sådana tjänster växer i USA har de inte blivit etablerade i Sverige ännu. Denna studie fokuserar därför på två frågor: (1) Hur ska en media catching-tjänst i Sverige utformas för att svara mot förutsättningarna i svenska medier och PR- landskap? (2) Vilka målgrupper inom media och PR skulle dra fördel av denna tjänst? För att svara på detta genomfördes kvalitativa intervjuer med tre företrädare för svenska medier och fem PR-agenter och / eller företagare. Litteratursökning, tidigare studier och ett benchmark analys av media catching -tjänster resulterade i vidare insikter. När det gäller att utforma en media catching-tjänst, visade resultaten att den bör tillåta journalister att välja om de vill bli kontaktade via e-post, telefon eller sociala medier; Den bör skicka ut förfrågningar via e-post; olika sammankopplingar via Facebook diskuteras på grund av sin höga popularitet; den bör innehålla en möjlighet för användare att samla kontakterna de får via tjänsten; frågan om den bör erbjuda en online community bör vidare undersökas; Användarna bör ges anpassningsmöjligheter för egna preferenser så att källor kan välja hur ofta och vilka mediaförfrågningar de vill ta emot, och så att innehållsskapare kan skicka sina ansökningar till deras specifika målgrupp. Minst fyra olika användargrupper hittades som skulle kunna dra nytta av en media catching tjänst i Sverige. På grund av deras olika behov, kunskap och målgrupp borde media fånga tjänsten antingen bara fokusera på två av dessa grupper eller erbjuda tillräckligt med inriktning och personaliseringsalternativ för att göra den framgångsrik för alla fyra. Slutligen är det viktigt att säkerställa kvaliteten på en media catching plattform vilket kan uppnås genom olika steg, som att erbjuda utbildningar för användarna eller att genomföra olika styrfunktioner. Därav, vid upprättandet av en media catching tjänst i Sverige, samtliga ovannämnda resultat kan tjäna som en riktlinje för att utforma tjänsten och för att förstå vilka grupper som skulle kunna utformas med den för att skapa en användarbas.

Mobility and safety are primary concerns for blind and visually impaired (BVI) users when navigating in unfamiliar environments. Typically, a sighted person can locate a place of interest if they are provided guidance while approaching within a few meters of the location. However, this resolution of guidance is often insufficient for blind travelers. In this thesis, we propose a crowdsourcing based micro navigation system for BVI users in both indoor and outdoor environments. To achieve this goal, our system includes three parts: crowdsourcing reports generated by volunteers using the volunteer application, landmarks validation performed by the system administrator using the admin application, and the BVI user navigation obtained through the BVI user application. In addition, we provide accessible audio navigation for indoor and outdoor environments required to deliver real time step by step landmark information to BVI users. Crowdsourcing is enabled by the contribution of many volunteers which use the proposed volunteer application to report specific landmarks in the environment including their location, description and surrounding landmarks. These reports which are uploaded to the server database, are validated by the admin application which updates the server database and deploy BLE tags for indoor environment. The BVI user application localizes users by GPS outdoors and BLE proximity technology indoors. Using the real-time location of users and the landmark node graph we built from updated server database, this application provides the shortest route to the destination and real time “micro-navigation” information describing how to get to the next landmark’s location with corresponding distance & orientation. This information is used ix to make users well aware of where they are, and guide users to their chosen destination within a cane’s distance. This application will improve the confidence and safety of BVI users by enabling them to explore and get navigation in both indoor and outdoor environments.

Enabled by the wave of online crowdsourcing activities, extracting the Wisdom of Crowds (WoC) has become an emerging research area, one that is used to aggregate judgments, opinions, or predictions from a large group of individuals for improved decision making. However, existing literature mostly focuses on eliciting the wisdom of crowds in an offline context—without tapping into the vast amount of data available on online crowdsourcing platforms. To extract WoC from participants on online platforms, there exist at least three challenges, including social influence, suboptimal aggregation strategies, and data sparsity. This dissertation aims to answer the research question of how to effectively extract WoC from crowdsourcing platforms for the purpose of making better decisions. In the first study, I designed a new opinions aggregation method, Social Crowd IQ (SCIQ), using a time-based decay function to eliminate the impact of social influence on crowd performance. In the second study, I proposed a statistical learning method, CrowdBoosting, instead of a heuristic-based method, to improve the quality of crowd wisdom. In the third study, I designed a new method, Collective Persuasibility, to solve the challenge of data sparsity in a crowdfunding platform by inferring the backers' preferences and persuasibility. My work shows that people can obtain business benefits from crowd wisdom, and it provides several effective methods to extract wisdom from online crowdsourcing platforms, such as StockTwits, Good Judgment Open, and Kickstarter. / Doctor of Philosophy / Since Web 2.0 and mobile technologies have inspired increasing numbers of people to contribute and interact online, crowdsourcing provides a great opportunity for the businesses to tap into a large group of online users who possess varied capabilities, creativity, and knowledge levels. Howe (2006) first defined crowdsourcing as a method for obtaining necessary ideas, information, or services by asking for contributions from a large group of individuals, especially participants in online communities. Many online platforms have been developed to support various crowdsourcing tasks, including crowdfunding (e.g., Kickstarter and Indiegogo), crowd prediction (e.g., StockTwits, Good Judgment Open, and Estimize), crowd creativity (e.g., Wikipedia), and crowdsolving (e.g., Dell IdeaStorm). The explosive data generated by those platforms give us a good opportunity for business benefits. Specifically, guided by the Wisdom of Crowds (WoC) theory, we can aggregate multiple opinions from a crowd of individuals for improving decision making. In this dissertation, I apply WoC to three crowdsourcing tasks, stock return prediction, event outcome forecast, and crowdfunding project success prediction. Our study shows the effectiveness of WoC and makes both theoretical and practical contributions to the literature of WoC.

Identifying people in historical photographs is an important task in many fields, including history, journalism, genealogy, and collecting. A wide variety of different methods, such as manual analysis, facial recognition, and crowdsourcing, have been used to identify the unknown photos. However, because of the large numbers of candidates and the poor quality or lack of source evidence, accurate historical person identification still remains challenging. Researchers especially struggle with the ``last mile problem" of historical person identification, where they must make a selection among a small number of highly similar candidates. Collaboration, including both human-AI collaboration and collaboration within human teams, has shown the advantages of improving data accuracy, but there is lack of research about how we can design a collaborative workspace to support the historical person identification. In this work, we present SleuthTalk, a web-based collaboration tool integrated into the public website Civil War Photo Sleuth which addresses the last-mile problem in historical person identification by providing support for shortlisting potential candidates from face recognition results, private collaborative workspaces, and structured feedback interfaces. We evaluated this feature in a mixed-method study involving 6 participants, who spent one week each using SleuthTalk and a comparable social media platform to identify an unknown photo. The results of this study show how our design helps with identifying historical photos in a collaborative way and suggests directions for improvement in future work. / Master of Science / Identifying people in historical photographs is an important task in many fields, including history, journalism, genealogy, and collecting. A wide variety of different methods, such as manual analysis, facial recognition, and crowdsourcing, have been used to identify the unknown photos. However, because of the large numbers of candidates and the poor quality or lack of source evidence, accurate historical person identification still remains challenging. Researchers especially struggle with the ``last mile problem" of historical person identification, where they must make a selection among a small number of highly similar candidates. Collaboration, including both human-AI collaboration and collaboration within human teams, has shown the advantages of improving data accuracy, but there is lack of research about how we can design a collaborative workspace to support the historical person identification. In this work, we present SleuthTalk, a web-based collaboration tool integrated into the public website Civil War Photo Sleuth which addresses the last-mile problem in historical person identification by providing support for shortlisting potential candidates from face recognition results, private collaborative workspaces, and structured feedback interfaces. We evaluated this feature in a mixed-method study involving 6 participants, who spent one week each using SleuthTalk and a comparable social media platform to identify an unknown photo. The results of this study show how our design helps with identifying historical photos in a collaborative way and suggests directions for improvement in future work.

Investigations are increasingly conducted online by not only novice sleuths but also by professionals -- in both competitive and collaborative environments. These investigations rely on publicly available information, called open source intelligence (OSINT). However, due to their online nature, OSINT investigations often present coordination, technological, and ethical challenges. Through semi-structured interviews with 14 professional OSINT investigators from nine different organizations, we examine the social collaboration and competition patterns that underlie their investigations. Instead of purely competitive or purely collaborative social models, we find that OSINT organizations employ a combination of both, and that each has its own advantages and disadvantages. We also describe investigators' use of and challenges with existing OSINT tools. Finally, we conclude with a discussion on supporting investigators' with more appropriable tools and making investigations more social. / Master of Science / Investigations are increasingly conducted online by not only novice investigators but also by professionals, such as private investigators or law enforcement agents. These investigations are conducted in competitive environments, such as Capture The Flag (CTF) events where contestants solve crimes and mysteries, but also in collaborative environments, such as teams of investigative journalists joining skills and knowledge to uncover and report on crimes and/or mysteries. These investigations rely on publicly available information called open source intelligence (OSINT) which includes public social media posts, public databases of information, public satellite imagery...etc. OSINT investigators collect and authenticate open source intelligence in order to conduct their investigations and synthesize the authenticated information they gathered to present their findings. However, due to their online nature, OSINT investigations often present coordination, technological, and ethical challenges. Through semi-structured interviews with 14 professional OSINT investigators from nine different organizations, we examine how these professionals conduct their investigations, and how they coordinate the different individuals and investigators involved throughout the process. By analyzing these processes, we can discern the social collaboration and competition patterns that enable these professionals to conduct their investigations. Instead of purely competitive or purely collaborative social models, we find that OSINT organizations employ a combination of both, and that each has its own advantages and disadvantages. In other words, professional OSINT investigators compete with each other but also collaborate with each other at different stages of their investigations or for different investigative tasks. We also describe investigators' use of and challenges with existing OSINT tools and technologies. Finally, we conclude with a discussion on supporting investigators with tools that can adapt to their different needs and investigation types and making investigations more social.

Practitioners in creative domains such as web design, data visualization, and software development face many challenges while trying to create novel solutions that satisfy the guidelines around practical constraints and quality considerations. My dissertation work addresses two of these challenges. First, guidelines may conflict with each other, creating a need for slow and time-consuming expert intervention. Second, guidelines may be hard to check programmatically, requiring experts to manually use multipage style guides that suffer from drawbacks related to searchability, navigation, conflict, and obsolescence. In my dissertation, I focus on exploring mixed-initiative methods as a solution to these challenges in two complex tasks: biological network visualization where guidelines may conflict, and web design where task requirements are hard to check programmatically. For biological network visualization, I explore the use of crowdsourcing to scale up time-consuming manual layout tasks. To support the network-based collaboration required for crowdsourcing, I first implemented a system called GraphSpace. It fosters online collaboration by allowing users to store, organize, explore, lay out, and share networks on a web platform. I then used GraphSpace as the infrastructure to support a novel mixed-initiative crowd-algorithm approach for creating high-quality, biological meaningful network visualizations. I also designed and implemented Flud, a system that gamifies the graph visualization task and uses flow theory concepts to make algorithmically generated suggestions more readily accessible to non-expert crowds. Then, I proposed DeepLayout, a novel learning-based approach as an alternative to the non-machine learning-based method used in Flud. It has the ability to learn how to balance complex conflicting guidelines from a layout process. Finally, in the domain of web design, I present a real-world iterative deployment of a system called Critter. Critter augments traditional quality assurance techniques used in structured domains, such as checklists and expert feedback, using mixed-initiative interactions. I hope this dissertation can serve to accelerate research on leveraging the complementary strengths of humans and computers in the context of creative processes that are generally considered out of bounds for automated methods. / Doctor of Philosophy / Practitioners in creative domains such as web design, data visualization, and software development face many challenges while trying to create novel solutions that satisfy the guidelines around practical constraints and quality considerations. My dissertation work addresses two of these challenges. First, sometimes the guidelines may conflict with each other under a certain scenario. In this situation, tasks require expert opinion to prioritize one guideline over the other. This dependence on expertise makes the design process slow and time-consuming. Second, sometimes it is difficult to determine which guidelines have been fulfilled. In this scenario, experts have to manually go through a list of guidelines and make sure applicable guidelines have been successfully applied to the final product. However, using a list of guidelines has its own drawbacks. Not all guidelines are applicable to a project, and finding a relevant guideline can be strenuous for experts. Moreover, a design process is not as simple as following a list of guidelines. Design processes are dynamic, non-linear, and iterative. Due to these reasons, a simple list of guidelines does not align with the designers' workflow. My dissertation focuses on exploring mixed-initiative methods where computers and humans collaborate in a tight feedback loop to help follow guidelines. To this end, I present solutions for two complex creative tasks: biological network visualization where we can compute how well a design adheres to the guidelines but guidelines may conflict and web design where task requirements are hard to check programmatically. I hope this dissertation can serve to accelerate research on leveraging the complementary strengths of humans and computers in the context of creative processes that are generally considered out of bounds for automated methods.

Many colleges face a lack of academic and emotional support for their computer science students. Previous research into this problem produced Maria, a virtual teaching assistant (TA). This initial version of Maria was able to answer student questions, provide error explanations, and praise students for effort on programming assignments. This research continues work on the Maria project with three design goals: (1) reducing obstacles to use of Maria, (2) allowing Maria to provide better academic support, and (3) allowing Maria to provide better emotional support (with less focus on this goal). Improvements were made to the initial version of Maria, including increasing the number of questions that Maria could answer, allowing Maria to suggest questions for students to ask, and adding longer back-and-forth dialogs between Maria and students. Following this, Maria was deployed to students for an evaluation. The evaluation revealed that certain features were popular (including the longer dialogs and easier access to error explanation) and that Maria was unable to provide relevant answers to many questions asked by students. Using data from the evaluation, more improvements were made to Maria to address some of her shortcomings and build on her popular features. Answers to more questions were added for questions about testing, general knowledge questions, questions about many other topics. Many of these new answers used the popular back-and-forth dialog feature. Additionally, this research discusses a system that could be used to automate the creation of new answers for Maria or any virtual teaching assistant using crowdsourcing. / Master of Science / Many colleges face a lack of academic and emotional support for their computer science students. Previous research into this problem produced Maria, a virtual teaching assistant (TA). This initial version of Maria was able to answer student questions, provide error explanations, and praise students for effort on programming assignments. This research continues work on the Maria project with three design goals: (1) reducing obstacles to use of Maria, (2) allowing Maria to provide better academic support, and (3) allowing Maria to provide better emotional support (with less focus on this goal). Improvements were made to the initial version of Maria, including increasing the number of questions that Maria could answer, allowing Maria to suggest questions for students to ask, and adding longer back-and-forth dialogs between Maria and students. Following this, Maria was deployed to students for an evaluation. The evaluation revealed that certain features were popular (including the longer dialogs and easier access to error explanation) and that Maria was unable to provide relevant answers to many questions asked by students. Using data from the evaluation, more improvements were made to Maria to address some of her shortcomings and build on her popular features. Answers to more questions were added for questions about testing, general knowledge questions, questions about many other topics. Many of these new answers used the popular back-and-forth dialog feature. Additionally, this research discusses a system that could be used to automate the creation of new answers for Maria or any virtual teaching assistant using crowdsourcing.

The increasing volume and complexity of text data are challenging the cognitive capabilities of expert analysts. Machine learning and crowdsourcing present new opportunities for large-scale sensemaking, but it remains a challenge to model the overall process so that many distributed agents can contribute to suitable components asynchronously and meaningfully. In this work, I explore how to crowdsource sensemaking for intelligence analysis. Specifically, I focus on the complex processes that include developing hypotheses and theories from a raw dataset and iteratively refining the analysis. I first developed Connect the Dots, a web application that implements the concept of "context slices" and supports novice crowds in building relationship networks for exploratory analysis. Then I developed CrowdIA, a software platform that implements the entire crowd sensemaking pipeline and the context slicing for each step, to enable unsupervised crowd sensemaking. Using the pipeline as a testbed, I probed the errors and bottlenecks in crowdsourced sensemaking,and suggested design recommendations for integrated crowdsourcing systems. Building on these insights and to support iterative crowd sensemaking, I developed the concept of "crowd auditing" in which an auditor examines a pipeline of crowd analyses and diagnoses the problems to steer future refinement. I explored the design space to support crowd auditing and developed CrowdTrace, a crowd auditing tool that enables novice auditors to effectively identify the important problems with the crowd analysis and create microtasks for crowd workers to fix the problems.The core contributions of this work include a pipeline that enables distributed crowd collaboration to holistic sensemaking processes, two novel concepts of "context slices" and "crowd auditing", web applications that support crowd sensemaking and auditing, as well as design implications for crowd sensemaking systems. The hope is that the crowd sensemaking pipeline can serve to accelerate research on sensemaking, and contribute to helping people conduct in-depth investigations of large collections of information. / Doctor of Philosophy / In today's world, we have access to large amounts of data that provide opportunities to solve problems at unprecedented depths and scales. While machine learning offers powerful capabilities to support data analysis, to extract meaning from raw data is cognitively demanding and requires significant person-power. Crowdsourcing aggregates human intelligence, yet it remains a challenge for many distributed agents to collaborate asynchronously and meaningfully. The contribution of this work is to explore how to use crowdsourcing to make sense of the copious and complex data. I first implemented the concept of ``context slices'', which split up complex sensemaking tasks by context, to support meaningful division of work. I developed a web application, Connect the Dots, which generates relationship networks from text documents with crowdsourcing and context slices. Then I developed a crowd sensemaking pipeline based on the expert sensemaking process. I implemented the pipeline as a web platform, CrowdIA, which guides crowds to solve mysteries without expert intervention. Using the pipeline as a testbed, I probed the errors and bottlenecks in crowd sensemaking and provided design recommendations for crowd intelligence systems. Finally, I introduced the concept of ``crowd auditing'', in which an auditor examines a pipeline of crowd analyses and diagnoses the problems to steer a top-down path of the pipeline and refine the crowd analysis. The hope is that the crowd sensemaking pipeline can serve to accelerate research on sensemaking, and contribute to helping people conduct in-depth investigations of large collections of data.