Thesis (M.A. in Security Studies (Homeland Security and Defense))--Naval Postgraduate School, March 2009. / Thesis Advisor(s): Bergin, Richard ; Simeral, Robert. "March 2009." Description based on title screen as viewed on April 24, 2009. Author(s) subject terms: Fire, Intelligence, Firefighter Training, First Responders, Fire Service Intelligence, Open Source Intelligence, Fire Service. Includes bibliographical references (p. 77-79). Also available in print.

The proliferation of the Internet has amplified the use of social networking sites by creating a platform that encourages individuals to share information. As a result there is a wealth of information that is publically and easily accessible. This research explores whether open source intelligence (OSINT), which is freely available, could be used as a digital forensic investigative tool. A survey was created and sent to digital forensic investigators to establish whether they currently use OSINT when performing investigations. The survey results confirm that OSINT is being used by digital forensic investigators when performing investigations but there are currently no guidelines or frameworks available to support the use thereof. Additionally, the survey results showed a belief amongst those surveyed that evidence gleaned from OSINT sources is considered supplementary rather than evidentiary. The findings of this research led to the development of a framework that identifies and recommends key processes to follow when conducting OSINT investigations. The framework can assist digital forensic investigators to follow a structured and rigorous process, which may lead to the unanimous acceptance of information obtained via OSINT sources as evidentiary rather than supplementary in the near future.

Investigations are increasingly conducted online by not only novice sleuths but also by professionals -- in both competitive and collaborative environments. These investigations rely on publicly available information, called open source intelligence (OSINT). However, due to their online nature, OSINT investigations often present coordination, technological, and ethical challenges. Through semi-structured interviews with 14 professional OSINT investigators from nine different organizations, we examine the social collaboration and competition patterns that underlie their investigations. Instead of purely competitive or purely collaborative social models, we find that OSINT organizations employ a combination of both, and that each has its own advantages and disadvantages. We also describe investigators' use of and challenges with existing OSINT tools. Finally, we conclude with a discussion on supporting investigators' with more appropriable tools and making investigations more social. / Master of Science / Investigations are increasingly conducted online by not only novice investigators but also by professionals, such as private investigators or law enforcement agents. These investigations are conducted in competitive environments, such as Capture The Flag (CTF) events where contestants solve crimes and mysteries, but also in collaborative environments, such as teams of investigative journalists joining skills and knowledge to uncover and report on crimes and/or mysteries. These investigations rely on publicly available information called open source intelligence (OSINT) which includes public social media posts, public databases of information, public satellite imagery...etc. OSINT investigators collect and authenticate open source intelligence in order to conduct their investigations and synthesize the authenticated information they gathered to present their findings. However, due to their online nature, OSINT investigations often present coordination, technological, and ethical challenges. Through semi-structured interviews with 14 professional OSINT investigators from nine different organizations, we examine how these professionals conduct their investigations, and how they coordinate the different individuals and investigators involved throughout the process. By analyzing these processes, we can discern the social collaboration and competition patterns that enable these professionals to conduct their investigations. Instead of purely competitive or purely collaborative social models, we find that OSINT organizations employ a combination of both, and that each has its own advantages and disadvantages. In other words, professional OSINT investigators compete with each other but also collaborate with each other at different stages of their investigations or for different investigative tasks. We also describe investigators' use of and challenges with existing OSINT tools and technologies. Finally, we conclude with a discussion on supporting investigators with tools that can adapt to their different needs and investigation types and making investigations more social.

Open source intelligence (OSINT) has always been strongly tied to the information and communication technology (ICT) of the day. This paper is an examination of the current state of OSINT as it relates to ICTs by looking at overarching problems that exist across multiple types of collection methods, as well as looking at specific cases where there are issues, such as China and the Middle East, and ending with some minor recommendations on how to fix or minimize the issues highlighted. / text

The research focuses on the utilisation of open-source information in augmentation of the all-source counterespionage endeavour. The study has the principal objective of designing, contextualising and elucidating a micro-theoretical framework for open-source environmental scanning within the civilian, statutory counterespionage sphere. The research is underpinned by the central assumption that the environmental scanning and the contextual analysis of overt information will enable the identification, description and prioritisation of espionage risks that would not necessarily have emerged through the statutory counterespionage process in which secretly collected information predominates. The environmental scanning framework is further assumed to offer a theoretical foundation to surmount a degenerative counterespionage spiral driven by an over-reliance on classified information. Flowing from the central assumption, five further assumptions formulated and tested in the research are the following: (1) A methodically demarcated referent premise enables the focusing and structuring of the counterespionage environmental scanning process amid the exponential proliferation of overt information. (2) Effective environmental scanning of overt information for counterespionage necessitates a distinctive definition of ‘risk’ and ‘threat’, as these are interlinked yet different concepts. It is therefore asserted that current notions of ‘threat’ and ‘risk’ are inadequate for feasible employment within an overt counterespionage environmental scanning framework. (3) A framework for overt counterespionage environmental scanning has as its primary requirement the ability to identify diverse risks, descriptively and predicatively, on a strategic as well as a tactical level. (4) The degree of adversity in the relationship between a government and an adversary constitutes the principal indicator and determinant of an espionage risk. (5) The logical accommodation of a framework for overt counterespionage environmental scanning necessitates a distinctive counterintelligence cycle, as existing conceptualisations of the intelligence cycle are inadequate. The study’s objective and the testing of these five assumptions are pursued on both the theoretical and pragmatic-utilitarian levels. The framework for counterespionage, open-source environmental scanning and risk assessment is presented as part of a multilayered unison of alternative theoretical propositions on the all-source intelligence, counterintelligence and counterespionage processes. It is furthermore advanced from the premise of an alternative proposition on an integrated approach to open-source intelligence. On a pragmatic-utilitarian level, the framework’s design is informed and its application elucidated through an examination of the 21st century espionage reality confronting the nation state, contemporary statutory counterintelligence measures and the ‘real-life’ difficulties of open-source intelligence confronting practitioners. Although with certain qualifications, the assumptions are in the main validated by the research. The research furthermore affirms this as an exploratory thesis in a largely unexplored field. / Thesis (Ph.D)--University of Pretoria, 2010. / Political Sciences / Unrestricted

This research attempts to create a novel process, Social Engineering Vulnerability Evaluation, SiEVE, to use open source data and open source intelligence (OSINT) to perform efficient and effectiveness spear phishing attacks. It is designed for use by "œred teams" and students learning to conduct a penetration test of an organization, using the vector of their workforce. The SiEVE process includes the stages of identifying targets, profiling the targets, and creating spear phishing attacks for the targets. The contributions of this research include the following: (1) The SiEVE process itself was developed using an iterative process to identify and fix initial shortcomings; (2) Each stage of the final version of the SiEVE process was evaluated in an experiment that compared performance of students using SiEVE against performance of those not using SiEVE in order to test effectiveness of the SiEVE process in a learning environment; Specifically, the study showed that those using the SiEVE process (a) did not identify more targets, (b) did identify more information about targets, and (c) did lead to more effective spear phishing attacks. The findings, limitations, and future work are discussed in order to provide next steps in developing formalized processes for red teams and students learning penetration testing.

<p dir="ltr">Open-source intelligence is utilized to identify individuals and compare changes in social media profiles and content. The proliferation of social media platforms and apps has facilitated the creation, distribution, and consumption of material related to human trafficking. Social media and internet service providers are not obligated to monitor users for trafficking-related activities or content. </p><p dir="ltr">However, an increase in minors joining social media leads to a rise in predatory activity. With the escalation of predatory behavior, research can focus on communication patterns, grooming, and victim profiles targeted by criminals. Technology has been developed to identify biometric points, aiding the identification of victims and criminals. Open-source intelligence is just one step toward gathering information about victims and criminals. It can be utilized throughout the investigative process to prevent human trafficking and related crimes.</p><p dir="ltr">This research employs open-source intelligence to provide investigators, law enforcement, and government agencies with preventative solutions for this global issue. The study focuses on extracting, collecting, and analyzing social media and OSINT, specifically social media intelligence (SOCMINT). Classification patterns were identified, and suspicious behavior indicative of human trafficking was detected using the JAPAN principle approach, reducing information overload. <br><br>Additionally, the research introduced a standardized investigation framework based on gathered data. This framework demonstrated the effectiveness of selected SOCMINT tools in enhancing human trafficking investigations. The study emphasizes the need for adaptive tools in SOCMINT, complemented by innovative approaches, to strengthen law enforcement efforts in deterring human trafficking. </p>

Nous modélisons une chaîne de traitement de documents comme un processus de décision markovien, et nous utilisons l’apprentissage par renforcement afin de permettre à l’agent d’apprendre à construire des chaînes adaptées à la volée, et de les améliorer en continu. Nous construisons une plateforme qui nous permet de mesurer l’impact sur l’apprentissage de divers modèles, services web, algorithmes, paramètres, etc. Nous l’appliquons dans un contexte industriel, spécifiquement à une chaîne visant à extraire des événements dans des volumes massifs de documents provenant de pages web et d’autres sources ouvertes. Nous visons à réduire la charge des analystes humains, l’agent apprenant à améliorer la chaîne, guidé par leurs retours (feedback) sur les événements extraits. Pour ceci, nous explorons des types de retours différents, d’un feedback numérique requérant un important calibrage, à un feedback qualitatif, beaucoup plus intuitif et demandant peu, voire pas du tout, de calibrage. Nous menons des expériences, d’abord avec un feedback numérique, puis nous montrons qu’un feedback qualitatif permet toujours à l’agent d’apprendre efficacement. / Information extraction (IE) is defined as the identification and extraction of elements of interest, such as named entities, their relationships, and their roles in events. For example, a web-crawler might collect open-source documents, which are then processed by an IE treatment chain to produce a summary of the information contained in them.We model such an IE document treatment chain} as a Markov Decision Process, and use reinforcement learning to allow the agent to learn to construct custom-made chains ``on the fly'', and to continuously improve them.We build a platform, BIMBO (Benefiting from Intelligent and Measurable Behaviour Optimisation) which enables us to measure the impact on the learning of various models, algorithms, parameters, etc.We apply this in an industrial setting, specifically to a document treatment chain which extracts events from massive volumes of web pages and other open-source documents.Our emphasis is on minimising the burden of the human analysts, from whom the agent learns to improve guided by their feedback on the events extracted. For this, we investigate different types of feedback, from numerical rewards, which requires a lot of user effort and tuning, to partially and even fully qualitative feedback, which is much more intuitive, and demands little to no user intervention. We carry out experiments, first with numerical rewards, then demonstrate that intuitive feedback still allows the agent to learn effectively.Motivated by the need to rapidly propagate the rewards learnt at the final states back to the initial ones, even on exploration, we propose Dora: an improved version Q-Learning.

The amount of open-source information – that is, data, images, and footage that is openly available to the public - is growing exponentially. With it, so is the number of citizens analysing this data to form open-source intelligence (OSINT). Using the 2022 invasion of Ukraine as a case study, this study highlights the motivations behind the citizen OSINT analysts who are uncovering events on the frontline and verifying significant amounts of data from such events. Through interviews with 10 citizen OSINT analysts – all voluntarily contributing to OSINT in relation to the invasion of Ukraine, as well as other major OSINT projects – this study demonstrates the motivations behind this growing community. The findings reflect a new era of participation and advocacy and are a demonstration of self-determination theory. The findings demonstrate citizens’ sceptic views towards traditional media but also, that with a more analytical approach, with improved transparency and collaboration there is reason to be optimistic about the future of journalism and audience engagement.

Indiana University-Purdue University Indianapolis (IUPUI) / Asynchronous JavaScript and XML (AJAX) is the primary method for enabling asynchronous communication over the Web. Although AJAX is providing warranted real-time capabilities to the Web, it requires unconventional programming methods at the expense of extensive resource usage. WebSockets, which is an emerging protocol, has the potential to address many challenges with implementing asynchronous communication over the Web. There, however, has been no in-depth study that quantitatively compares AJAX and WebSockets. This thesis therefore provides two contributions to Web development. First, it provides an experience report for adding real-time monitoring support over the Web to the Open-source Architecture of Software Instrumentation of Systems(OASIS), which is open-source real-time instrumentation middleware for distributed real-time and embedded (DRE) systems. Secondly, it quantitatively compares using AJAX and WebSockets to stream collected instrumentation data over the Web in real-time. Results from quantitative comparison between WebSockets and AJAX show that a WebSockets server consumes 50% less network bandwidth than an AJAX server; a WebSockets client consumes memory at constant rate, not at an increasing rate; and WebSockets can send up to 215.44% more data samples when consuming the same amount network bandwidth as AJAX.