Android malware collusion is a new threat model that occurs when multiple Android apps communicate in order to execute an attack. This threat model threatens all Android users' private information and system resource security. Although recent research has made advances in collusion detection and classification, security analysts still do not have robust tools which allow them to definitively identify colluding Android applications. Specifically, in order to determine whether an alert produced by a tool scanning for Android collusion is a true-positive or a false-positive, the analyst must perform manual analysis of the suspected apps, which is both time consuming and prone to human errors. In this thesis, we present a new approach to definitive Android collusion detection and confirmation by rendering inter-component communications between a set of potentially collusive Android applications. Inter-component communications (abbreviated to ICCs), are a feature of the Android framework that allows components from different applications to communicate with one another. Our approach allows Android security analysts to inspect all ICCs within a set of suspicious Android applications and subsequently identify collusive attacks which utilize ICCs. Furthermore, our approach also visualizes all potentially collusive data-flows within each component within a set of apps. This allows analysts to inspect, step-by-step, the the data-flows that are currently used by collusive attacks, or the data-flows that could be used for future collusive attacks. Our tool effectively visualizes the malicious and benign ICCs in sets of proof-of-concept and real-world colluding applications. We conducted a user study which revealed that our approach allows for accurate and efficient identification of true- and false-positive collusive ICCs while still maintaining usability. / Master of Science

Malicious software, or malware, is often employed as a tool to maintain access to previously compromised systems. It enables the intruders to utilize system resources, harvest legitimate credentials, and maintain a level of stealth throughout the process. During incident response, identifying systems infected with malware is necessary for effective remediation of an attack. When analysts lack sufficient indicators of compromise they are forced to conduct a comprehensive examination to identify anomalous behavior on a system, a time consuming and challenging task. Malware authors use several techniques to conceal malware on a system, with a common method being DLL injection. In this dissertation we present a system for automatically generating Windows 7 x86 memory images infected with malware, identifying the malicious DLLs injected into a process, and extracting the features associated with those DLLs. A set of 3,240 infected memory images was produced and analyzed to identify common characteristics of malicious DLLs in memory. From this analysis a feature set was constructed and two datasets were used to evaluate five classification algorithms. The ZeroR method was used as a baseline for comparison with accuracy and false positive rate (misclassifying malicious DLLs as legitimate) being the two metrics of interest. The results of the experiments showed that learning using the feature set is viable and that the performance of the classifiers can be further improved through the use of feature selection. Each of the classification methods outperformed the ZeroR method with the J48 Decision Tree obtaining the, overall, best results.

The popularity of smart phones has risen throughout the years since first introduced. With the popularity of the devices growing so too has the number of malicious applications flooding the devices’ marketplaces. With more usage there becomes a larger target for malware and exploitation creation. As threats to these devices continue to grow there is a constant need for security to safeguard against these threats. Some attempts to protect smart phones involve building software to analyze applications running on the devices. This attempt has cut back on the amount of malicious software on the marketplace. These attempts however only catch malicious applications after they have been running. This dissertation presents the Secure Android Development Model. The goal of this model is to contribute to security of these devices by having a development model that implicitly builds security into applications. The model ensures a minimal amount of open permissions thus limiting the number of attack vectors that malicious software can make on the devices. By following the model, developers will have all information available during development to make appropriate security decisions in their applications.

Malware analysts and researchers around the world are looking for innovative means of malware detection and classification. However, one concept of malware analysis that lacks focus is the rating of malware based on their feature set and capabilities. Malware severity rating is needed in order to prioritize the utilization of resources towards the analysis of a malware by an organization. This thesis proposes the utilization of threat trees for calculating malware severity using a goal oriented approach. This approach is applied to a set of sophisticated malware to study its contribution towards articulation of a useful severity rating.

Malware detection is typically performed using either software scanners running inside the operating system or external devices designed to validate the integrity of the kernel. This thesis proposes a hypervisor-based malware that compromises the system by targeting the hard disk drive and leaving the kernel unmodified. The hypervisor is able to issue read and write commands to the disk while actively hiding these actions from the operating system and any detection software therein. Additionally, the hypervisor’s presence has minimal impact on the performance of the system. The ability to perform these commands compromises the confidentiality, integrity, and availability of the stored data. As a result, this thesis has widespread implications affecting personal, corporate, and government users alike.

Malware is a fast growing threat that consists of a malicious script or piece of software that is used to disrupt the integrity of a user's experience. Antivirus software can help protect a user against these threats and there are numerous vendors users can choose from for their antivirus protection. However, each vendor has their own set of virus definitions varying in resources and capabilities in recognizing new threats. Currently, a persistent system is not in place that measures and displays data on the performance of antivirus vendors in responding to new malware over a continuous period of time. There is a need for a system that can evaluate antivirus performance in order to better inform end users of their security options, in addition to informing clients of prevalent threats occurring in their network. This project is dedicated to assessing the viability of a cloud sourced malware reporting framework that uses distributed agents to evaluate the performance of antivirus software based on malware signatures.

Natural Language machine learning methods are applied to rules generated to identify malware at the network level. These rules use a computer-based signature specification "language" called Snort. Using Natural Language processing techniques and other machine learning methods, new rules are generated based on a training set of existing Snort rule signatures for a specific type of malware family. The performance is then measured, in terms of the detection of existing types of malware and the number of "false positive" triggering events.

Yes / The current rise in malicious attacks shows that existing security systems are bypassed by malicious files. Similarity hashing has been adopted for sample triaging in malware analysis and detection. File similarity is used to cluster malware into families such that their common signature can be designed. This paper explores four hash types currently used in malware analysis for portable executable (PE) files. Although each hashing technique produces interesting results, when applied independently, they have high false detection rates. This paper investigates into a central issue of how different hashing techniques can be combined to provide a quantitative malware score and to achieve better detection rates. We design and develop a novel approach for malware scoring based on the hashes results. The proposed approach is evaluated through a number of experiments. Evaluation clearly demonstrates a significant improvement (> 90%) in true detection rates of malware.

Malware is the source or a catalyst for many of the attacks on our cyberspace. Malware analysts and other cybersecurity professionals are responsible for responding to and understanding attacks to mount a defense against the attacks in our cyberspace. The sheer amount of malware alone makes this a difficult task, but malware is also increasing in complexity. This research provides empirical evidence that a hybrid approach using token-based and semantic-based code clones can identify similarities between malware. In addition, the use of different normalization techniques and the use of undirected matrices versus directed matrices were studied. Lastly, the impact of the use of inexact code clones was evaluated. Our results showed that our approach to determining the similarity between malware outperforms two methods currently used in malware analyses. In addition, we showed that overly generalized normalization of code sections would hinder the performance of the proposed method. At the same time, there is no significant difference between the use of directed and undirected matrices. This research also confirmed the positive impact of using inexact code clones when determining similarity.

Detectar e analisar malwares é um processo essencial para aprimorar os sistemas de segurança. As soluções atuais apresentam limitações no processo de investigação e detecção de códigos maliciosos sofisticados. Mais do que utilizar técnicas para evadir sistemas de análise, malwares sofisticados requerem condições específicas no ambiente em que são executados para revelar seu comportamento malicioso. Com o surgimento das Redes Definidas por Software (SDN), notou-se uma oportunidade para aprimorar o processo de investigação de malware propondo uma arquitetura flexível apta a detectar variações comportamentais de maneira automática. Esta tese apresenta uma arquitetura especializada para analisar códigos maliciosos que permite controlar de maneira unificada o ambiente de análise, incluindo o sandbox e os elementos que o circundam. Dessa maneira, é possível gerenciar regras de contenção, configuração dinâmica de recursos, e manipular o tráfego de rede gerado pelos malwares. Para avaliar a arquitetura foi analisado um conjunto de malwares em dois cenários de avaliação. No primeiro cenário de avaliação, as funcionalidades descritas pela solução proposta revelaram novos eventos comportamentais em 100% dos malwares analisados. Já, no segundo cenários de avaliação, foi analisado um conjunto de malwares projetados para dispositivos IoT. Em consequência, foi possível bloquear ataques, monitorar a comunicação do malware com seu controlador de botnet, e manipular comandos de ataques. / Mechanisms to detect and analyze malicious software are essential to improve security systems. Current security mechanisms have limited success in detecting sophisticated malicious software. More than to evade analysis system, many malware require specific conditions to activate their actions in the target system. The flexibility of Software-Defined Networking (SDN) provides an opportunity to develop a malware analysis architecture that can detect behavioral deviations in an automated way. This thesis presents a specialized architecture to analyze malware by managing the analysis environment in a centralized way, including to control the sandbox and the elements that surrounds it. The proposed architecture enables to determine the network access policy, to handle the analysis environment resource configuration, and to manipulate the network connections performed by the malware. To evaluate our solution we have analyzed a set of malware in two evaluation scenarios. In the first evaluation scenario, we showed that the mechanisms proposed have increased the number of behavioral events in 100% of the malware analyzed. In the second evaluation scenario, we have analyzed malware designed for IoT devices. As a result, by using the MARS features, it was possible to block attacks, to manipulate attack commands, and to enable the malware communication with the respective botnet controller. The experimental results showed that our solution can improve the dynamic malware analysis process by providing this configuration flexibility to the analysis environment.