Malware analysts often need to search large corpuses of obfuscated binaries for particular sequences of related instructions. The use of simple tactics, such as dead code insertion and register renaming, prevents the use of conventional, big-data search indexes. Current, state of the art malware detectors are unable to handle the size of the dataset due to their iterative approach to comparing files. Furthermore, current work is also frequently designed to act as a detector and not a search tool. I propose a system that exploits the observation that many data/control-flow relationships between instructions are preserved in the presence of obfuscations. The system will extract chains of flow-dependent instructions from a binary’s Program Dependence Graph (PDG). It will then use a representation of each chain as a key for an index that points to lists of functions (and their corresponding files). Analysts will be able to quickly search for instruction sequences by querying the index.

Automatic malware analysis is an essential part of today's computer security practices. Nearly one million malware samples were delivered to the analysts on a daily basis on year 2014 alone while the number of samples submitted for analysis increases almost exponentially each year. Given the size of the threat we are facing today and the amount of malicious codes emerging every day, the ability to automatically analyze unknown and unwanted software is critically important more than ever. On the other hand, malware writers adapt their malicious codes to new security measurements to protect them from being exposed and detected. This is usually achieved by employing obfuscation techniques that complicate the reverse engineering and analysis of the code by adding lots of unnecessary and irrelevant computations. Most of the malicious samples found in the wild are obfuscated and equipped with complicated anti-analysis defenses intended to hide the malicious intent of the malware by defeating the analysis and/or increasing the analysis time. Deobfuscation (reversing the obfuscation) requires automatic techniques to extract the original logic embedded in the obfuscated code for further analysis. Presumably the deobfuscated code requires less analysis time and is easier to analyze compared to the obfuscated one. Previous approaches in this regard target specific types of obfuscations by making strong assumptions about the underlying protection scheme leaving opportunities for the adversaries to attack. This work addresses this limitation by proposing new program analysis techniques that are effective against code obfuscations while being generic by minimizing the assumptions about the underlying code. We found that standard program analysis techniques, including well-known data and control flow analyses and/or symbolic execution, suffer from imprecision due to the obfuscation and show how to mitigate this loss of precision. Using more precise program analysis techniques, we propose a deobfuscation technique that is successful in reversing the complex obfuscation techniques such as virtualization-obfuscation and/or Return-Oriented Programming (ROP).

Darbo tikslas – sudaryti ir praktiškai realizuoti metodą užmaskuoto kenkėjiško programinio kodo tinklalapiuose aptikimui pagal jo savybes. Darbe nagrinėjamos tinklalapiuose talpinamo užmaskuoto kenkėjiško kodo aptikimo problemos. Išanalizuoti kenkėjiško kodo maskavimo metodai ir jo savybės. Aprašytas užmaskuoto JavaScript kodo aptikimo metodas, paremtas nustatytomis užmaskuoto kodo savybėmis ir pagal jas aprašytais paieškos kriterijais: žodžio ilgiu, simbolių skaičiumi žodyje ir simbolių dažniu žodyje. Metodas pristatytas pranešime 14-oje Lietuvos jaunųjų mokslininkų konferencijoje „Mokslas - Lietuvos ateitis“, įvykusioje Vilniuje 2011 m. balandžio 15 d. Remiantis šiuo metodu atliktas tyrimas, kurio rezultatai rodo pasiūlyto metodo veiksmingumą – pasiektas 98% užmaskuoto kodo aptikimo tinklalapiuose tikslumas. Tyrimo rezultatai paskelbti straipsnyje, kuris priimtas spausdinimui recenzuojamame periodiniame mokslo žurnale „Jaunųjų mokslininkų darbai“. Darbą sudaro: įvadas, 6 skyriai, išvados, literatūros sąrašas, priedai. Darbo apimtis – 55 p. teksto be priedų, 23 iliustr., 4 lent., 44 bibliografiniai šaltiniai. Atskirai pridedami darbo priedai. / The aim of this thesis is to suggest and practically implement a method of malicious obfuscated code detection using its characteristics. In this thesis we analyze problems of obfuscated malicious code detection in websites, malicious code obfuscation techniques and obfuscated code characteristics. In this paper suggested method of malicious obfuscated code detection in websites using its characteristics is described. Method is based on three search characteristics: word size, number of characters in word and frequency of particular characters. Method was presented in the 14th Conference for Lithuania Junior Researchers SCIENCE FOR FUTURE held in Vilnius, April 15, 2011. An experiment based on this study was made. Results show the effectiveness of the proposed method – 98% accuracy of obfuscated code detection in websites was reached. Experiment results were published in an article, which is being published in a reviewed periodical academic journal "Young Scientists". Structure: introduction, 6 chapters, conclusions and suggestions, references. Thesis consists of – 55 p. text without appendixes, 23 pictures, 4 tables, 44 bibliographical entries. Appendixes included.