A Methodology for Detecting and Classifying Rootkit Exploits

A Methodology for Detecting and Classifying Rootkit Exploits
John G. Levine
164 Pages
Directed by Dr. Henry L. Owen
We propose a methodology to detect and classify rootkit exploits. The goal of this research is to provide system administrators, researchers, and security personnel with the information necessary in order to take the best possible recovery actions concerning systems that are compromised by rootkits. There is no such methodolgoy available at present to perform this function. This may also help to detect and fingerprint additional instances and prevent further security instances involving rootkits. A formal framework was developed in order to define rootkit exploits as an existing rootkit, a modification to an exisiting, or an entirely new rootkit. A methodology was then described in order to apply this framework against rootkits that are to be investigated. We then proposed some new methods to detect and characterize specific types of rootkit exploits. These methods consisted of identifying unique string signatures of binary executable files as well as examining the system call table within the system kernel. We established a Honeynet in order to aid in our research efforts and then applied our methodology to a previously unseen rootkit that was targeted against the Honeynet. By using our methodology we were able to uniquely characterize this rootkit and identify some unique signatures that could be used in the detection of this specific rootkit. We applied our methodolgy against nine additional rootkit exploits and were were able to identify unique characterstics for each of these rootkits. These charactersitics could also be used in the prevention and detection of these rootkits.

Identiferoai:union.ndltd.org:GATECH/oai:smartech.gatech.edu:1853/5139
Date18 March 2004
CreatorsLevine, John G. (John Glenn)
PublisherGeorgia Institute of Technology
Source SetsGeorgia Tech Electronic Thesis and Dissertation Archive
Languageen_US
Detected LanguageEnglish
TypeDissertation
Format1180304 bytes, application/pdf

Page generated in 0.002 seconds