Spam is an increasing menace in email: 90% of email is spam, and over 90% of spam is sent by botnets---networks of compromised computers under the control of miscreants. In this dissertation, we introduce email spam filtering using network-level features of spammers. Network-level features are based on lightweight measurements that can be made in the network, often without processing or storing a message. These features stay relevant for longer periods, are harder for criminals to alter at will (e.g., a bot cannot act independently of other bots in the botnet), and afford the unique opportunity to observe the coordinated behavior of spammers. We find that widely-used IP address-based reputation systems (e.g., IP blacklists) cannot keep up with the threats of spam from previously unseen IP addresses, and from new and stealthy attacks---to thwart IP-based reputation systems, spammers are reconnoitering IP Blacklists and sending spam from hijacked IP address space. Finally, spammers are "gaming" collaborative filtering by users in Web-based email by casting fraudulent "Not Spam" votes on spam email. We present three systems that detect each attack that uses spammer behavior rather than their IP address. First, we present IP blacklist counter-intelligence, a system that can passively enumerate spammers performing IP blacklist reconnaissance. Second, we present SpamTracker, a system that distinguishes spammers from legitimate senders by applying clustering on the set of domains to which email is sent. Third, we analyze vote-gaming attacks in large Web-based email systems that pollutes user feedback on spam emails, and present an efficient clustering-based method to mitigate such attacks.

Universiẗat, Diss.--Wien.

With the amount of email spam received these days it is hard to imagine that spammers act individually. Nowadays, most of the spam emails have been sent from a collection of compromised machines controlled by some spammers. These compromised computers are often called bots, using which the spammers can send massive volume of spam within a short period of time. The motivation of this work is to understand and analyze the behavior of spammers through a large collection of spam mails. My research examined a the data set collected over a 2.5-year period and developed an algorithm which would give the botnet features and then classify them into various groups. Principal component analysis was used to study the association patterns of group of spammers and the individual behavior of a spammer in a given domain. This is based on the features which capture maximum variance of information we have clustered. Presence information is a growing tool towards more efficient communication and providing new services and features within a business setting and much more. The main contribution in my thesis is to propose the willingness estimator that can estimate the callee's willingness without his/her involvement, the model estimates willingness level based on call history. Finally, the accuracy of the proposed willingness estimator is validated with the actual call logs.

Unsolicited e-mails generally present a major problem within the e-mail communication nowadays. There exist several methods that can detect spam and distinguish it from the requested messages. The theoretical part of the masters thesis introduces the ways of detecting unsolicited messages by using artificial immune systems. It presents and subsequently analyses several methods of the artificial immune systems that can assist in the fight against spam. The practical part of the masters thesis deals with the implementation of a spam filter on the basis of the artificial immune systems. The project ends with comparison of effectiveness of the newly designed spam filter and the one which uses common methods for spam detection.

A dissertation submitted in partial ful llment of the requirements for the degree Master of Science. School of Computer Science and Applied Mathematics, Faculty of Science, University of the Witwatersrand, Johannesburg. November 2017 / In the cyberspace, spam emails are used as a way to divulge sensitive information of victims through social engineering. There are various classi cation systems that have been employed previously to identify spam emails. The primary objective of email spam classi cation systems is to classify incoming email as either legitimate (non-spam) or spam emails. The spam classi cation task can thus be regarded as a two-class classi cation problem. This kind of a problem involves the use of various classi ers such as Decision Trees (DTs) and Support Vector Machines (SVMs). DTs and SVMs have been shown to perform well on email spam classi cation tasks. Several studies have failed to mention how these classi ers were optimized in terms of their hyperparameters. As a result, poor performance was encountered with complex datasets. This is because SVM classi er is dependent on the selection of the kernel function and the optimization of kernel hyperparameters. Additionally, many studies on spam email ltering task use words and characters to compute Term-Frequency (TF) based feature space. However, TF based feature space leads to sparse representation due to the continuous vocabulary growth. This problem is linked with the curse of dimensionality. Overcoming dimensionality issues involves the use of feature reduction techniques. Traditional feature reduction techniques, for instance, Information Gain (IG) may cause feature representations to lose important features for identifying spam emails. This proposed study demonstrates the use of Distributed Memory (DM), Distributed Bag of Words (DBOW), Cosine Similarity (CS) and Autoencoder for feature representation to retain a better class separability. Generated features enable classi ers to identify spam emails in a lower dimension feature space. The use of the Autoencoder for feature reduction led to improved classi cation performance. Furthermore, a comparison of kernel functions and CS measure is taken into consideration to evaluate their impacts on classi ers when employed for feature transformation. The study further shows that removal of more frequent words, which have been regarded as noisy words and stemming process, may negatively a ect the performance of the classi ers when word order is taken into consideration. In addition, this study investigates the performance of DTs and SVM classi ers on the publicly available datasets. This study makes a further investigation on the selection of optimal kernel function and optimization of kernel hyperparameters for each feature representation. It is further investigated whether the use of Stacked Autoencoder as a pre-processing step for multilayer perceptron (MLP) will lead to improved classi cation results. / MT 2018

Thesis (M.S.)--Brigham Young University. Dept. of Computer Science, 2006. / Includes bibliographical references (p. 43-45).

Unsolicited e-mail has brought much annoyance to users, thus, making e-mail less reliable as a communication tool. This has happened because current email architecture has key limitations. For instance, while it allows senders to send as many messages as they want, it does not provide adequate capability to recipients to prevent unrestricted access to their mailbox. This research develops a new approach to equip recipients with ability to control access to their mailbox.<p>This thesis builds an ownership-based approach to control mailbox usage employing the CyberOrgs model. CyberOrgs is a model that provides facilities to control resources in multi-agent systems. We consider a mailbox to be a precious resource of its owner. Any access to the resource requires its owner's permission. Thus, we give recipients a capability to manage their valuable resource - mailbox. In our approach, message senders obtain a permission to send messages through negotiation. In this negotiation, a sender makes a proposal and the intended recipient evaluates the proposal according to their own policies. A sender's desired outcome of a negotiation is a contract, which conducts the subsequent communication between the sender and the recipient. Contracts help senders and recipients construct a long-term relationship.<p>Besides allowing individuals to control their mailbox, we consider groups, which represent organizations in human society, in order to allow organizations to manage their resources including mailboxes, message sending allowances, and contracts.<p>A prototype based on our approach is implemented. In the prototype, policies are separated from the mechanisms. Examples of policies are presented and a public policy interface is exposed to allow programmers to develop custom policies. Experimental results demonstrate that the system performance is policy-dependent. In other words, as long as policies are carefully designed, communication involving negotiation has minimal overhead compared to communication in which senders deliver messages to recipients directly.

This thesis investigates ways to reduce or eliminate the necessity of user input to learning-based personal email spam filters. Personal spam filters have been shown in previous studies to yield superior effectiveness, at the cost of requiring extensive user training which may be burdensome or impossible. This work describes new approaches to solve the problem of building a personal spam filter that requires minimal user feedback. An initial study investigates how well a personal filter can learn from different sources of data, as opposed to user’s messages. Our initial studies show that inter-user training yields substantially inferior results to intra-user training using the best known methods. Moreover, contrary to previous literature, it is found that transfer learning degrades the performance of spam filters when the source of training and test sets belong to two different users or different times. We also adapt and modify a graph-based semi-supervising learning algorithm to build a filter that can classify an entire inbox trained on twenty or fewer user judgments. Our experiments show that this approach compares well with previous techniques when trained on as few as two training examples. We also present the toolkit we developed to perform privacy-preserving user studies on spam filters. This toolkit allows researchers to evaluate any spam filter that conforms to a standard interface defined by TREC, on real users’ email boxes. Researchers have access only to the TREC-style result file, and not to any content of a user’s email stream. To eliminate the necessity of feedback from the user, we build a personal autonomous filter that learns exclusively on the result of a global spam filter. Our laboratory experiments show that learning filters with no user input can substantially improve the results of open-source and industry-leading commercial filters that employ no user-specific training. We use our toolkit to validate the performance of the autonomous filter in a user study.

Unsolicited e-mail has brought much annoyance to users, thus, making e-mail less reliable as a communication tool. This has happened because current email architecture has key limitations. For instance, while it allows senders to send as many messages as they want, it does not provide adequate capability to recipients to prevent unrestricted access to their mailbox. This research develops a new approach to equip recipients with ability to control access to their mailbox.<p>This thesis builds an ownership-based approach to control mailbox usage employing the CyberOrgs model. CyberOrgs is a model that provides facilities to control resources in multi-agent systems. We consider a mailbox to be a precious resource of its owner. Any access to the resource requires its owner's permission. Thus, we give recipients a capability to manage their valuable resource - mailbox. In our approach, message senders obtain a permission to send messages through negotiation. In this negotiation, a sender makes a proposal and the intended recipient evaluates the proposal according to their own policies. A sender's desired outcome of a negotiation is a contract, which conducts the subsequent communication between the sender and the recipient. Contracts help senders and recipients construct a long-term relationship.<p>Besides allowing individuals to control their mailbox, we consider groups, which represent organizations in human society, in order to allow organizations to manage their resources including mailboxes, message sending allowances, and contracts.<p>A prototype based on our approach is implemented. In the prototype, policies are separated from the mechanisms. Examples of policies are presented and a public policy interface is exposed to allow programmers to develop custom policies. Experimental results demonstrate that the system performance is policy-dependent. In other words, as long as policies are carefully designed, communication involving negotiation has minimal overhead compared to communication in which senders deliver messages to recipients directly.

The spread of spam mails have become a serious threat in the Internet. In addition to commercial messages, some malicious messages such as phishing, pornography messages, fraudulent messages and malicious codes are spread via spam. A practical server-side anti-spam system should have ability to (1) filter out growing volume of spam mails correctly; (2) recognize new type of spam mails and (3) manage the increasing spam rules automatically. Most work only focused on single aspect (especially for spam rule generation) to prevent spam mail. However, in real world, spam prevention is not just applying data mining algorithm for rule generation. To filter out spam mails correctly and efficiently in a real world, there are still many issues should be considered in addition to spam rule generation. In this research, we propose and integrate three sub-systems to form a practical anti-spam system, the sub-systems are spam rule generation sub-system, spam rule sharing sub-system and spam rule management sub-system. In this research, rule-based data mining approach is used to generate manageable and shareable spam rules. The latest spam rules are shared through machine-readable XML format. Spam rules stored in mail servers are managed based on statistical testing approach. The Rule management sub-system can automatically enable high performance rules and disable out-of-date rules to improve the miss rate and efficiency of spam filter. This research will develop and integrate the three sub-systems to achieve the goal of spam prevention.