The detection of similar code can support many so ware engineering tasks such as program understanding and program classification. Many excellent approaches have been proposed to detect programs having similar syntactic features. However, these approaches are unable to identify programs dynamically or statistically close to each other, which we call behaviorally similar programs. We believe the detection of behaviorally similar programs can enhance or even automate the tasks relevant to program classification. In this thesis, we will discuss our current approaches to identify programs having similar behavioral features in multiple perspectives. We first discuss how to detect programs having similar functionality. While the definition of a program’s functionality is undecidable, we use inputs and outputs (I/Os) of programs as the proxy of their functionality. We then use I/Os of programs as a behavioral feature to detect which programs are functionally similar: two programs are functionally similar if they share similar inputs and outputs. This approach has been studied and developed in the C language to detect functionally equivalent programs having equivalent I/Os. Nevertheless, some natural problems in Object Oriented languages, such as input generation and comparisons between application-specific data types, hinder the development of this approach. We propose a new technique, in-vivo detection, which uses existing and meaningful inputs to drive applications systematically and then applies a novel similarity model considering both inputs and outputs of programs, to detect functionally similar programs. We develop the tool, HitoshiIO, based on our in-vivo detection. In the subjects that we study, HitoshiIO correctly detect 68.4% of functionally similar programs, where its false positive rate is only 16.6%. In addition to functional I/Os of programs, we attempt to discover programs having similar execution behavior. Again, the execution behavior of a program can be undecidable, so we use instructions executed at run-time as a behavioral feature of a program. We create DyCLINK, which observes program executions and encodes them in dynamic instruction graphs. A vertex in a dynamic instruction graph is an instruction and an edge is a type of dependency between two instructions. The problem to detect which programs have similar executions can then be reduced to a problem of solving inexact graph isomorphism. We propose a link analysis based algorithm, LinkSub, which vectorizes each dynamic instruction graph by the importance of every instruction, to solve this graph isomorphism problem efficiently. In a K Nearest Neighbor (KNN) based program classification experiment, DyCLINK achieves 90 + % precision. Because HitoshiIO and DyCLINK both rely on dynamic analysis to expose program behavior, they have better capability to locate and search for behaviorally similar programs than traditional static analysis tools. However, they suffer from some common problems of dynamic analysis, such as input generation and run-time overhead. These problems may make our approaches challenging to scale. Thus, we create the system, Macneto, which integrates static analysis with machine topic modeling and deep learning to approximate program behaviors from their binaries without truly executing programs. In our deobfuscation experiments considering two commercial obfuscators that alter lexical information and syntax in programs, Macneto achieves 90 + % precision, where the groundtruth is that the behavior of a program before and after obfuscation should be the same. In this thesis, we offer a more extensive view of similar programs than the traditional definitions. While the traditional definitions of similar programs mostly use static features, such as syntax and lexical information, we propose to leverage the power of dynamic analysis and machine learning models to trace/collect behavioral features of pro- grams. These behavioral features of programs can then apply to detect behaviorally similar programs. We believe the techniques we invented in this thesis to detect behaviorally similar programs can improve the development of software engineering and security applications, such as code search and deobfuscation.

<p> </p><p>In this paper I present the domain specific language HaGPipe for graphics programming in Haskell. HaGPipe has a clean, purely functional and strongly typed interface and targets the whole graphics pipeline including the programmable shaders of the GPU. It can be extended for use with various backends and this paper provides two different ones. The first one generates vertex and fragment shaders in Cg for the GPU, and the second one generates vertex shader code for the SPUs on PlayStation 3. I will demonstrate HaGPipe's many capabilities of producing optimized code, including an extensible rewrite rule framework, automatic packing of vertex data, common sub expression elimination and both automatic basic block level vectorization and loop vectorization through the use of structures of arrays.</p>

The benefits of programming in a functional style are well known. In particular, algorithms that are expressed as compositions of functions operating on series/vectors/streams of data elements are much easier to understand and modify than equivalent algorithms expressed as loops. Unfortunately, many programmers hesitate to use series expressions, because they are typically implemented very inefficiently---the prime source of inefficiency being the creation of intermediate series objects. A restricted class of series expressions, obviously synchronizable series expressions, is defined which can be evaluated very efficiently. At the cost of introducing restrictions which place modest limits on the series expressions which can be written, the restrictions guarantee that the creation of intermediate series objects is never necessary. This makes it possible to automatically convert obviously synchronizable series expressions into highly efficient loops using straight forward algorithms.

M.S. / Computer Science & Engineering / The Graph Reduction Machine (G-Machine) is an architecture intended to achieve high performance in executing functional language programs. The success or failure of this novel architecture can only be determined by its performance in executing "real" programs. The simulator of the G-Machine, described in this thesis, makes possible detailed studies of the performance of the G-Machine architecture even though the hardware implementation of a G-Machine is not complete.

Interpretation is an implicit part of today’s programming; it has great power but is overused and has significant costs. For example, interpreters are typically significantly hard to understand and hard to reason about. The methodology of “Totally Functional Programming” (TFP) is a reasoned attempt to redress the problem of interpretation. It incorporates an awareness of the undesirability of interpretation with observations that definitions and a certain style of programming appear to offer alternatives to it. Application of TFP is expected to lead to a number of significant outcomes, theoretical as well as practical. Primary among these are novel programming languages to lessen or eliminate the use of interpretation in programming, leading to better-quality software. However, TFP contains a number of lacunae in its current formulation, which hinder development of these outcomes. Among others, formal semantics and type-systems for TFP languages are yet to be discovered, the means to reduce interpretation in programs is to be determined, and a detailed explication is needed of interpretation, definition, and the differences between the two. Most important of all however is the need to develop a complete understanding of the nature of interpretation. In this work, suitable type-systems for TFP languages are identified, and guidance given regarding the construction of appropriate formal semantics. Techniques, based around the ‘fold’ operator, are identified and developed for modifying programs so as to reduce the amount of interpretation they contain. Interpretation as a means of language-extension is also investigated. Finally, the nature of interpretation is considered. Numerous hypotheses relating to it considered in detail. Combining the results of those analyses with discoveries from elsewhere in this work leads to the proposal that interpretation is not, in fact, symbol-based computation, but is in fact something more fundamental: computation that varies with input. We discuss in detail various implications of this characterisation, including its practical application. An often more-useful property, ‘inherent interpretiveness’, is also motivated and discussed in depth. Overall, our inquiries act to give conceptual and theoretical foundations for practical TFP.

No description available.

No description available.

No description available.

No description available.

No description available.