Researchers and practitioners alike have long sought to integrate programming languages and databases. Today's integration solutions focus on the data-types of the two domains, but today's programs lack transparency. A transparently persistent program operates over all objects in a uniform manner, regardless of whether those objects reside in memory or in a database. Transparency increases modularity and lowers the barrier of adoption in industry. Unfortunately, fully transparent programs perform so poorly that no one writes them. The goal of this dissertation is to increase the performance of these programs to make transparent persistence a viable programming paradigm. This dissertation contributes two novel techniques that integrate programming languages and databases. Our first contribution--called query extraction--is based purely on program analysis. Query extraction analyzes a transparent, object-oriented program that retrieves and filters collections of objects. Some of these objects may be persistent, in which case the program contains implicit queries of persistent data. Our interprocedural program analysis extracts these queries from the program, translates them to explicit queries, and transforms the transparent program into an equivalent one that contains the explicit queries. Query extraction enables programmers to write programs in a familiar, modular style and to rely on the compiler to transform their program into one that performs well. Our second contribution--called RBI-DB+--is an extension of a new programming language construct called a batch block. A batch block provides a syntactic barrier around transparent code. It also provides a latency guarantee: If the batch block compiles, then the code that appears in it requires only one client-server communication trip. Researchers previously have proposed batch blocks for databases. However, batch blocks cannot be modularized or composed, and database batch blocks do not permit programmers to modify persistent data. We extend database batch blocks to address these concerns and formalize the results. Today's technologies integrate the data-types of programming languages and databases, but they discourage programmers from using procedural abstraction. Our contributions restore procedural abstraction's use in enterprise applications, without sacrificing performance. We argue that industry should combine our contributions with data-type integration. The result would be a robust, practical integration of programming languages and databases. / text

Many enterprise applications persist data beyond their lifetimes, usually in a database management system. Orthogonal persistence provides a clean programming model for communicating with databases. A program using orthogonal persistence operates over persistent and non-persistent data uniformly. However, a straightforward implementation of orthogonal persistence results in a large number of small queries each of which incurs a large overhead when accessing a remote database. In addition, the program cannot take advantage of a database's query optimizations for large and complex queries. Instead, most programs compose smaller queries into a single large query explicitly and send the query to the database through a command-level interface. These explicit queries compromise the modularity of programs because they do not compose well and they contain information about the program's future data access patterns. Consequently, programs with explicit queries are harder to maintain and reason about. In this thesis, we first define transparent persistence, a relaxation of orthogonal persistence. We show how transparent persistence in current tools can be made more practical by developing AutoFetch. The key idea in AutoFetch is to dynamically observe a program's data access patterns and use that information to reduce the number of queries. While AutoFetch is constrained by existing Java technology and tools, Remote Batch Invocation (RBI) adds the batch statement to the Java language. The batch statement is a general purpose mechanism for optimizing distributed communication using batching. RBI-DB specializes the ideas in RBI for databases. Both of these ideas help bridge the performance gap between orthogonally persistent systems and traditional database interfaces. / text