This paper deals with the possibilities for load balancing and clustering of an existing server for speech processing. The paper analyzes problems of load balancing and clustering. There are also described the concepts of network programming and options for I/O processing. A new design of a load balancer is created, fully customized for the needs of speech processing server. This newly designed load balancer is implemented and thoroughly tested.

Existing database drivers use blocking socket I/O to exchange data with relational database management systems (RDBMS). To concurrently send multiple requests to a RDBMS with blocking database drivers, a separate thread must be used for each request. This approach has been used successfully for many years. However, we propose that using non-blocking socket I/O is faster and scales better under load. In this paper we introduce the Asynchronous Database Connectivity in Java (ADBCJ) framework. ADBCJ provides a common API for asynchronous RDBMS interaction. Various implementations of the ADBCJ API are used to show how utilizing non-blocking socket I/O is faster and scales better than using conventional database drivers and multiple threads for concurrency. Our experiments show a significant performance increase when using non- blocking socket I/O for asynchronous RDBMS access while using a minimal number of OS threads. Non-blocking socket I/O enables the ability to pipeline RDBMS requests which can improve performance significantly, especially over high latency networks. We also show the benefits of asynchronous database drivers on different web server architectures.