Providing service via theWeb differs from other service provision environments in that it is possible for the unexpected arrival of a massive number of service requests in a small time-frame, a situation commonly referred to as a flash crowd. Events of this nature are beyond the control of the service provider, and have the potential to severely degrade service quality and, in the worst case, to deny service to all clients completely. The occurrence, severity and sought Web content of a flash crowd is beyond the control of service provision software. How this software reacts to such a flash crowd, however, is not. Given the short-lived nature of flash crowds, it is unreasonable to expect such systems to increase the system resources they can apply to a particular flash crowd event. It is also difficult to predict the particular nature of any flash crowd, and subsequently which system resources will bottleneck. The driving hypothesis of this research is that, if we are to reasonably expect to have software react effectively to flash crowd events, we need to alter that software at runtime to remove system bottlenecks, whilst a flash crowd event is in progress. This is a special case of what is usually known as "unanticipated software evolution". This thesis reports on an investigation into how unanticipated software evolution can be applied to running Web service provision software to remove system bottlenecks. It does so by introducing automated dynamic Web content degradation to running software currently subject to simulated flash crowd events. The thesis describes and validates appropriate runtime extensions to allow generative object communication architectures (a promising class of architecture for unanticipated software evolution) to be converted initially into a Web application server, and then later accept further runtime behaviour changes. Such changes could alter system bottlenecks by replacing the key programming logic causing system bottlenecks at runtime.

Advantages of cache cooperation on edge cache networks serving dynamic web content were studied. Design of cooperative edge cache grid a large-scale cooperative edge cache network for delivering highly dynamic web content with varying server update frequencies was presented. A cache clouds-based architecture was proposed to promote low-cost cache cooperation in cooperative edge cache grid. An Internet landmarks-based scheme, called selective landmarks-based server-distance sensitive clustering scheme, for grouping edge caches into cooperative clouds was presented. Dynamic hashing technique for efficient, load-balanced, and reliable documents lookups and updates was presented. Utility-based scheme for cooperative document placement in cache clouds was proposed. The proposed architecture and techniques were evaluated through trace-based simulations using both real-world and synthetic traces. Results showed that the proposed techniques provide significant performance benefits. A framework for automatically detecting cache-effective fragments in dynamic web pages was presented. Two types of fragments in web pages, namely, shared fragments and lifetime-personalization fragments were identified and formally defined. A hierarchical fragment-aware web page model called the augmented-fragment tree model was proposed. An efficient algorithm to detect maximal fragments that are shared among multiple documents was proposed. A practical algorithm for detecting fragments based on their lifetime and personalization characteristics was designed. The proposed framework and algorithms were evaluated through experiments on real web sites. The effect of adopting the detected fragments on web-caches and origin-servers is experimentally studied.