University of Southampton
An architecture for management of large, distributed, scientific data
Abstract
dc:description.abstract<p>This thesis describes research into Web-based management of non-traditional data. Three prototype systems are discussed, <i>GBIS, DBbrowse</i> and <i>EASIA</i>, each of which provided examples of new ideas in this area.</p><p>In 1999 concepts from GBIS and DBbrowse were used as the starting point for examining new architectures for archiving scientific datasets. Data from numerical simulations generated by the UK Turbulence Consortium was used as a case study. Due to the large datasets produced, new Web-based mechanisms were required for storage, searching, retrieval and manipulation of simulation results in the hundreds of gigabytes range. A prototype architecture and user interface, EASIA (<i>Extensible Architecture for Scientific Data Archives</i>) [182] [183] is described. EASIA demonstrates several new concepts of active digital libraries of scientific data. Result files are archived in-place thereby avoiding costs associated with transmitting results in a centralised site. The method used shows that a database can meet the apparently divergent requirements of storing both the relatively small simulation result metadata, and the large, distributed result files, in a unified, secure way. EASIA also shows that separation of user interface specification from user interface processing can simplify the extensibility of such systems. EASIA archives not only data in a distributed fashion, but also applications. These are loosely coupled to the archived datasets via a user interface specification file that uses a vocabulary defined by a markup language. Archived applications can provide reusable dynamic server-side post-processing operations. This can reduce bandwidth requirements for request data through server-side data reduction. The archive allows post-processing to be performed directly without the cost of having to rematerialise to files, and it also reduces access bottlenecks and processor loading at individual sites.</p>
Degree
thesis:*- Name dc:type.qualificationname
- Ph.D.
- Level dc:type.qualificationlevel
- doctoral
- Grantor dc:publisher.institution
- University of Southampton
- Year dc:date.issued
- 2000
Author and committee
dc:creator, dc:contributor.*- Author dc:creator
-
- Papiani, Mark
Rights
- Language dc:language
- English, eng