In transactional systems development literature, maintenance is reported as being a phase in the software development life cycle. In practice, this phase is often neglected as it occurs post-deployment and other ongoing projects take a higher priority. In data warehouse (DW) systems development literature, maintenance is not reported as being a phase but an ongoing iteration to the DW development project. It should therefore not be treated as a phase by DW systems professionals. Although there is this fundamental difference in the approach to maintenance, transaction systems maintenance and DW maintenance share many of the same challenges. DW literature and methodologies inherently contain utilities and methods to assist in alleviating these challenges in a DW system. Transactional systems do not deal with these challenges inherently. Research aspects were extracted from the literature review conducted. The literature review conducted demonstrates what the challenges in maintenance are, how the challenges of transactional systems compare to the challenges of DW maintenance and how the utilities and methods used in DW methodologies can inherently assist in managing these challenges from DW perspective. These research aspects were used to formulate an interpretive questionnaire. This research portion of the study explores the use of DW systems development and maintenance methodologies in the industry among DW professionals. This is done by conducting an interpretive study using the interpretive questionnaire developed from the literature review. The interpretive questionnaire focusses on maintenance and dealing with the challenges thereof. Many themes evolved from the analysis of the interpretive study by using the content analysis method. The final conclusions of this study is drawn by comparing and combining the information gathered from the literature review with the information gathered from the interpretive study. Gaps are identified between practice and literature and recommendations are made based on these gaps. / MSc (Computer Science), North-West University, Potchefstroom Campus, 2015

In transactional systems development literature, maintenance is reported as being a phase in the software development life cycle. In practice, this phase is often neglected as it occurs post-deployment and other ongoing projects take a higher priority. In data warehouse (DW) systems development literature, maintenance is not reported as being a phase but an ongoing iteration to the DW development project. It should therefore not be treated as a phase by DW systems professionals. Although there is this fundamental difference in the approach to maintenance, transaction systems maintenance and DW maintenance share many of the same challenges. DW literature and methodologies inherently contain utilities and methods to assist in alleviating these challenges in a DW system. Transactional systems do not deal with these challenges inherently. Research aspects were extracted from the literature review conducted. The literature review conducted demonstrates what the challenges in maintenance are, how the challenges of transactional systems compare to the challenges of DW maintenance and how the utilities and methods used in DW methodologies can inherently assist in managing these challenges from DW perspective. These research aspects were used to formulate an interpretive questionnaire. This research portion of the study explores the use of DW systems development and maintenance methodologies in the industry among DW professionals. This is done by conducting an interpretive study using the interpretive questionnaire developed from the literature review. The interpretive questionnaire focusses on maintenance and dealing with the challenges thereof. Many themes evolved from the analysis of the interpretive study by using the content analysis method. The final conclusions of this study is drawn by comparing and combining the information gathered from the literature review with the information gathered from the interpretive study. Gaps are identified between practice and literature and recommendations are made based on these gaps. / MSc (Computer Science), North-West University, Potchefstroom Campus, 2015

Data warehousing is becoming an increasingly important technology for information integration and data analysis. Given the dynamic nature of modern distributed environments, both source data updates and schema changes are likely to occur autonomously and even concurrently in different data sources. Current approaches to maintain a data warehouse in such dynamic environments sequentially schedule maintenance processes to occur in isolation. Furthermore, each maintenance process is handling the maintenance of one single source update. This limits the performance of current data warehouse maintenance systems in a distributed environment where the maintenance of source updates endures the overhead of network delay as well as IO costs for each maintenance query. In this thesis work, we propose two different optimization strategies which can greatly improve data warehouse maintenance performance for a set of source updates in such dynamic environments. Both strategies are able to support source data updates and schema changes. The first strategy, the parallel data warehouse maintainer, schedules multiple maintenance processes concurrently. Based on the DWMS_Transaction model, we formalize the constraints that exist in maintaining data and schema changes concurrently and propose several parallel maintenance process schedulers. The second strategy, the batch data warehouse maintainer, groups multiple source updates and then maintains them within one maintenance process. We propose a technique for compacting the initial sequence of updates, and then for generating delta changes for each source. We also propose an algorithm to adapt/maintain the data warehouse extent using these delta changes. A further optimization of the algorithm also is applied using shared queries in the maintenance process. We have designed and implemented both optimization strategies and incorporated them into the existing DyDa/TxnWrap system. We have conducted extensive experiments on both the parallel as well as the batch processing of a set of source updates to study the performance achievable under various system settings. Our findings include that our parallel maintenance gains around 40 ~ 50% performance improvement compared to sequential processing in environments that use single-CPU machines and little network delay, i.e, without requiring any additional hardware resources. While for batch processing, an improvement of 400 ~ 500% improvement compared with sequential maintenance is achieved, however at the cost of less frequent refreshes of the data warehouse content.