Thesis (MPhil)--Stellenbosch University, 2015. / ENGLISH ABSTRACT: Information System Success Evaluation has been a concept of interest to many organizations for decades now. The main motivation for this activity is usually from the financial view of things, that is, the main interest of an organization carrying out IS success evaluation is mainly to find out whether investing in such a system was a good business decision. This question is often answered by the realization of the direct and indirect benefits realized by deployment of using such a system. Like many organizations the government of Botswana has invested millions of Pula’s in the past number of years in acquiring and implementing different Information Systems in the various Ministries and Departments. One such system is the Vehicle Registration and Licensing System, which was deployed by the Department of Roads Transport and Safety of the Ministry of Works, Transport and Communication. After the VRLS implementation there has not been any formal scientific research or investigation conducted to find out the success of this Information System, especially with regards to the stakeholders who use this system for their daily jobs. This study therefore engaged in a task, which evaluated the success of the Vehicle Registration and Licensing System and answered the question of how effective the system is from the user’s point of view. A well-known Information System success evaluation model, known as the Delone and McLean IS evaluation model, was applied to test the hypothesis of this research. A structured questionnaire was used to collect the research data. The results were analyzed using a common research tool, the SPSS, including cross tabulation and chisquare tests. The findings of this study will further enrich the IS success evaluation Body of Knowledge on the key factors which can contribute to a successful or unsuccessful Information System acquisition investment. Furthermore, the researchers who will further look into this subject may find these findings very useful and also stimulating to do further research on this subject to unveil further evidence of the ingredients of an effective Information System investment which has been accepted by and accredited by its users. / AFRIKAANSE OPSOMMING: Inligtingstelselsuksesevaluering is ‘n konsep wat baie organisasies nou al vir dekades interesseer. Die motivering hiervoor is hoofsaaklik geleë in finansiële redes. Organisasies wil hiermee vasstel of dit ‘n goeie besigheidsbesluit was. Hierdie vraag word dan dikwels beantwoord deur die verkryging van direkte en indirekte voordele deur gebruik te maak van so ‘n stelsel. Soos baie organisasies het die regering van Botswana miljoene pula belê in die implementering van inligtingstelsels in verskeie staatsdepartemente. Die Voertuigregistrasie en Lisensiestelsel is ‘n voorbeeld hiervan wat deur die Departement van Padvervoer en -Veiligheid van die Ministerie van Werke, Vervoer en Kommunikasie ontplooi is. Geen formele wetenskaplike navorsing is geloods na afloop van die implementering van die stelsel nie, veral nie sedert die gebruik deur motoriste nie. Hierdie studie poog juis om die effektiwiteit van die stelsel te meet, veral vanuit die verbruiker se perspektief. Die Delone en Mclean Inligtingstelselevalueringsmodel word gebruik om die hipotese van die navorsing te toets. ‘n Gestruktureerde vraelys is opgestel om data te versamel. Die SPSS as ‘n algemene navorsingsinstrument is gebruik vir analise, asook vir kruistabulasie en chikwadraattoetse. Die bevindinge van die studie sal die korpus van kennis van inligtingstelselsuksesevaluering verryk ten opsigte van die sleutelfaktore wat kan bydra tot die sukses of nie-sukses van inligtingstelselimplementering. Verdere studie in hierdie betrokke gebied word deur die navorsing gestimuleer, om veral die effektiwiteit van Inligtingstelselbeleggings te bepaal.

Despite numerous frameworks and models proposed in the literature, Information Systems (IS) assessment still remains elusive. In addition, little agreement exists on the contribution of the IS function within an organization and on how IS is related to the other organizational dimensions. Frameworks that show the relationship between IS and the organization are in the developmental stage and this work proposes a more comprehensive framework to assist in better understanding the relationship between IS and organizational quality. This research examines two popular IS quality assessment frameworks - Malcolm Baldrige National Quality Award (MBNQA) and Information Systems Assessment (ISA) - and suggests a new framework, IS-MBNQA. This work integrates these two IS quality assessment frameworks into a single comprehensive model that provides a holistic view on how IS quality is interrelated to organizational quality. The existing two IS assessment frameworks attempted to measure IS quality at different levels within an organization. The MBNQA model is the most comprehensive quality framework because it takes an organization wide perspective. On the other hand, ISA employs an IS specific perspective and reflects the relationships of eight major IS success dimensions. ISA is a modified version of DeLone & McLean's model with the inclusion of a success factor for Service Quality. For this study, survey instruments are developed from the MBNQA and ISA frameworks and they are consolidated to allow testing of the single IS-MBNQA framework. Exploratory factor analysis is performed for instrument refinement and confirmatory factor analysis for validity of the models. The instruments developed in this work are utilized as a foundation for identifying the relationships among the dimensions within and between each model. A major contribution of this work is the validation of the 2000 MBNQA model and the extension of existing models/frameworks to better explain the IS contribution to an organization.

The purpose of this research is to develop a comprehensive, IS assessment framework using existing IS assessment theory as a base and incorporating suggestions from other disciplines. To validate the framework and to begin the investigation of current IS assessment practice, a survey instrument was developed. A small group of subject matter experts evaluated and improved the instrument. The instrument was further evaluated using a small sample of IS representatives. Results of this research include a reexamination of the IS function measurement problem using new frameworks of analyses yielding (a) guidance for the IS manager or executive on which IS measures might best fit their organization, (b) a further verification of the important measures most widely used by IS executives, (c) a comprehensive, theoretically-derived, IS assessment framework, and by (d) the enhancement of IS assessment theory by incorporating ideas from actual practice. The body of knowledge gains a comprehensive, IS assessment framework that can be further tested for usefulness and applicability. Future research is recommended to substantiate and improve on these findings. Chapter 2 is a complete survey of prior research, subdivided by relevant literature divisions, such as organizational effectiveness, quality management, and IS assessment. Chapter 3 includes development of and support for the research questions, IS assessment framework, and the research model. Chapter 4 describes how the research was conducted. It includes a brief justification for the research approach, a description of how the framework was evaluated, a description of how the survey instrument was developed and evaluated, a description of the participants and how they were selected, a synopsis of the data collection procedures, a brief description of follow-up procedures, and a summary. Chapter 5 presents the results of the research. Chapter 6 is a summary and conclusion of the research. Finally, included in the appendices are definitions of terms, and copies of the original and improved survey instruments.

This thesis explores the problem of information systems evaluation by conceptualising it as a process in which the manager comes to an understanding about a system. In other words, information systems evaluation is a hermeneutic process. The thesis explicates this notion through an argument that is itself hermeneutic in its development, beginning with the mainstream functionalist view of information systems evaluation, and then considering an interpretive view of IS evaluation, each of which points to one of two stereotypes of IS evaluation and the manager engaged in this process: the objective/rational manager utilising objective/rational methods versus the subjective/political manager engaged in political manoeuvring, utilising objective/rational methods only as ritual or symbolism. Neither of these opposing stereotypes is satisfactory. Instead, this thesis proposes a dialectic view of information systems evaluation, in terms of which, rather than being a decision maker, the manager is in-the-world, evaluating systems in order to get the job done, on the basis of her thrownness in-the-world. This conceptualisation provides an intuitively appropriate account of evaluation on the part of an individual manager, but we must still consider how managers as members of the organisation, reach a common understanding about a system. This they do through a process of organisational learning as encultured knowing, in terms of which a narrative, situated, pragmatic knowledge is most useful in evaluation. Evaluation, in other words, happens in the course of skilful conversation. Such conversation is, however, not always skilful because the organisation is not just a collection of individuals but also a network of power relations. Conversations as generators of meaning are never held outside of power: systems evaluations as conversations cannot take place outside of a regime of truth. A post-dualist view of action as both constituted by and constituting structure, however, suggests that there is always the potential for genuinely hermeneutic and ethical conversation, provided it is both improvisatory and deconstructive. Having understood the requirement for improvisation and deconstruction, it is possible to suggest some heuristics for information systems evaluation based on these ideas. / Dissertation (Phd (Information Technology))--University of Pretoria, 2003. / Informatics / unrestricted

If knowledge has been gathered by the practitioners’ community in the area of health data warehousing evaluation, it is mostly relying on anecdotal evidence instead of academic research. Isolated dimensions have received more attention and benefit from definitions and performance measures. However, very few cases can be found in the literature which describe how the assessment of the technology can be made, and these cases do not provide insight on how to systematize such assessment. The research in this dissertation is aimed at bridging this knowledge gap by developing an evaluation framework, and conducting an empirical study to further investigate the state of health data warehousing evaluation and the use of the technology to improve healthcare efficiency, as well as to compare these findings with the proposed framework. The empirical study involved an exploratory approach and used a qualitative method, i.e. audio-taped semi-structured interviews. The interviews were conducted in collaboration with the Healthcare Data Warehousing Association and involved 21 participants who were members of the Association working in a mid- to upper-level management capacity on the development and implementation of health data warehousing. All audio-taped interviews were transcribed and transcripts were coded using a qualitative analysis software package (NVivo, QSR International). Results were obtained in three areas. First, the study established that current health data warehousing systems are typically not formally evaluated. Systematic assessments relying on predetermined indicators and commonly accepted evaluation methods are very seldom performed and Critical Success Factors are not used as a reference to guide the system’s evaluation. This finding appears to explain why a literature review on the topic returns so few publications. Second, from patient throughput to productivity tracking and cost optimization, the study provided evidence of the contribution of data warehousing to the improvement of healthcare systems’ efficiency. Multiple examples were given by participants to illustrate the ways in which the technology contributed to streamlining the care process and increase healthcare efficiency in their respective organizations. Third, the study compared the proposed framework with current practices. Because formal evaluations were seldom performed, the empirical study offered limited feedback on the framework’s structure and rather informed its content and the assessment factors initially defined. / Graduate

Avaliação de sistemas de informação e frames tecnológicos (estruturas cognitivas compartilhadas em relação à tecnologia) constituem-se as temáticas focais desta pesquisa cujo objetivo foi compreender como se estabelece a relação entre processos de avaliação formais e informais de um sistema de informação e frames tecnológicos. A estrutura de avaliação conteúdo-contexto-processo e o arcabouço teórico de frames tecnológicos, especialmente seus elementos constitutivos - conhecimentos, pressupostos e expectativas - foram associados visando a esta compreensão. A pesquisa foi realizada em uma instituição de ensino superior e o objeto de estudo tratou-se de um sistema de informação acadêmico (Peoplesoft), implementado e em uso nesta instituição. Especificamente analisaram-se: avaliações formais e informais do sistema acadêmico em relação à estrutura conteúdo-contexto-processo; a natureza e extensão de diferenças em frames tecnológicos de grupos sociais distintos; e como frames tecnológicos moldam e são moldados por avaliações formais e informais de um sistema de informação. Com pressupostos epistemológicos sustentados pela perspectiva interpretativista, metodologicamente o estudo caracterizou-se por uma abordagem qualitativa do tipo estudo de caso único e incorporado (grupos sociais como sub-unidades de análise). Entrevistas em profundidade, observação participante, revisão documental e artefato físico constituíram-se as fontes de dados empíricos. Adotaram-se os fundamentos do método hermenêutico-dialético para a análise e interpretação dos dados coletados. O estudo propiciou identificar que: frames tecnológicos e avaliações do sistema de informação, sejam formais ou informais, influenciam-se reciprocamente; características pessoais, contextuais e tecnológicas integram-se criando, reforçando e modificando frames tecnológicos; diferentes avaliações sobre o sistema acadêmico são socialmente construídas pelos indivíduos inseridos em grupos sociais e fortemente afetadas pelo que acreditam, conhecem e esperam deste sistema; frames tecnológicos manifestam-se tanto no conteúdo da avaliação quanto no processo avaliativo, influenciando a identificação de fatores do conteúdo assim como o teor do julgamento destes fatores, o modo pelo qual se avalia e o próprio contexto da avaliação; em contrapartida, processos de avaliações informais, decorrentes da interação social, afetam frames tecnológicos; nestes processos, conhecimentos prévios são validados, ampliados ou refutados, pressupostos e expectativas criados e recriados; igualmente, o modo como avaliações formais são configuradas desencadeia conhecimentos, pressupostos e expectativas sobre a tecnologia. / Information systems evaluation and technological frames (shared cognitive structures that relate to technology) are the main themes of this research. Its general goal has been to establish how the relationship between formal and informal processes of information systems evaluation and technological frames operates. The structure of evaluation (content-context-process) and the theoretical stance of the technological frames, especially in respect to its constitutive elements, have been analyzed and linked to each other. The research has been conducted in a graduate institution and its object of analysis, a software called \"Peoplesoft\", which was already implemented and that is in use nowadays. Specific analysis deal with the following topics: a) both formal and informal evaluations of the information system in relation to the \"content-context-process\" structure; the nature and extension of the differences that sustain technological frames for different groups as well how these technological frames are molded and formed by formal and informal assessment (evaluations) of such system. Epistemological suppositions were held by an interpretative perspective while, methodologically, this study is characterized by a qualitative perspective of a unique case in which social groups act as sub-units of analysis. In-depth interviews, paticipative observations, documental review and physical artifacts have been the sources of empirical data. Data analysis and interpretation has been backed up by a dialectic-hermeneutic method. The results point to reciprocal influences between technological frames and information systems evaluations (both formal and informal ones). Personal and contextual characteristics as well as technological ones are integrated creating, reinforcing and modifying technological frames. Thus, different evaluations of the academic information system are socially constructed by the individuals and become greatly affected by their prior knowledge of the system, their assumptions and expectations towards it. Technological frames manifest themselves in relation to both the content and processes of evaluation, thus influencing the identification of generating factors of such judgments, as well as their substance; frames also influence the manners how evaluations are made and the context where they take place. On the other hand, informal evaluation processes, the product of social interaction, affect technological frames. In these processes, previous knowledge is validated, amplified or refuted; assumptions and expectations are created and/or recreated. At the same time, the ways in which formal evaluations are proposed trigger certain knowledges, expectations and assumptions about technology.

A number of approaches of assessment associated with IS effectiveness have been examined, fundamental guidelines for research in this area have been derived, and a novel model of IS effectiveness has been proposed. A process based assessment method (PB-ISAM) based on the proposed effectiveness model has been elaborated. The new model and the new assessment method have been evaluated via three case studies. Specific implications have been drawn concerning the relationships between processes and the information system assessed. The three case studies have provided insight into the IS effectiveness field and for future work.

Avaliação de sistemas de informação e frames tecnológicos (estruturas cognitivas compartilhadas em relação à tecnologia) constituem-se as temáticas focais desta pesquisa cujo objetivo foi compreender como se estabelece a relação entre processos de avaliação formais e informais de um sistema de informação e frames tecnológicos. A estrutura de avaliação conteúdo-contexto-processo e o arcabouço teórico de frames tecnológicos, especialmente seus elementos constitutivos - conhecimentos, pressupostos e expectativas - foram associados visando a esta compreensão. A pesquisa foi realizada em uma instituição de ensino superior e o objeto de estudo tratou-se de um sistema de informação acadêmico (Peoplesoft), implementado e em uso nesta instituição. Especificamente analisaram-se: avaliações formais e informais do sistema acadêmico em relação à estrutura conteúdo-contexto-processo; a natureza e extensão de diferenças em frames tecnológicos de grupos sociais distintos; e como frames tecnológicos moldam e são moldados por avaliações formais e informais de um sistema de informação. Com pressupostos epistemológicos sustentados pela perspectiva interpretativista, metodologicamente o estudo caracterizou-se por uma abordagem qualitativa do tipo estudo de caso único e incorporado (grupos sociais como sub-unidades de análise). Entrevistas em profundidade, observação participante, revisão documental e artefato físico constituíram-se as fontes de dados empíricos. Adotaram-se os fundamentos do método hermenêutico-dialético para a análise e interpretação dos dados coletados. O estudo propiciou identificar que: frames tecnológicos e avaliações do sistema de informação, sejam formais ou informais, influenciam-se reciprocamente; características pessoais, contextuais e tecnológicas integram-se criando, reforçando e modificando frames tecnológicos; diferentes avaliações sobre o sistema acadêmico são socialmente construídas pelos indivíduos inseridos em grupos sociais e fortemente afetadas pelo que acreditam, conhecem e esperam deste sistema; frames tecnológicos manifestam-se tanto no conteúdo da avaliação quanto no processo avaliativo, influenciando a identificação de fatores do conteúdo assim como o teor do julgamento destes fatores, o modo pelo qual se avalia e o próprio contexto da avaliação; em contrapartida, processos de avaliações informais, decorrentes da interação social, afetam frames tecnológicos; nestes processos, conhecimentos prévios são validados, ampliados ou refutados, pressupostos e expectativas criados e recriados; igualmente, o modo como avaliações formais são configuradas desencadeia conhecimentos, pressupostos e expectativas sobre a tecnologia. / Information systems evaluation and technological frames (shared cognitive structures that relate to technology) are the main themes of this research. Its general goal has been to establish how the relationship between formal and informal processes of information systems evaluation and technological frames operates. The structure of evaluation (content-context-process) and the theoretical stance of the technological frames, especially in respect to its constitutive elements, have been analyzed and linked to each other. The research has been conducted in a graduate institution and its object of analysis, a software called \"Peoplesoft\", which was already implemented and that is in use nowadays. Specific analysis deal with the following topics: a) both formal and informal evaluations of the information system in relation to the \"content-context-process\" structure; the nature and extension of the differences that sustain technological frames for different groups as well how these technological frames are molded and formed by formal and informal assessment (evaluations) of such system. Epistemological suppositions were held by an interpretative perspective while, methodologically, this study is characterized by a qualitative perspective of a unique case in which social groups act as sub-units of analysis. In-depth interviews, paticipative observations, documental review and physical artifacts have been the sources of empirical data. Data analysis and interpretation has been backed up by a dialectic-hermeneutic method. The results point to reciprocal influences between technological frames and information systems evaluations (both formal and informal ones). Personal and contextual characteristics as well as technological ones are integrated creating, reinforcing and modifying technological frames. Thus, different evaluations of the academic information system are socially constructed by the individuals and become greatly affected by their prior knowledge of the system, their assumptions and expectations towards it. Technological frames manifest themselves in relation to both the content and processes of evaluation, thus influencing the identification of generating factors of such judgments, as well as their substance; frames also influence the manners how evaluations are made and the context where they take place. On the other hand, informal evaluation processes, the product of social interaction, affect technological frames. In these processes, previous knowledge is validated, amplified or refuted; assumptions and expectations are created and/or recreated. At the same time, the ways in which formal evaluations are proposed trigger certain knowledges, expectations and assumptions about technology.

M. Tech. (Information Technology, Faculty of Applied and Computer Sciences) Vaal University of Technology / A reliable and efficient information system (IS) is critical for Eskom so that it is able to manage and meet its energy demands. A reliable power supply provides stakeholders with the confidence that supply of power is managed sustainably, effectively and efficiently. Thus, an information system is integral in the effective and efficient generation, distribution and transmission of electricity. The purpose of the study was to investigate IS evaluation criteria and to develop a comprehensive framework that will serve as basis for IS evaluation across Eskom. The research study additionally investigated IS evaluation methods and instruments that are currently used in Eskom. This study produced an information systems success evaluation framework. The proposed model was built by reviewing well-established information systems success models and information systems theories found in the literature. This research study followed the interpretive research paradigm combining it with qualitative case study. The research findings linked information systems success to top management support, change management process and information quality. The findings of the study alsorevealed that quality of IS department’s service as perceived by users can greatly influence IS success. The results of this study provided enlightening reference benefit for Eskom, which was in line with Eskom’s goal of improving business processes, efficiencies and eliminating waste.

Submitted in fulfillment of the requirements for the degree of the Master of Information Technology, Durban University of Technology, Durban, South Africa, 2017. / The assignment of identifying Critical Success Factors (CSFs) for the successful implementation of the Enterprise Resource Planning (ERP) systems has become an important problem in the information system (IS) research. The necessity to identify CSFs becomes perceptible because of the failure often associated with the ERP system implementation in corporate organisations. The investigation and identification of CSFs will help cut costs of implementing ERP systems in organisations by giving higher precedence to the most critical factors. Literature has indicated that some factors of ERP system implementation labelled as critical are, in most cases, not critical for achieving success in the ERP system implementation. It can be argued that the inherent prediction error in the identification of CSFs is associated with the method employed for identifying criticality. Certain researchers have asserted that many of the studies on CSFs have based their findings on the use of content analysis method to identify and classify implementation factors of ERP systems as critical or not, rather than empirical findings. This intrinsic drawback has led researchers to suggest the use of sound scientific methods such as the structural equation modelling technique to identify CSFs to help guide the implementation of ERP systems in organizations. However, because of the limitations of the existing findings, the expectation is still much higher in effectively resolving the problem of identifying CSFs, in general. The overarching aim of this study was to determine those factors that are deemed critical for the successful implementation of the on-line registration system as an archetype of ERP system at HEIs. It was necessary to, firstly, identify common factors that have a significant impact on ERP system implementation and, secondly, to ascertain whether the identified factors are applicable in HEI settings, particularly to the on-line registration system. This study plans an in-depth exploration of the implementation of an on-line registration system with the identified factors forming the precursor to unearth those factors that are critical for the success of implementing on-line registration systems. The study has adopted a post-positivism mixed methods approach to identify and verify CSFs of the on-line registration system implementation, taking into consideration higher-order relationships between the factors. Data gathering took place using expert judgement with the involvement of role players in the implementation of on-line registration systems. The ADVIAN classification method provides the analytic tool for identifying factors that are deemed critical for successful implementation of on-line registration systems. The results reveal the existence of various dimensions of criticality with organisational culture and ERP strategy and implementation methodology emerging as critical factors, while the driving factors for implementation include ERP vendor support and guidance, senior and top management support, project plan with agreed objectives and goals, project management to implement project plan and project leader. It is established that the driven factors that should be observed when intervention measures are implemented include change management, post-implementation evaluation, software testing and troubleshooting, user training and user involvement. It is hoped that the CSFs discovered in this study will contribute towards the under-researched area of ERP and pragmatically aid the improvement of a process area that is in desperate need of business process re-engineering at HEIs. / M