The objective of this thesis is to obtain knowledge regarding how effective user centred evaluation methods are and how user centred evaluations are conducted by IT professionals. This will be achieved by exploring user centred evaluation in experimental and practical settings. The knowledge gained in these studies should inspire suggestions for further research and suggestions for improvements on the user centred evaluation activity. Two experimental studies were conducted. One compares the results from using three user centred evaluation methods, and the other examines two factors while conducting heuristic evaluation. The results show that the think-aloud evaluation method was the most effective method in finding realistic usability problems of the three methods. The number of critical problems found during think-aloud evaluation increases, if heuristic evaluation is conducted prior to the think-aloud evaluations. Further, two studies of user centred evaluation in practical settings were performed. The IT professionals participating in those studies were using the software development process Scrum to plan their work. The results show that user centred evaluation is infrequently conducted in Scrum projects, compared to testing activities like acceptance testing. The main type of evaluation is qualitative. Few participants measure user performance or use surveys to gather quantitative results on the usability and the user experience. IT professionals get feedback from users in an informal way and gather informal feedback from peers. Many participants use a mixture of methods for gathering feedback on their work. The outcome of this thesis shows that IT professionals should be encouraged to include users whenever possible when evaluating software, for example by using the think-aloud method. Using heuristic evaluation prior to conducting think-aloud evaluations is also recommended. In addition, IT professionals are encouraged to evaluate their software in an informal way frequently, rather than waiting for the right time to conduct a thorough quantitative evaluation. To advance this field further, researchers who want to improve the evaluation activity for the IT professionals should study how user centred evaluation methods could be combined in an efficient way and how the use of qualitative evaluation methods could be made more effective. / QC 20120522

Bibliography: leaves 207-224 / Abstracts and keywords following the Appendices / This thesis evaluates Ethiopian higher learning institutes’ digital libraries (DLs) for their usability and usefulness. The outcome contributes knowledge by helping to propel development efforts; satisfy DL stakeholders; provide information and fill information gaps; serve as cross-reference for academicians and researchers in the discipline; make decisions and solve managerial issues in DLs of the developing world; and provide scientific interests and merits for the advancement of information systems. When undertaking usability and usefulness evaluation research in DLs, there is no agreed-upon established conceptual framework that guides researchers in the developing world. Therefore, the overall objective of this research is to propose a framework for DL evaluation that fits into the context of developing countries. As the study is concerned with users’ internal attributes, the interpretive research paradigm is applied. A qualitative research approach is employed and a case study research design is followed. Multiple data-collection techniques, namely semi-structured interviews (17 respondents), open-ended questionnaires (17 respondents) and observations (4 observations) are employed. The study encompasses four public university DLs in Ethiopia. The participants have a positive perception of all attributes of usability. Most of the attributes of usefulness are also perceived positively, except coverage. The major challenges encountered by participants are network accessibility, interruption and speed, and access restrictions imposed on some DL content. DL benefits include easy access to the DL content, enhanced teaching and research, lower cost and easy sharing of contents. The expectations of users are ensuring resource availability and sustainability, overcoming the absence of user feedback and federated search problems, and improving network infrastructure and speed. The interaction triptych framework (ITF), which is used in the current study, is a well-known framework that is commonly implemented by researchers. However, since it does not incorporate the contextual situation of developing countries, ITF has some limitations. Therefore, based on the output of this empirical research and considering the usability and usefulness themes of ITF and its relationships with other socio-technical and contextual themes, a contextual DL evaluation framework is proposed. The proposed framework emphasises the social, institutional and contextual aspects of DLs. The proposed framework has six DL components, namely: DL user; content and collection; system and technology; services and support; user interface; and context. The proposed framework is called a digital library components interaction evaluation framework. / Lo mqondo uhlaziya imitapo yolwazi yedijithali (i-DLs) yezikhungo zokufunda ephezulu zase-Ethiopia ngokusebenziseka kwazo nokuba wusizo. Umphumela unikela ngolwazi ngokusiza ukuhambisa imizamo yentuthuko; ukwanelisa ababambiqhaza be-DL; ukuhlinzeka ngolwazi nokugcwalisa izikhala zolwazi; ukusebenza okubhekiswe kukho kwezifundiswa nabaphenyi emkhakheni; yenza izinqumo futhi ixazulule izingqinamba zokuphatha kuma-DL wezwe elisathuthuka; futhi unikeze izintshisekelo zesayensi kanye nokufaneleka ekuthuthukisweni kwezinhlelo zolwazi. Lapho wenza ucwaningo lokusebenziseka neliwusizo okuhlaziya kuma-DL, akukho luhlaka lomqondo okuvunyelwene ngalo oluhola abacwaningi ezweni elisathuthuka. Ngakho-ke, inhloso ephelele yalolu cwaningo ukuphakamisa uhlaka lokuhlaziya kwe-DL oluhambisana nomongo wamazwe asathuthuka. Njengoba ucwaningo luphathelene nezimpawu zangaphakathi zabasebenzisi, inqubo yocwaningo ehumushekayo iyasetshenziswa. Indlela yokucwaninga esezingeni elifanele iyasetshenziswa bese kulandelwa ukwakheka kocwaningo lwesifundo. Kusetshenziswa amasu amaningi wokuqoqa imininingwane, njengezingxoxo ezihlelwe kahle (abaphenduli abayi-17), imibuzo evulekile (abaphenduli abayi-17) kanye nokubhekwayo (okubhekwayo oku-4). Lolu cwaningo luhlanganisa ama-DL amane wemfundo ephakeme yomphakathi wase-Ethiopia. Abahlanganyeli banombono omuhle wazo zonke izici zokusebenziseka. Iningi lezimpawu eziwusizo zibonwa kahle futhi, ngaphandle kokumbozwa. Izinselela ezinkulu ababambiqhaza abahlangabezana ngazo ukufinyeleleka kwenethiwekhi, ukuphazamiseka nejubane, nemikhawulo yokufinyelela ebekiwe kokunye okuqukethwe kwe-DL. Izinzuzo ze-DL zifaka ukufinyelela okulula kokuqukethwe kwe-DL, ukufundiswa okuthuthukisiwe nocwaningo, izindleko eziphansi nokwabelana okulula kokuqukethwe. Okulindelwe abasebenzisi ukuqinisekisa ukutholakala kwezinsiza nokusimama, ukunqoba ukungabi bikho kwempendulo yomsebenzisi nezinkinga zokusesha ezihlanganisiwe, nokwenza ngcono ingqalasizinda yenethiwekhi nejubane. Uhlaka lokusebenzisana lwe-triptych (ITF), olusetshenziswa ocwaningweni lwamanje, uhlaka olwaziwa kakhulu olusetshenziswa ngokuvamile ngabaphenyi. Kodwa-ke, njengoba lungafaki isimo somongo samazwe asathuthuka, i-ITF inemikhawulo ethile. Ngakho-ke, ngokuya ngokukhishwa kwalolu cwaningo olunamandla kanye nokubheka ukusebenziseka nosizo lwezindikimba ze-ITF kanye nobudlelwano bayo nezinye izingqikithi zomphakathi nezobuchwepheshe nezingqikithi zomongo, kuhlongozwa uhlaka lokuhlaziya lomongo lwe-DL. Uhlaka oluhlongozwayo lugcizelela izici zenhlalo, zesikhungo nezingqikithi zomongo zama-DL. Uhlaka oluhlongozwayo lunezingxenye eziyisithupha ze-DL, okuyilezi: Umsebenzisi we-DL; okuqukethwe neqoqo; uhlelo nobuchwepheshe; izinsizakalo nokwesekwa; isikhombimsebenzisi somsebenzisi nomongo. Uhlaka oluhlongozwayo lubizwa ngohlaka lokuhlaziya kokusebenzisana kokuqukethwe komtapo wolwazi wedijithali. / Hierdie proefskrif evalueer Ethiopië se hoër leerinstellings se digitale biblioteke (DBe) ten opsigte van hulle bruikbaarheid en nuttigheid. Die uitkoms dra by tot kennis wat kan help om die ontwikkelingspogings aan te dryf; om belanghebbers van digitale biblioteke tevrede te hou; om inligting te verskaf en inligtingsgapings te vul; deur te dien as kruisverwysing vir akademici en navorsers in die vakgebied; om besluite te neem en bestuursaangeleenthede in digitale biblioteke van die ontwikkelende wêreld op te los; en om wetenskaplike belange en meriete vir die bevordering van inligtingstelsels te voorsien. Wanneer bruikbaarheids- en nuttigheidsevalueringsnavorsing in digitale biblioteke onderneem word, is daar geen ooreengekome, gevestigde raamwerk wat leiding aan navorsers in die ontwikkelende wêreld kan gee nie. Die algehele doelwit van hierdie navorsing was derhalwe om ’n raamwerk vir die evaluaering van digitale biblioteke voor te stel wat in die konteks van ontwikkelende lande pas. Omdat die studie gemoeid is met gebruikers se interne eienskappe, is die vertolkende paradigma gebruik. ’n Kwalitatiewe navorsingsbenadering is gebruik en ’n gevallestudie-navorsingsontwerp is gevolg. Veelvuldige data-insamelingstegnieke, naamlik semigestruktureerde onderhoude (17 respondente), oop vraelyste (17 respondente) en waarnemings (vier waarnemigs), is ingespan. Die studie sluit vier openbare universiteite se digitale biblioteke in Ethiopië in. Die deelnemers het ’n positiewe persepsie van al die eienskappe van bruikbaarheid. Die meeste van die eienskappe van nuttigheid word ook as positief waargeneem, met die uitsluiting van voorsiening. Die grootste uitdagings wat die deelnemers ervaar het, was netwerktoeganklikheid, -onderbrekings en -spoed en die toegangbeperkings wat sommige digitale biblioteke oplê. Voordele van digitale biblioteke sluit die volgende in: toegang tot digitalebiblioteekinhoud; beter onderrig en navorsing; laer koste; en die maklike deel van inhoud met ander. Die verwagtings van gebruikers verseker hulpbronbeskikbaarheid en -volhoubaarheid, en om die afwesigheid van gebruikersterugvoer en gefedereerde soekprobleme en die verbetering van netwerkinfrastruktuur en -spoed te oorkom. Die interaksietriptiekraamwerk (ITF) wat in hierdie studie gebruik is, is ’n bekende raamwerk wat algemeen deur navorsers geïmplementeer word. Omdat dit egter nie die kontekstuele situasie van ontwikkelende lande inkorporeer nie, het ITF sekere beperkings. Gebaseer op die uitset van hierdie empiriese navorsing en met die inagneming van die bruikbaarheids- en nuttigheidstemas van ITF en sy verhouding met ander sosio-tegniese en kontekstuele temas, word ’n kontekstuele digitalebiblioteekraamwerk derhalwe voorgestel. Die voorgestelde raamwerk beklemtoon die sosiale, institusionele en kontekstuele aspekte van digitale biblioteke. Die voorgestelde raamwerk het ses digitalebiblioteekkomponente, naamlik: digitalebiblioteekgebruiker; inhoud en insameling; stelsel en tegnologie; dienste en ondersteuning; gebruikerkoppelvlak; en konteks. Die voorgestelde raamwerk word ’n interaksie-evalueringsraamwerk vir digitalebiblioteekkomponente genoem. / School of Computing / D. Phil. (Information Systems)