This diploma thesis is focused on determining the degree of presence of Benford's law in citation data. The data and their acquisition are described in detail. The most extensive analysis was performed on a dataset of 8.6 million records of scientific outputs from the Web of Science database, over a five-year period, with a selection of the three most numerous and most cited types of documents. Descriptive MAD (Mean Absolute Deviation) statistic were used to determine the degree of presence of Benford's law. The degree of presence of Benford's law was also determined for two datasets, the production of public universities in the Czech Republic and the Academy of Sciences of the Czech Republic under the same conditions as in other analyses.

Opsomming Daar bestaan 'n behoefte aan 'n objektiewe maatstaf om die gehalte van akademiese publikasies te bepaal en te vergelyk. Hierdie navorsing het die invloed of reaksie wat deur 'n publikasie gegenereer is uit verwysingsdata bepaal. Daar is van 'n iteratiewe algoritme gebruik gemaak wat gewigte aan verwysings toeken. In die Internetomgewing word hierdie benadering reeds met groot sukses toegepas deur onder andere die PageRank-algoritme van die Google soekenjin. Hierdie en ander algoritmes in die Internetomgewing is bestudeer om 'n algoritme vir akademiese artikels te ontwerp. Daar is op 'n variasie van die PageRank-algoritme besluit wat 'n Invloedwaarde bepaal. Die algoritme is op gevallestudies getoets. Die empiriese studie dui daarop dat hierdie variasie spesialisnavorsers se intu¨ıtiewe gevoel beter weergee as net die blote tel van verwysings. Abstract Ranking of journals are often used as an indicator of quality, and is extensively used as a mechanism for determining promotion and funding. This research studied ways of extracting the impact, or influence, of a journal from citation data, using an iterative process that allocates a weight to the source of a citation. After evaluating and discussing the characteristics that influence quality and importance of research with specialist researchers, a measure called the Influence factor was introduced, emulating the PageRankalgorithm used by Google to rank web pages. The Influence factor can be seen as a measure of the reaction that was generated by a publication, based on the number of scientists who read and cited itA good correlation between the rankings produced by the Influence factor and that given by specialist researchers were found. / Mathematical Sciences / M.Sc. (Operasionele Navorsing)

Opsomming Daar bestaan 'n behoefte aan 'n objektiewe maatstaf om die gehalte van akademiese publikasies te bepaal en te vergelyk. Hierdie navorsing het die invloed of reaksie wat deur 'n publikasie gegenereer is uit verwysingsdata bepaal. Daar is van 'n iteratiewe algoritme gebruik gemaak wat gewigte aan verwysings toeken. In die Internetomgewing word hierdie benadering reeds met groot sukses toegepas deur onder andere die PageRank-algoritme van die Google soekenjin. Hierdie en ander algoritmes in die Internetomgewing is bestudeer om 'n algoritme vir akademiese artikels te ontwerp. Daar is op 'n variasie van die PageRank-algoritme besluit wat 'n Invloedwaarde bepaal. Die algoritme is op gevallestudies getoets. Die empiriese studie dui daarop dat hierdie variasie spesialisnavorsers se intu¨ıtiewe gevoel beter weergee as net die blote tel van verwysings. Abstract Ranking of journals are often used as an indicator of quality, and is extensively used as a mechanism for determining promotion and funding. This research studied ways of extracting the impact, or influence, of a journal from citation data, using an iterative process that allocates a weight to the source of a citation. After evaluating and discussing the characteristics that influence quality and importance of research with specialist researchers, a measure called the Influence factor was introduced, emulating the PageRankalgorithm used by Google to rank web pages. The Influence factor can be seen as a measure of the reaction that was generated by a publication, based on the number of scientists who read and cited itA good correlation between the rankings produced by the Influence factor and that given by specialist researchers were found. / Mathematical Sciences / M.Sc. (Operasionele Navorsing)

Ist die Zitationsanalyse ein geeignetes Instrument zur Forschungsevaluation? Diese Dissertation untersucht, ob die zugrunde liegenden Zitationsdaten ausreichend fehlerfrei sind, um aussagekräftige Ergebnisse der Analysen zu erzielen, beziehungsweise sollte dies nicht der Fall sein, ob der Prozess, der die zitierenden und zitierten Artikel einander zurordnet, ausreichend robust gegenüber Ungenauigkeiten in den Daten ist. Ungenauigkeiten wurden als Unterschiede in den Datenwerten der bibliographischen Angaben definiert. Die untersuchten Daten setzen sich aus gezielt ausgewählten Publikationen des Web of Science (WoS) zusammen, welche eine geschichtete Stichprobe ergeben. Die bibliographischen Daten von 3.929 Referenzen wurden in einer qualitativen Inhaltsanalyse bewertet und die bibliographischen Ungenauigkeiten in einer Taxonomie zusammengefasst. Um genau festzulegen, welche von diesen tatsächlich den Zuordnungsprozess von Zitationen beeinflussen, wurde eine spezifische Untergruppe von Zitationen, d.h. Zitationen die von WoS nicht erfolgreich dem jeweilig zitierten Artikel zugeordnet wurden, untersucht. Die Ergebnisse wurden mit den Daten zweier weiterer bibliographischen Datenbanken, Scopus und Google Scholar, sowie den Daten dreier angewandter bibliometrischer Forschungsgruppen, CWTS, iFQ und Science-Metrix, trianguliert. Die Zuordnungsalgorithmen von CWTS und iFQ konnten rund zwei Drittel dieser Zitierungen erfolgreich zuordnen. Scopus und Google Scholar konnten ebenso über 60% der fehlenden Zitierungen erfolgreich mit dem entsprechenden zitierten Artikel verbinden, während Science-Metrix nur eine geringe Anzahl an Referenzen (5%) schaffte. Vollkommen falsche erste Seitenzahlen sowie Zahlendreher in Publikationsjahren können in allen Datenquellen nicht richtig zugeordnete Zitierungen verursachen. Basierend auf den Ergebnissen wurden Lösungsvorschläge formuliert, die im Stande sind den Zuordnungsprozess von Zitationen in bibliometrischen Datenquellen zu verbessern. / Is citation analysis an adequate tool for research evaluation? This doctoral research investigates whether the underlying citation data is sufficiently accurate to provide meaningful results of the analyses and if not, whether the citation matching process can rectify inaccurate citation data. Inaccuracies are defined as discrepancies in the data values of bibliographic references, since they are the essential part in the citation matching process. A stratified, purposeful data sample was selected to examine typical cases of publications in Web of Science (WoS). The bibliographic data of 3,929 references was assessed in a qualitative content analysis to identify prevailing inaccuracies in bibliographic references that can interfere with the citation matching process. The inaccuracies were categorized into a taxonomy. Their frequency was studied to determine any strata-specific patterns. To pinpoint the types of inaccuracies that influence the citation matching process, a specific subset of citations, i.e. citations not successfully matched by WoS, was investigated. The results were triangulated with five other data sources: with data from two bibliographic databases in their role as citation indexes (Scopus and Google Scholar) and with data from three applied bibliometric research groups (CWTS, iFQ and Science-Metrix). The matching algorithms of CWTS and iFQ were able to match around two thirds of these citations correctly. Scopus and Google Scholar also handled more than 60% successfully in their matching. Science-Metrix only matched a small number of references (5%). Completely incorrect starting page numbers and transposed publication years can cause a citation to be missed in all data sources. However, more often it is a combination of more than one kind of inaccuracy in more than one field that leads to a non-match. Based on these results, proposals are formulated that could improve the citation matching processes of the different data sources.