The purpose of this thesis is to contribute to the methodology at the intersection of relational and evaluative bibliometrics. Experimental investigations are presented that address the question of how we can most successfully produce estimates of the subject similarity between documents. The results from these investigations are then explored in the context of citation-based research evaluations in an effort to enhance existing citation normalization methods that are used to enable comparisons of subject-disparate documents with respect to their relative impact or perceived utility. This thesis also suggests and explores an approach for revealing the uncertainty and stability (or lack thereof) coupled with different kinds of citation indicators.This suggestion is motivated by the specific nature of the bibliographic data and the data collection process utilized in citation-based evaluation studies. The results of these investigations suggest that similarity-detection methods that take a global view of the problem of identifying similar documents are more successful in solving the problem than conventional methods that are more local in scope. These results are important for all applications that require subject similarity estimates between documents. Here these insights are specifically adopted in an effort to create a novel citation normalization approach that – compared to current best practice – is more in tune with the idea of controlling for subject matter when thematically different documents are assessed with respect to impact or perceived utility. The normalization approach is flexible with respect to the size of the normalization baseline and enables a fuzzy partition of the scientific literature. It is shown that this approach is more successful than currently applied normalization approaches in reducing the variability in the observed citation distribution that stems from the variability in the articles’ addressed subject matter. In addition, the suggested approach can enhance the interpretability of normalized citation counts. Finally, the proposed method for assessing the stability of citation indicators stresses that small alterations that could be artifacts from the data collection and preparation steps can have a significant influence on the picture that is painted by the citationindicator. Therefore, providing stability intervals around derived indicators prevents unfounded conclusions that otherwise could have unwanted policy implications. Together, the new normalization approach and the method for assessing the stability of citation indicators have the potential to enable fairer bibliometric evaluative exercises and more cautious interpretations of citation indicators.

In today's significant and rapidly increasing amount of scientific publications, exploring recent studies in a given research area and building an effective scientific collaboration has become more challenging than any time before. Scientific production growth has been increasing the difficulties for identifying the most relevant papers to cite or to find an appropriate conference or journal to submit a paper to publish. As a result, authors and publishers rely on different analytical approaches in order to measure the relationship among the citation network. Different parameters have been used such as the impact factor, number of citations, co-citation to assess the impact of the produced research publication. However, using one assessing factor considers only one level of relationship exploration, since it does not reflect the effect of the other factors. In this thesis, we propose an approach to measure the Academic Citation Impact that will help to identify the impact of articles, authors, and venues at their extended nearby citation network. We combine the content similarity with the bibliometric indices to evaluate the citation impact of articles, authors, and venues in their surrounding citation network. Using the article metadata, we calculate the semantic similarity between any two articles in the extended network. Then we use the similarity score and bibliometric indices to evaluate the impact of the articles, authors, and venues among their extended nearby citation network. Furthermore, we propose an academic recommendation model to identify the latent preferences among the citation network of the given article in order to expose the concealed connection between the academic objects (articles, authors, and venues) at the citation network of the given article. To reveal the degree of trust for collaboration between academic objects (articles, authors, and venues), we use the similarity learning to estimate the collaborative confidence score that represents the anticipation of a prospect relationship between the academic objects among a scientific community. We conducted an offline experiment to measure the accuracy of delivering personalized recommendations, based on the user’s selection preferences; real-world datasets were used. Our evaluation results show a potential improvement to the quality of the recommendation when compared to baseline recommendation algorithms that consider co-citation information.

Nutzungsdaten von elektronischen wissenschaftlichen Publikationen und insbesondere die Anzahl ihrer Downloads rücken mit der Verbreitung des Internets zunehmend in den Blickpunkt des Interesses der Autoren, der Herausgeber, der technischen Anbieter und der Nutzer solcher Publikationen. Downloadzahlen von Publikationen, welche durch Auswertung der Protokolle der IT-Systeme der Anbieter ermittelt werden, sind solche Nutzungsdaten. Die Erhebung erfolgt durch Filterung aller stattgefundenen Zugriffe und Summierung über eine definierte Zeiteinheit. Downloadzahlen sind Gegenstand wissenschaftlicher Untersuchungen, in welchen das Konzept des Citation Impact auf die Nutzungshäufigkeit einer Publikation übertragen und der sogenannte Download Impact gebil-det wird. Besonderes Augenmerk wird dem Zusammenhang von Citation Impact und Download Impact gewidmet. Handelt es sich um Open-Access-Publikationen, muss davon ausgegangen werden, dass in den Downloadzahlen nicht nur menschliche, sondern auch maschinelle Zugriffe erfasst wurden, da eine sichere Unterscheidung unmöglich ist. Das hat zur Folge, dass die gewonnenen Daten für die einzelnen Publikationen unzuverlässig sind und starken Schwankungen unterliegen. Trotzdem enthalten sie wertvolle Informationen, welche mit Hilfe der Mathematischen Statistik nutzbar gemacht werden können. Mit nichtparametrischen Methoden ausgewertet, geben Downloadzahlen Auskunft über die Sichtbarkeit von elektronischen Publikationen im Internet. Diese Methoden bilden den Kern von NoRA (Non-parametric Repository Analysis), mit deren Hilfe die Betreiber von Open Access Repositories die Downloadzahlen ihrer elektronischen Publikationen auswerten können, um Sichtbarkeitsdefizite zu ermitteln und zu beheben und so die Qualität ihres Online-Angebotes zu erhöhen. Die Analysemethode NoRA wurde auf die Daten von vier universitären Institutional Repositories erfolgreich angewendet. Es konnten jeweils Gruppen von Publikationen identifiziert werden, die sich hinsichtlich ihrer Nutzung signifikant unterscheiden. Die Parallelen in den Ergebnissen weisen auf Einflussfaktoren für die Nutzungsdaten hin, welche in der gegenwärtigen Diskussion bisher keine Berücksichtigung finden. Hier erschließen sich weitere Anwendungsfelder für NoRA. Gleichzeitig geben die Ergebnisse Anlass, den Informationsgehalt von Downloadzahlen für die einzelne Publikation kritisch zu hinterfragen. / With the spread of internet usage over the past decades, access characteristics of electronic scientific publica-tions, especially the number of document downloads, are of increasing interest to the authors, publishers, technical providers and users of such publications. These download data of publications are usually obtained from the protocols of the IT systems of the provider. A data set is then created by filtering all accesses and subsequent summarizing over a certain time unit. Download data are the subject of scientific investigations, in which the concept of the Citation Impact is applied to the rate of use of a publication and the so-called Download Impact is formed. Special attention is paid to the relation between Citation Impact and Download Impact. In the case of Open Access publications, two types of access need to be distinguished. Human access and machine access are both captured and a reliable distinction is not possible yet. As a result, the data obtained for single publications are unreliable and subject to strong fluctuations. Nevertheless, they contain valuable information that can be made useful with the help of mathematical statistics. Analyzed with nonparametric methods, download data give information about the visibility of electronic publications on the Internet. These methods form the core of NoRA (Non-parametric Repository Analysis). With the help of NoRA, the operators of Open Access Repositories are able to analyze the download data of their electronic publications, to identify and correct deficiencies of visibility and to increase the quality of their online platform. The analytical method NoRA was successfully applied to data from Institutional Repositories of four universities. In each case, groups of publications were identified that differed significantly in their usage. Similarities in the results reveal factors that influence the usage data, which have not been taken into account previously. The presented results imply further applications of NoRA but also raise doubts about the value of download data of single publications.