In this Research Commentary, we describe 3 journal metrics–the Web of Science's Impact Factor, Scopus's SCImago Journal Rank, and Google Scholar Metrics' h5-index—and compile the rankings (if they exist) for 69 mathematics education journals. We then discuss 2 paths that the mathematics education community should consider with regard to these citation-based metrics of journal quality: either working within the system to enhance our positioning or resisting or modifying the system itself.

自2003年布達佩斯宣言公佈起，國際間學術文獻開始開放取用的趨勢。於此背景下，本研究以諾貝爾生物醫學獎近十年23位得主為研究樣本，評比在商業資料庫(SCIE、Scopus)及開放取用系統(生物醫學類：Pubmed、Highwire；綜合類：Google Scholar)的文獻收錄狀況，除了比較其內部重複性與完整性，並交叉比對五個資料庫與系統的重複性、獨特性及完整性，同時也觀看能否取得全文的比率，來了解現今開放取用文獻的狀況，進而觀察開放取用系統和商業資料庫兩者是否可以互補，或是開放取用系統有代替商業資料庫的可能性。 研究結果顯示五個資料庫及系統的檢索形式多元。針對作者檢索而言，Scopus最完善，資料收錄也較齊全；SCIE及Pubmed兩者則是檢索結果最為相似。如果以學術出版收錄而言，則是Highwire較完整；至於Google Scholar的獨特性較高。整體而言，開放取用系統比商業資料庫的全文收錄比例高，但Scopus是收錄最多全文的資料庫。本研究同時也發現PNAS是五個資料庫與系統之重複來源及獨特來源。另外，使用PubMed及Highwire檢索生物醫學文獻會比Google Scholar來得專業。 根據研究結果建議，商業資料庫可考慮將網路開放資源納入收錄範圍，以便妥善整理及應用網路資源的書目及全文。開放取用系統則應改善索引書目之正確性及著錄完整性。另外，針對圖書館的服務宜採取以下之因應措施：(1)加強推廣商業資料庫之正確檢索方式及使用時機；(2)教導如何正確使用開放取用系統的檢索模式；(3)平衡商業資料庫和開放取用系統的使用，以達成圖書館經費的合理運用。 本研究後續可延伸至生物醫學領域的臨床及實證醫學上，以了解生物醫學中兩個最具時效性的學術文獻系統是否達到開放取用的立即性及實用性。再者，使用者對開放取用的滿意度研究是學術出版界急欲了解的課題，也是後續研究可加強努力的方向。 / The International Scholarly Communication has gradually forwarded open access system since the publication of Budapest Declaration in 2003. Under this research background, this study uses biomedical Nobel Prize winners in recent years for the study of 23 samples of appraisal in the commercial database (SCIE, Scopus) and open access systems (biomedical categories: Pubmed, Highwire ; Comprehensive: Google Scholar) literature collection status, in addition to comparing repeatability and integrity of its internal and cross-comparison of the five databases and system repeatability, uniqueness and integrity, while also viewing the ability to obtain the ratio of text to understand current status of open access literature, and then observe the open access systems and commercial databases whether the two can complement each other, or open access database system instead of commercial possibilities. The results showed that five databases have different retrieval systems in many different forms. For the purposes of retrieval, Scopus collections are more complete; SCIE and Pubmed are the most similar two databases in the search results. Inclusion academic publishing purposes, Highwire is the most complete one. For Google Scholar, the collection’s uniqueness is the highest. Overall, comparing the open access system with commercial database, open access system contains a high proportion of full text. Scopus is the most one of full text collections. The PNAS study also found that five of the duplicate database and system sources and unique source. In addition, the use of PubMed and Highwire retrieved biomedical literature is more professional than Google Scholar. According to the study results suggest that commercial databases can be considered included in the scope of network resources into the open, in order to properly organize network resources and application of bibliographic and full-text. Open access system should improve the accuracy and bibliographic indexing bibliographic completeness. In addition, for the library service should take the following measures in response to: (a) enhance the promotion of commercial database retrieval methods and the use of proper timing; (2) to teach the proper use of open access system retrieval mode; (3) Balance Business open access database and use of the system, in order to achieve rational use of library funds. The follow-up research of this study can be extended to the field of clinical and biomedical evidence-based medicine research. The follow-up research results can be used to understand the biomedical literature’ timeliness, whether the system reaches an open access immediate or practicality. Furthermore, users' satisfaction with open access scholarly publishing research is also an anxious subject to know, and the follow-up study will strengthen efforts.

Ist die Zitationsanalyse ein geeignetes Instrument zur Forschungsevaluation? Diese Dissertation untersucht, ob die zugrunde liegenden Zitationsdaten ausreichend fehlerfrei sind, um aussagekräftige Ergebnisse der Analysen zu erzielen, beziehungsweise sollte dies nicht der Fall sein, ob der Prozess, der die zitierenden und zitierten Artikel einander zurordnet, ausreichend robust gegenüber Ungenauigkeiten in den Daten ist. Ungenauigkeiten wurden als Unterschiede in den Datenwerten der bibliographischen Angaben definiert. Die untersuchten Daten setzen sich aus gezielt ausgewählten Publikationen des Web of Science (WoS) zusammen, welche eine geschichtete Stichprobe ergeben. Die bibliographischen Daten von 3.929 Referenzen wurden in einer qualitativen Inhaltsanalyse bewertet und die bibliographischen Ungenauigkeiten in einer Taxonomie zusammengefasst. Um genau festzulegen, welche von diesen tatsächlich den Zuordnungsprozess von Zitationen beeinflussen, wurde eine spezifische Untergruppe von Zitationen, d.h. Zitationen die von WoS nicht erfolgreich dem jeweilig zitierten Artikel zugeordnet wurden, untersucht. Die Ergebnisse wurden mit den Daten zweier weiterer bibliographischen Datenbanken, Scopus und Google Scholar, sowie den Daten dreier angewandter bibliometrischer Forschungsgruppen, CWTS, iFQ und Science-Metrix, trianguliert. Die Zuordnungsalgorithmen von CWTS und iFQ konnten rund zwei Drittel dieser Zitierungen erfolgreich zuordnen. Scopus und Google Scholar konnten ebenso über 60% der fehlenden Zitierungen erfolgreich mit dem entsprechenden zitierten Artikel verbinden, während Science-Metrix nur eine geringe Anzahl an Referenzen (5%) schaffte. Vollkommen falsche erste Seitenzahlen sowie Zahlendreher in Publikationsjahren können in allen Datenquellen nicht richtig zugeordnete Zitierungen verursachen. Basierend auf den Ergebnissen wurden Lösungsvorschläge formuliert, die im Stande sind den Zuordnungsprozess von Zitationen in bibliometrischen Datenquellen zu verbessern. / Is citation analysis an adequate tool for research evaluation? This doctoral research investigates whether the underlying citation data is sufficiently accurate to provide meaningful results of the analyses and if not, whether the citation matching process can rectify inaccurate citation data. Inaccuracies are defined as discrepancies in the data values of bibliographic references, since they are the essential part in the citation matching process. A stratified, purposeful data sample was selected to examine typical cases of publications in Web of Science (WoS). The bibliographic data of 3,929 references was assessed in a qualitative content analysis to identify prevailing inaccuracies in bibliographic references that can interfere with the citation matching process. The inaccuracies were categorized into a taxonomy. Their frequency was studied to determine any strata-specific patterns. To pinpoint the types of inaccuracies that influence the citation matching process, a specific subset of citations, i.e. citations not successfully matched by WoS, was investigated. The results were triangulated with five other data sources: with data from two bibliographic databases in their role as citation indexes (Scopus and Google Scholar) and with data from three applied bibliometric research groups (CWTS, iFQ and Science-Metrix). The matching algorithms of CWTS and iFQ were able to match around two thirds of these citations correctly. Scopus and Google Scholar also handled more than 60% successfully in their matching. Science-Metrix only matched a small number of references (5%). Completely incorrect starting page numbers and transposed publication years can cause a citation to be missed in all data sources. However, more often it is a combination of more than one kind of inaccuracy in more than one field that leads to a non-match. Based on these results, proposals are formulated that could improve the citation matching processes of the different data sources.