Esta tese visa analisar a constituição, o funcionamento e os limites da Avaliação por Pares (AP) no âmbito do Conselho Nacional de Desenvolvimento Científico e Tecnológico (CNPq). A pesquisa operou por dois caminhos distintos: um diacrônico e outro sincrônico. No primeiro, elaborou-se uma reconstrução histórica da AP praticada no CNPq entre 1951 e 2015 contextualizada pelas mudanças institucionais a que passou o órgão durante o citado período. O segundo caminho centrou-se na análise de dois estudos de caso que redundaram na elaboração de perfis dos Bolsistas de Produtividade em Pesquisa de Filosofia e Educação com bolsas vigentes no ano de 2016. Do um ponto de vista teórico, os resultados alcançados foram interpretados segundo as noções de Campo Científico de Pierre Bourdieu e Reconhecimento de Axel Honneth. Quanto à história da AP no CNPq constatou-se uma ligação entre o crescimento na quantidade de bolsas implementadas pelo órgão e uma maior formalização legal e institucional da AP, especialmente a partir da década de 1970. O próprio crescimento da comunidade científica brasileira possibilitou uma transição de procedimentos pessoais e informais de AP para um rígido, formal e impessoal circuito de imposições legais quanto a AP praticada no órgão. Quanto aos dois estudos de caso constatou-se que apesar de todas as restrições formais que atualmente regulam a AP no Conselho, especificidades históricas, geográficas, econômicas e relativas a gênero acabam por influir nos resultados dos processos de AP. Com base na pesquisa realizada, verificou-se a necessidade em se instituir uma maior transparência nos processos de AP no órgão de modo a se coadunar um aperfeiçoamento da própria prática da AP assim como um controle social da destinação dos recursos públicos mais democrático e acessível a sociedade civil. / This theses analyzes the institution, the mechanisms and the limits of peer review (PR) in Brazil’s National Council for Technological and Scientific Development (CNPq). Research was conducted at both diachronic and synchronic levels. At the diachronic level, I accomplished a historical reconstruction of PR practices in CNPq from 1975 to 2015, linking this reconstruction to the institutional changes undergone by the Council during this period. At the synchronic level, I focused on two case studies, which allowed me to outline the profile of researchers in Philosophy and Education with productivity grants (in 2016). Research findings were interpreted with the help of Pierre Bourdieu’s notion of Scientific Field and Axel Honneth’s idea of Recognition. As for the history of PR in CNPq, research results established a correlation between the increase in grants funded and the legal and institutional formalization of PR, particularly since the 1970s. The growth of the Brazilian scientific community entailed a shift from informal and personal PR procedures to the rigid, formal and impersonal mediating rules now in place. As for the two case studies, I found that despite all formal restrictions now governing the Council’s PR practices, historical, geographical, economical and gender based specificities still have an influence on the outcome of PR processes. Research results indicate the need for a greater transparency regarding the Council’s PR processes in order to improve practices and expand social control over allocation of public resources.

Esta tese visa analisar a constituição, o funcionamento e os limites da Avaliação por Pares (AP) no âmbito do Conselho Nacional de Desenvolvimento Científico e Tecnológico (CNPq). A pesquisa operou por dois caminhos distintos: um diacrônico e outro sincrônico. No primeiro, elaborou-se uma reconstrução histórica da AP praticada no CNPq entre 1951 e 2015 contextualizada pelas mudanças institucionais a que passou o órgão durante o citado período. O segundo caminho centrou-se na análise de dois estudos de caso que redundaram na elaboração de perfis dos Bolsistas de Produtividade em Pesquisa de Filosofia e Educação com bolsas vigentes no ano de 2016. Do um ponto de vista teórico, os resultados alcançados foram interpretados segundo as noções de Campo Científico de Pierre Bourdieu e Reconhecimento de Axel Honneth. Quanto à história da AP no CNPq constatou-se uma ligação entre o crescimento na quantidade de bolsas implementadas pelo órgão e uma maior formalização legal e institucional da AP, especialmente a partir da década de 1970. O próprio crescimento da comunidade científica brasileira possibilitou uma transição de procedimentos pessoais e informais de AP para um rígido, formal e impessoal circuito de imposições legais quanto a AP praticada no órgão. Quanto aos dois estudos de caso constatou-se que apesar de todas as restrições formais que atualmente regulam a AP no Conselho, especificidades históricas, geográficas, econômicas e relativas a gênero acabam por influir nos resultados dos processos de AP. Com base na pesquisa realizada, verificou-se a necessidade em se instituir uma maior transparência nos processos de AP no órgão de modo a se coadunar um aperfeiçoamento da própria prática da AP assim como um controle social da destinação dos recursos públicos mais democrático e acessível a sociedade civil. / This theses analyzes the institution, the mechanisms and the limits of peer review (PR) in Brazil’s National Council for Technological and Scientific Development (CNPq). Research was conducted at both diachronic and synchronic levels. At the diachronic level, I accomplished a historical reconstruction of PR practices in CNPq from 1975 to 2015, linking this reconstruction to the institutional changes undergone by the Council during this period. At the synchronic level, I focused on two case studies, which allowed me to outline the profile of researchers in Philosophy and Education with productivity grants (in 2016). Research findings were interpreted with the help of Pierre Bourdieu’s notion of Scientific Field and Axel Honneth’s idea of Recognition. As for the history of PR in CNPq, research results established a correlation between the increase in grants funded and the legal and institutional formalization of PR, particularly since the 1970s. The growth of the Brazilian scientific community entailed a shift from informal and personal PR procedures to the rigid, formal and impersonal mediating rules now in place. As for the two case studies, I found that despite all formal restrictions now governing the Council’s PR practices, historical, geographical, economical and gender based specificities still have an influence on the outcome of PR processes. Research results indicate the need for a greater transparency regarding the Council’s PR processes in order to improve practices and expand social control over allocation of public resources.

No description available.

The measurement of research output is common practice among public institutions internationally, and is increasingly contested and controversial. The term “research” is itself contested and can cover quite a wide range of activities, from carefully designed studies by independent, university-based researchers to analysis of data for particular administrative or political purposes to arguments for specific policy positions that may be more or less well grounded in evidence. Such measurement of research output is needed for decisions about professional staff and resource allocations. Measures of research productivity, covering both quantity and quality at national level, support government decisions on setting priorities and funding. With increasing competitive allocation of research funding and declining public funds for higher education, institutions around the world are facing increasing pressure to produce research outputs. The revenue generated through published research has therefore come to assume greater and greater significance in institutional budgets and in academic reward systems. Moreover, research in public institutions is funded mainly according to the number and quality of publications of members of staff. On the other hand, the growing international trend towards ranking institutions in competitive terms has assigned considerable value to research output as a measure of institutional standing in the global marketplace. What counts as an acceptable unit of measurement therefore becomes the subject of considerable debate within and outside institutions as they seek to enhance institutional standing and revenue. Whilst measurable output such as scientific publications and research reports are usually considered for government subsidy, it is difficult to accept that other output types such as patents, software, advisory work for government, consulting, or technical assistance, are not measurable, and do not have any relevance with respect to research subsidy. This thesis was set out to critically examine the effects that current government policy on the measurement of research output of public higher education institutions will have on the performance of South African Higher Education Institutions (HEIs). The purpose of this study was to trace and explain the differential impact of new government policies on the measurement of institutional research output in four different university faculties. The study has highlighted key challenges facing the universities in implementing the new research subsidy policy; and made recommendations and proposals on how best can the policy be implemented with the view of increasing or improving the institutions’ research output. / Dissertation (M.Ed (Education Management, Law and Policy))--University of Pretoria, 2007. / Education Management and Policy Studies / unrestricted

Multi-rater tools also referred to as 360-degree feedback tools, are frequently used in addition to traditional supervisory appraisals due to sources (i.e., supervisor, peer, direct report) unique perspectives and opportunities to view different aspects of job performance. Research has found that the differences among sources are most prevalent between self and other ratings, and the direction of agreement is related to overall job performance. Research has typically focused on one form of agreement, the direction of an individual's self-ratings compared to others' ratings. The current study expanded on past research on rater agreement using a data set (n = 215) consisting of multi-rater data for professionals participating in a leadership development process. The study examined the ability to predict job performance with three different measures of self-other agreement (i.e., difference between overall mean scores (difference), mean absolute difference across items (difference), and mean correlation across items (similarity)). The study also examined how the relationships may differ across performance dimensions. The final purpose was to explore how the importance of the performance dimensions, as rated by the participant, may moderate the relationship between self-other agreement and job performance. Partial support for study's hypotheses was found. The direction and difference measures of agreement on the overall multi-rater tool and performance dimensions accounted for a significant amount of the variance in job performance. The relationship between the similarity measure of agreement and job performance, and the moderating effect of importance were not supported in the current study.

This study reports on an investigation to determine the applicability of the peer rating technique to Air Force ROTC cadets at North Texas State College which has an enrollment of approximately 7000 students. The specific problem investigated was whether or not the peer rating would be useful in solving the leader identification problem in Air Force ROTC.

The African Peer Review Mechanism (APRM) was developed under the New Partnership for Africa’s Development (NEPAD), the development framework of the African Union (AU) which replaced the Organization of African Unity (OAU) in 2002. Through the APRM, the AU has established a system for assessment of governance in participating countries and for the development of programmes of action to address identified shortcomings. The APRM is a voluntary, ‘soft’ mechanism of supervision which combines self-assessment with regional monitoring. The APRM takes a holistic approach to governance with a mandate covering democracy and political governance, economic governance, corporate governance and socio-economic development. This study considers the role that the APRM plays in the realisation of human rights. It examines the manner in which human rights are reflected in the APRM framework documents as well as the manner in which rights-based principles such as participation, accountability and transparency are reflected in the process. The strengths and weaknesses of various methods of international monitoring to ensure compliance with human rights are examined. The APRM country review reports and implementation reports of Ghana, Rwanda and Kenya are studied in conjunction with reports from domestic and international human rights monitoring bodies and national development plans. The aim of the study is to ascertain whether the APRM adds value to mechanisms established with the purport of assisting in the realisation of human rights. This study illustrates that the APRM plays a complementary role in human rights monitoring. It is clear, however, that it is only able to play a meaningful role if the state under review is motivated to undertake reform. Human rights have a role to play with regard to the APRM process itself and in identifying and addressing governance shortcomings. The specific and time-bound commitments in the Programme of Action are unique to the APRM. If these commitments are developed through a rights-based approach and their implementation adequately monitored the APRM could play an important role in inducing compliance with human rights. / Thesis (LLD)--University of Pretoria, 2010. / Centre for Human Rights / Unrestricted

Esta tese visa analisar a constituição, o funcionamento e os limites da Avaliação por Pares (AP) no âmbito do Conselho Nacional de Desenvolvimento Científico e Tecnológico (CNPq). A pesquisa operou por dois caminhos distintos: um diacrônico e outro sincrônico. No primeiro, elaborou-se uma reconstrução histórica da AP praticada no CNPq entre 1951 e 2015 contextualizada pelas mudanças institucionais a que passou o órgão durante o citado período. O segundo caminho centrou-se na análise de dois estudos de caso que redundaram na elaboração de perfis dos Bolsistas de Produtividade em Pesquisa de Filosofia e Educação com bolsas vigentes no ano de 2016. Do um ponto de vista teórico, os resultados alcançados foram interpretados segundo as noções de Campo Científico de Pierre Bourdieu e Reconhecimento de Axel Honneth. Quanto à história da AP no CNPq constatou-se uma ligação entre o crescimento na quantidade de bolsas implementadas pelo órgão e uma maior formalização legal e institucional da AP, especialmente a partir da década de 1970. O próprio crescimento da comunidade científica brasileira possibilitou uma transição de procedimentos pessoais e informais de AP para um rígido, formal e impessoal circuito de imposições legais quanto a AP praticada no órgão. Quanto aos dois estudos de caso constatou-se que apesar de todas as restrições formais que atualmente regulam a AP no Conselho, especificidades históricas, geográficas, econômicas e relativas a gênero acabam por influir nos resultados dos processos de AP. Com base na pesquisa realizada, verificou-se a necessidade em se instituir uma maior transparência nos processos de AP no órgão de modo a se coadunar um aperfeiçoamento da própria prática da AP assim como um controle social da destinação dos recursos públicos mais democrático e acessível a sociedade civil. / This theses analyzes the institution, the mechanisms and the limits of peer review (PR) in Brazil’s National Council for Technological and Scientific Development (CNPq). Research was conducted at both diachronic and synchronic levels. At the diachronic level, I accomplished a historical reconstruction of PR practices in CNPq from 1975 to 2015, linking this reconstruction to the institutional changes undergone by the Council during this period. At the synchronic level, I focused on two case studies, which allowed me to outline the profile of researchers in Philosophy and Education with productivity grants (in 2016). Research findings were interpreted with the help of Pierre Bourdieu’s notion of Scientific Field and Axel Honneth’s idea of Recognition. As for the history of PR in CNPq, research results established a correlation between the increase in grants funded and the legal and institutional formalization of PR, particularly since the 1970s. The growth of the Brazilian scientific community entailed a shift from informal and personal PR procedures to the rigid, formal and impersonal mediating rules now in place. As for the two case studies, I found that despite all formal restrictions now governing the Council’s PR practices, historical, geographical, economical and gender based specificities still have an influence on the outcome of PR processes. Research results indicate the need for a greater transparency regarding the Council’s PR processes in order to improve practices and expand social control over allocation of public resources.

Le système d’évaluation par les pairs est le gold standard de la publication scientifique. Ce système a deux objectifs: d’une part filtrer les articles scientifiques erronés ou non pertinents et d’autre part améliorer la qualité de ceux jugés dignes de publication. Le rôle des revues scientifiques et des rédacteurs en chef est de veiller à ce que des connaissances scientifiques valides soient diffusées auprès des scientifiques concernés et du public. Cependant, le système d’évaluation par les pairs a récemment été critiqué comme étant intenable sur le long terme, inefficace et cause de délais de publication des résultats scientifiques. Dans ce projet de doctorat, j’ai utilisé une modélisation par systèmes complexes pour étudier le comportement macroscopique des systèmes de publication et d’évaluation par les pairs. Dans un premier projet, j’ai modélisé des données empiriques provenant de diverses sources comme Pubmed et Publons pour évaluer la viabilité du système. Je montre que l’offre dépasse de 15% à 249% la demande d’évaluation par les pairs et, par conséquent, le système est durable en termes de volume. Cependant, 20% des chercheurs effectuent 69% à 94% des revues d’articles, ce qui souligne un déséquilibre significatif en termes d’efforts de la communauté scientifique. Les résultats ont permis de réfuter la croyance largement répandue selon laquelle la demande d’évaluation par les pairs dépasse largement l’offre mais ont montré que la majorité des chercheurs ne contribue pas réellement au processus. Dans mon deuxième projet, j’ai développé un modèle par agents à grande échelle qui imite le comportement du système classique d’évaluation par les pairs, et que j’ai calibré avec des données empiriques du domaine biomédical. En utilisant ce modèle comme base pour mon troisième projet, j’ai modélisé cinq systèmes alternatifs d’évaluation par les pairs et évalué leurs performances par rapport au système conventionnel en termes d’efficacité de la revue, de temps passé à évaluer des manuscrits et de diffusion de l’information scientifique. Dans mes simulations, les deux systèmes alternatifs dans lesquels les scientifiques partagent les commentaires sur leurs manuscrits rejetés avec les éditeurs du prochain journal auquel ils les soumettent ont des performances similaires au système classique en termes d’efficacité de la revue. Le temps total consacré par la communauté scientifique à l’évaluation des articles est cependant réduit d’environ 63%. En ce qui concerne la dissémination scientifique, le temps total de la première soumission jusqu’à la publication est diminué d’environ 47% et ces systèmes permettent de diffuser entre 10% et 36% plus d’informations scientifiques que le système conventionnel. Enfin, le modèle par agents développé peut être utilisé pour simuler d’autres systèmes d’évaluation par les pairs ou des interventions, pour ainsi déterminer les interventions ou modifications les plus prometteuses qui pourraient être ensuite testées par des études expérimentales en vie réelle. / The peer-review system is undoubtedly the gold standard of scientific publication. Peer review serves a two-fold purpose; to screen out of publication articles containing incorrect or irrelevant science and to improve the quality of the ones deemed suitable for publication. Moreover, the role of the scientific journals and editors is to ensure that valid scientific knowledge is disseminated to the appropriate target group of scientists and to the public. However, the peer review system has recently been criticized, in that it is unsustainable, inefficient and slows down publication. In this PhD thesis, I used complex-systems modeling to study the macroscopic behavior of the scientific publication and peer-review systems. In my first project, I modeled empirical data from various sources, such as Pubmed and Publons, to assess the sustainability of the system. I showed that the potential supply has been exceeding the demand for peer review by 15% to 249% and thus, the system is sustainable in terms of volume. However, 20% of researchers have been performing 69% to 94% of the annual reviews, which emphasizes a significant imbalance in terms of effort by the scientific community. The results provided evidence contrary to the widely-adopted, but untested belief, that the demand for peer review over-exceeds the supply, and they indicated that the majority of researchers do not contribute to the process. In my second project, I developed a large-scale agent-based model, which mimicked the behavior of the conventional peer-review system. This model was calibrated with empirical data from the biomedical domain. Using this model as a base for my third project, I developed and assessed the performance of five alternative peer-review systems by measuring peer-review efficiency, reviewer effort and scientific dissemination as compared to the conventional system. In my simulations, two alternative systems, in which scientists shared past reviews of their rejected manuscripts with the editors of the next journal to which they submitted, performed equally or sometimes better in terms of peer-review efficiency. They also each reduced the overall reviewer effort by ~63%. In terms of scientific dissemination, they decreased the median time from first submission until publication by ~47% and diffused on average 10% to 36% more scientific information (i.e., manuscript intrinsic quality x journal impact factor) than the conventional system. Finally, my agent-based model may be an approach to simulate alternative peer-review systems (or interventions), find those that are the most promising and aid decisions about which systems may be introduced into real-world trials.

No description available.