Esta tese visa analisar a constituição, o funcionamento e os limites da Avaliação por Pares (AP) no âmbito do Conselho Nacional de Desenvolvimento Científico e Tecnológico (CNPq). A pesquisa operou por dois caminhos distintos: um diacrônico e outro sincrônico. No primeiro, elaborou-se uma reconstrução histórica da AP praticada no CNPq entre 1951 e 2015 contextualizada pelas mudanças institucionais a que passou o órgão durante o citado período. O segundo caminho centrou-se na análise de dois estudos de caso que redundaram na elaboração de perfis dos Bolsistas de Produtividade em Pesquisa de Filosofia e Educação com bolsas vigentes no ano de 2016. Do um ponto de vista teórico, os resultados alcançados foram interpretados segundo as noções de Campo Científico de Pierre Bourdieu e Reconhecimento de Axel Honneth. Quanto à história da AP no CNPq constatou-se uma ligação entre o crescimento na quantidade de bolsas implementadas pelo órgão e uma maior formalização legal e institucional da AP, especialmente a partir da década de 1970. O próprio crescimento da comunidade científica brasileira possibilitou uma transição de procedimentos pessoais e informais de AP para um rígido, formal e impessoal circuito de imposições legais quanto a AP praticada no órgão. Quanto aos dois estudos de caso constatou-se que apesar de todas as restrições formais que atualmente regulam a AP no Conselho, especificidades históricas, geográficas, econômicas e relativas a gênero acabam por influir nos resultados dos processos de AP. Com base na pesquisa realizada, verificou-se a necessidade em se instituir uma maior transparência nos processos de AP no órgão de modo a se coadunar um aperfeiçoamento da própria prática da AP assim como um controle social da destinação dos recursos públicos mais democrático e acessível a sociedade civil. / This theses analyzes the institution, the mechanisms and the limits of peer review (PR) in Brazil’s National Council for Technological and Scientific Development (CNPq). Research was conducted at both diachronic and synchronic levels. At the diachronic level, I accomplished a historical reconstruction of PR practices in CNPq from 1975 to 2015, linking this reconstruction to the institutional changes undergone by the Council during this period. At the synchronic level, I focused on two case studies, which allowed me to outline the profile of researchers in Philosophy and Education with productivity grants (in 2016). Research findings were interpreted with the help of Pierre Bourdieu’s notion of Scientific Field and Axel Honneth’s idea of Recognition. As for the history of PR in CNPq, research results established a correlation between the increase in grants funded and the legal and institutional formalization of PR, particularly since the 1970s. The growth of the Brazilian scientific community entailed a shift from informal and personal PR procedures to the rigid, formal and impersonal mediating rules now in place. As for the two case studies, I found that despite all formal restrictions now governing the Council’s PR practices, historical, geographical, economical and gender based specificities still have an influence on the outcome of PR processes. Research results indicate the need for a greater transparency regarding the Council’s PR processes in order to improve practices and expand social control over allocation of public resources.

No description available.

The measurement of research output is common practice among public institutions internationally, and is increasingly contested and controversial. The term “research” is itself contested and can cover quite a wide range of activities, from carefully designed studies by independent, university-based researchers to analysis of data for particular administrative or political purposes to arguments for specific policy positions that may be more or less well grounded in evidence. Such measurement of research output is needed for decisions about professional staff and resource allocations. Measures of research productivity, covering both quantity and quality at national level, support government decisions on setting priorities and funding. With increasing competitive allocation of research funding and declining public funds for higher education, institutions around the world are facing increasing pressure to produce research outputs. The revenue generated through published research has therefore come to assume greater and greater significance in institutional budgets and in academic reward systems. Moreover, research in public institutions is funded mainly according to the number and quality of publications of members of staff. On the other hand, the growing international trend towards ranking institutions in competitive terms has assigned considerable value to research output as a measure of institutional standing in the global marketplace. What counts as an acceptable unit of measurement therefore becomes the subject of considerable debate within and outside institutions as they seek to enhance institutional standing and revenue. Whilst measurable output such as scientific publications and research reports are usually considered for government subsidy, it is difficult to accept that other output types such as patents, software, advisory work for government, consulting, or technical assistance, are not measurable, and do not have any relevance with respect to research subsidy. This thesis was set out to critically examine the effects that current government policy on the measurement of research output of public higher education institutions will have on the performance of South African Higher Education Institutions (HEIs). The purpose of this study was to trace and explain the differential impact of new government policies on the measurement of institutional research output in four different university faculties. The study has highlighted key challenges facing the universities in implementing the new research subsidy policy; and made recommendations and proposals on how best can the policy be implemented with the view of increasing or improving the institutions’ research output. / Dissertation (M.Ed (Education Management, Law and Policy))--University of Pretoria, 2007. / Education Management and Policy Studies / unrestricted

Multi-rater tools also referred to as 360-degree feedback tools, are frequently used in addition to traditional supervisory appraisals due to sources (i.e., supervisor, peer, direct report) unique perspectives and opportunities to view different aspects of job performance. Research has found that the differences among sources are most prevalent between self and other ratings, and the direction of agreement is related to overall job performance. Research has typically focused on one form of agreement, the direction of an individual's self-ratings compared to others' ratings. The current study expanded on past research on rater agreement using a data set (n = 215) consisting of multi-rater data for professionals participating in a leadership development process. The study examined the ability to predict job performance with three different measures of self-other agreement (i.e., difference between overall mean scores (difference), mean absolute difference across items (difference), and mean correlation across items (similarity)). The study also examined how the relationships may differ across performance dimensions. The final purpose was to explore how the importance of the performance dimensions, as rated by the participant, may moderate the relationship between self-other agreement and job performance. Partial support for study's hypotheses was found. The direction and difference measures of agreement on the overall multi-rater tool and performance dimensions accounted for a significant amount of the variance in job performance. The relationship between the similarity measure of agreement and job performance, and the moderating effect of importance were not supported in the current study.

This study reports on an investigation to determine the applicability of the peer rating technique to Air Force ROTC cadets at North Texas State College which has an enrollment of approximately 7000 students. The specific problem investigated was whether or not the peer rating would be useful in solving the leader identification problem in Air Force ROTC.

The African Peer Review Mechanism (APRM) was developed under the New Partnership for Africa’s Development (NEPAD), the development framework of the African Union (AU) which replaced the Organization of African Unity (OAU) in 2002. Through the APRM, the AU has established a system for assessment of governance in participating countries and for the development of programmes of action to address identified shortcomings. The APRM is a voluntary, ‘soft’ mechanism of supervision which combines self-assessment with regional monitoring. The APRM takes a holistic approach to governance with a mandate covering democracy and political governance, economic governance, corporate governance and socio-economic development. This study considers the role that the APRM plays in the realisation of human rights. It examines the manner in which human rights are reflected in the APRM framework documents as well as the manner in which rights-based principles such as participation, accountability and transparency are reflected in the process. The strengths and weaknesses of various methods of international monitoring to ensure compliance with human rights are examined. The APRM country review reports and implementation reports of Ghana, Rwanda and Kenya are studied in conjunction with reports from domestic and international human rights monitoring bodies and national development plans. The aim of the study is to ascertain whether the APRM adds value to mechanisms established with the purport of assisting in the realisation of human rights. This study illustrates that the APRM plays a complementary role in human rights monitoring. It is clear, however, that it is only able to play a meaningful role if the state under review is motivated to undertake reform. Human rights have a role to play with regard to the APRM process itself and in identifying and addressing governance shortcomings. The specific and time-bound commitments in the Programme of Action are unique to the APRM. If these commitments are developed through a rights-based approach and their implementation adequately monitored the APRM could play an important role in inducing compliance with human rights. / Thesis (LLD)--University of Pretoria, 2010. / Centre for Human Rights / Unrestricted

Esta tese visa analisar a constituição, o funcionamento e os limites da Avaliação por Pares (AP) no âmbito do Conselho Nacional de Desenvolvimento Científico e Tecnológico (CNPq). A pesquisa operou por dois caminhos distintos: um diacrônico e outro sincrônico. No primeiro, elaborou-se uma reconstrução histórica da AP praticada no CNPq entre 1951 e 2015 contextualizada pelas mudanças institucionais a que passou o órgão durante o citado período. O segundo caminho centrou-se na análise de dois estudos de caso que redundaram na elaboração de perfis dos Bolsistas de Produtividade em Pesquisa de Filosofia e Educação com bolsas vigentes no ano de 2016. Do um ponto de vista teórico, os resultados alcançados foram interpretados segundo as noções de Campo Científico de Pierre Bourdieu e Reconhecimento de Axel Honneth. Quanto à história da AP no CNPq constatou-se uma ligação entre o crescimento na quantidade de bolsas implementadas pelo órgão e uma maior formalização legal e institucional da AP, especialmente a partir da década de 1970. O próprio crescimento da comunidade científica brasileira possibilitou uma transição de procedimentos pessoais e informais de AP para um rígido, formal e impessoal circuito de imposições legais quanto a AP praticada no órgão. Quanto aos dois estudos de caso constatou-se que apesar de todas as restrições formais que atualmente regulam a AP no Conselho, especificidades históricas, geográficas, econômicas e relativas a gênero acabam por influir nos resultados dos processos de AP. Com base na pesquisa realizada, verificou-se a necessidade em se instituir uma maior transparência nos processos de AP no órgão de modo a se coadunar um aperfeiçoamento da própria prática da AP assim como um controle social da destinação dos recursos públicos mais democrático e acessível a sociedade civil. / This theses analyzes the institution, the mechanisms and the limits of peer review (PR) in Brazil’s National Council for Technological and Scientific Development (CNPq). Research was conducted at both diachronic and synchronic levels. At the diachronic level, I accomplished a historical reconstruction of PR practices in CNPq from 1975 to 2015, linking this reconstruction to the institutional changes undergone by the Council during this period. At the synchronic level, I focused on two case studies, which allowed me to outline the profile of researchers in Philosophy and Education with productivity grants (in 2016). Research findings were interpreted with the help of Pierre Bourdieu’s notion of Scientific Field and Axel Honneth’s idea of Recognition. As for the history of PR in CNPq, research results established a correlation between the increase in grants funded and the legal and institutional formalization of PR, particularly since the 1970s. The growth of the Brazilian scientific community entailed a shift from informal and personal PR procedures to the rigid, formal and impersonal mediating rules now in place. As for the two case studies, I found that despite all formal restrictions now governing the Council’s PR practices, historical, geographical, economical and gender based specificities still have an influence on the outcome of PR processes. Research results indicate the need for a greater transparency regarding the Council’s PR processes in order to improve practices and expand social control over allocation of public resources.

Le système d’évaluation par les pairs est le gold standard de la publication scientifique. Ce système a deux objectifs: d’une part filtrer les articles scientifiques erronés ou non pertinents et d’autre part améliorer la qualité de ceux jugés dignes de publication. Le rôle des revues scientifiques et des rédacteurs en chef est de veiller à ce que des connaissances scientifiques valides soient diffusées auprès des scientifiques concernés et du public. Cependant, le système d’évaluation par les pairs a récemment été critiqué comme étant intenable sur le long terme, inefficace et cause de délais de publication des résultats scientifiques. Dans ce projet de doctorat, j’ai utilisé une modélisation par systèmes complexes pour étudier le comportement macroscopique des systèmes de publication et d’évaluation par les pairs. Dans un premier projet, j’ai modélisé des données empiriques provenant de diverses sources comme Pubmed et Publons pour évaluer la viabilité du système. Je montre que l’offre dépasse de 15% à 249% la demande d’évaluation par les pairs et, par conséquent, le système est durable en termes de volume. Cependant, 20% des chercheurs effectuent 69% à 94% des revues d’articles, ce qui souligne un déséquilibre significatif en termes d’efforts de la communauté scientifique. Les résultats ont permis de réfuter la croyance largement répandue selon laquelle la demande d’évaluation par les pairs dépasse largement l’offre mais ont montré que la majorité des chercheurs ne contribue pas réellement au processus. Dans mon deuxième projet, j’ai développé un modèle par agents à grande échelle qui imite le comportement du système classique d’évaluation par les pairs, et que j’ai calibré avec des données empiriques du domaine biomédical. En utilisant ce modèle comme base pour mon troisième projet, j’ai modélisé cinq systèmes alternatifs d’évaluation par les pairs et évalué leurs performances par rapport au système conventionnel en termes d’efficacité de la revue, de temps passé à évaluer des manuscrits et de diffusion de l’information scientifique. Dans mes simulations, les deux systèmes alternatifs dans lesquels les scientifiques partagent les commentaires sur leurs manuscrits rejetés avec les éditeurs du prochain journal auquel ils les soumettent ont des performances similaires au système classique en termes d’efficacité de la revue. Le temps total consacré par la communauté scientifique à l’évaluation des articles est cependant réduit d’environ 63%. En ce qui concerne la dissémination scientifique, le temps total de la première soumission jusqu’à la publication est diminué d’environ 47% et ces systèmes permettent de diffuser entre 10% et 36% plus d’informations scientifiques que le système conventionnel. Enfin, le modèle par agents développé peut être utilisé pour simuler d’autres systèmes d’évaluation par les pairs ou des interventions, pour ainsi déterminer les interventions ou modifications les plus prometteuses qui pourraient être ensuite testées par des études expérimentales en vie réelle. / The peer-review system is undoubtedly the gold standard of scientific publication. Peer review serves a two-fold purpose; to screen out of publication articles containing incorrect or irrelevant science and to improve the quality of the ones deemed suitable for publication. Moreover, the role of the scientific journals and editors is to ensure that valid scientific knowledge is disseminated to the appropriate target group of scientists and to the public. However, the peer review system has recently been criticized, in that it is unsustainable, inefficient and slows down publication. In this PhD thesis, I used complex-systems modeling to study the macroscopic behavior of the scientific publication and peer-review systems. In my first project, I modeled empirical data from various sources, such as Pubmed and Publons, to assess the sustainability of the system. I showed that the potential supply has been exceeding the demand for peer review by 15% to 249% and thus, the system is sustainable in terms of volume. However, 20% of researchers have been performing 69% to 94% of the annual reviews, which emphasizes a significant imbalance in terms of effort by the scientific community. The results provided evidence contrary to the widely-adopted, but untested belief, that the demand for peer review over-exceeds the supply, and they indicated that the majority of researchers do not contribute to the process. In my second project, I developed a large-scale agent-based model, which mimicked the behavior of the conventional peer-review system. This model was calibrated with empirical data from the biomedical domain. Using this model as a base for my third project, I developed and assessed the performance of five alternative peer-review systems by measuring peer-review efficiency, reviewer effort and scientific dissemination as compared to the conventional system. In my simulations, two alternative systems, in which scientists shared past reviews of their rejected manuscripts with the editors of the next journal to which they submitted, performed equally or sometimes better in terms of peer-review efficiency. They also each reduced the overall reviewer effort by ~63%. In terms of scientific dissemination, they decreased the median time from first submission until publication by ~47% and diffused on average 10% to 36% more scientific information (i.e., manuscript intrinsic quality x journal impact factor) than the conventional system. Finally, my agent-based model may be an approach to simulate alternative peer-review systems (or interventions), find those that are the most promising and aid decisions about which systems may be introduced into real-world trials.

No description available.

Pesquisas têm indicado que o diálogo colaborativo poderia ser o lugar onde o uso da língua e a aprendizagem da língua poderiam coocorrer (SWAIN, 1997), tornando-se uma atividade social e cognitiva (SWAIN, 2000). A prática de revisão por pares proporciona a seus participantes um tipo de aprendizagem colaborativa, com foco na escrita, que permite a interação entre os alunos e que os leva à co-construção de conhecimento. Ela tem como objetivo auxiliar no desenvolvimento da autonomia dos aprendizes, levando-os de um estágio de dependência para outros estágios de inter e independência, em um processo contínuo de tomada de controle para a realização de seus objetivos. Unindo seus conhecimentos, os aprendizes, ao trabalharem juntos, alcançam níveis mais elevados de desempenho, através da assistência mútua (OHTA, 2000). Assim, esta pesquisa teve por objetivo propor a prática de revisão por pares de produção escrita a um 8º ano do Ensino Fundamental de uma escola com currículo bilíngue Português-Inglês do sul do Brasil. Ela visou a analisar a atividade colaborativa em que os participantes se envolveram enquanto estavam engajados na atividade de revisão por pares, destacando os efeitos que essa revisão apresentou nas versões finais dos textos e, para tal, os dados foram analisados de forma quantitativa e qualitativa. A geração dos dados ocorreu durante o segundo trimestre letivo nas aulas de Inglês Aplicado, em que os alunos produziram individualmente e revisaram colaborativamente cinco textos cada. As propostas para a produção dos textos estavam de acordo com as diretrizes do exame de Cambridge FCE for Schools, que os alunos participantes prestariam no ano seguinte, e envolveram gêneros textuais variados. As expectativas dos aprendizes antes da prática, bem como sua avaliação depois das sessões de revisão foram analisadas Os resultados apontam para a receptividade dos participantes a essa prática pedagógica e para o trabalho colaborativo, e demonstram o otimismo dos aprendizes em relação à possibilidade de melhoria dos seus textos após a revisão do colega. Em relação aos textos produzidos, foi feito o levantamento de quais aspectos linguísticos e textuais foram abordados nas interações através das categorias apresentadas por Storch (2005). Também se buscou investigar se as sugestões dadas pelos revisores foram benéficas para a escrita de uma segunda versão melhor do texto e se as sugestões recebidas foram incorporadas à nova versão do texto. Foi possível observar que, nos Episódios Relacionados à Língua (ERL, SWAIN, 2002), foram discutidos diferentes aspectos linguísticos e estruturais dos textos, focando tanto na forma quanto no conteúdo Os dados sugerem que a maioria dos ERLs foram resolvidos e resolvidos corretamente, impactando positivamente na reescrita dos textos e suportando evidência empírica já existente nesse sentido (NIU, 2009; STORCH, 2001; STORCH e ALDOSARI, 2013). Da mesma forma, a maioria das sugestões foi incorporada pelos escritores quando reescreveram seus textos. Por fim, a interação entre os participantes e os padrões de interação desenvolvidos foram analisados, observando se houve andaimento, negociação, mediação e troca de papéis, e as duplas foram situadas em um continuum, desenvolvido para a análise desses dados, que vai do mais conflitivo para o mais colaborativo. Os resultados apontam que mesmo em pares em que os aprendizes demonstram habilidade linguística semelhante na L2 e não seja possível identificar um indivíduo mais experiente, os participantes foram capazes de oferecer andaimento e engajar-se em negociação e em um processo colaborativo de aprendizagem. Esta pesquisa pretende contribuir com professores ao evidenciar que a colaboração entre aprendizes é um caminho para que a aprendizagem de L2 aconteça, mantendo-os motivados, indo ao encontro de seus interesses e desenvolvendo sua autonomia como aprendizes de L2. / Research has shown that collaborative dialogue could be where language use and language learning would co-occur (SWAIN, 1997) becoming a cognitive and social activity (SWAIN, 2000). The peer review practice engages its participants in a collaborative type of learning, focused on writing, which allows the interaction among students and leads them to coconstruct knowledge. Its objective is to help develop learners’ autonomy, taking them from a stage of dependence to others of inter- and independence, in a continuous process of gaining greater control to reach their objectives. When working together, learners achieve greater levels of performance through shared knowledge and mutual assistance (OHTA, 2000). Therefore, this research had the objective of implementing the practice of writing peer review in a Portuguese-English Bilingual school in the South of Brazil with students from the 8th year of Elementary School. It aimed at analyzing the collaborative activity students engaged in while peer reviewing, highlighting the impact the reviews had on the final versions of the texts. For this purpose, the data were analyzed quantitatively and qualitatively. Data were generated during the second term of a school year in the Applied English class, when students individually wrote and collaboratively reviewed five texts each. The writing tasks were based on the genres tested in the FCE for schools Cambridge proficiency exam, which students would take on the following year. The participants’ expectations before the practice, as well as their evaluation after the peer review sessions were analyzed Results show that participants welcome the opportunity of working collaboratively and show optimism regarding the improvement of their texts. In relation to the written texts, the contents of suggestions approached during interactions were listed and categorized according to Storch (2005). The suggestions were also analyzed regarding their validity and take-up rates were calculated. It was possible to observe that different linguistic aspects and text structure features were discussed in the Language Related Episodes (LRE, SWAIN, 2002) focusing both on form and content. Data suggest that most LREs were resolved and resolved correctly, having a positive impact on the second version of the text, supporting already existing empirical evidence (NIU, 2009; STORCH, 2001; STORCH and ALDOSARI, 2013). Similarly, most suggestions were incorporated by writers when reviewing their texts. Finally, the participants’ interactions and the interaction patterns they presented were analyzed, with the aim of observing if there was scaffolding, negotiation, mediation and changing of roles. Pairs were situated in a continuum developed for data analysis. This continuum goes from the most conflictive to the most collaborative behavior. Results indicate that even when the members of the pair show similar L2 language ability and there is no identifiable expert, participants were able to scaffold and engage in negotiation in a collaborative learning process. This research intends to contribute to the language teaching professionals by showing evidences that peer collaboration is key to L2 language learning, keeping students motivated and developing their autonomy as language learners.