No description available.

Part I. Claims to knowledge require justification. In science, such justification is made possible by the ability to reproduce or replicate experiments, thereby confirming their validity. Additionally, reproducibility serves as a self-correcting tool in science as it weeds out faulty experiments. It is therefore essential that experimental studies be replicated and confirmed. Recently, attempts to reproduce studies in several fields have failed, leading to what has been referred to as "a crisis of reproducibility." This crisis is largely a result of the current culture in the scientific world. Specifically, it is a result of a system that incentivizes individual success in the form of publications in high-impact journals over collaboration and careful conductance of research. This environment contributes to the crisis of reproducibility by increasing biases, incentivizing researchers to engage in manipulative statistics, decreasing quality control and transparency, and increasing the likelihood of researchers engaging in fraudulent behavior. Possible solutions to the problem of irreproducibility could tackle individual factors. A more prudent approach would be to focus on changing the current culture in the scientific world. Increased transparency had been suggested as a way to solve this problem. There is currently a movement advocating for increased transparency in science through "open science." Part II. Retraction of scientific papers due to evidence of research misconduct is on the rise, having increased tenfold from 2000 to 2009. Previous work on this topic focused on published retraction notices, using notices to identify the percent of retracted articles that were caused by research misconduct. This study utilized a different approach. Using the Office of Research Integrity database, we first identified publications that resulted from research misconduct. We then searched those articles to determine whether they were indeed retracted. Once retraction notices were identified, they were scored based on scoring elements reflecting guidelines for transparency. Lastly, we investigated whether a correlation exists between the quality of a retraction notice and journal impact factor. Our findings suggest that 21% of papers containing data derived from scientific misconduct are not retracted. Moreover, the quality of retraction notices varies, with some elements more likely to be present than others. No significant correlation between retraction notices and journal impact factor was found.

Experimental psychology is said to be having a reproducibility crisis, marked by a low rate of successful replication. Researchers attempting to respond to the problem lack a framework for consistently interpreting the results of statistical tests, as well as standards for judging the outcomes of replication studies. In this paper I introduce an error-statistical framework for addressing these issues. I demonstrate how the severity requirement (and the associated severity construal of test results) can be used to avoid fallacious inferences that are complicit in the perpetuation of unreliable results. Researchers, I argue, must probe for error beyond the statistical level if they want to support substantive hypotheses. I then suggest how severity reasoning can be used to address standing questions about the interpretation of replication results. / Master of Arts

The general topic of this Ph.D. thesis regards "Questionable Research Practices" (QRPs) and their effect on the replicability crisis in psychology and social sciences. QRPs are a set of bad research practices carried out to increase the likelihood of obtaining a significant result by providing some sort of manipulation of collected data. QRPs are related to the so-called replicability crisis, defined as the inability to replicate the results of an increasing number of studies. The proposed dissertation focuses on a specific form of QRPs named "Questionable Interim Analysis" (QIA). QIA implies that the data collection stopping rule depends on the previous interim analysis result. While the effect of this practice on the growth of false positive results is well known, its impact on the replicability of studies is still unclear. Furthermore, an objective tool to assess the incidence of this practice in a single study is, to our knowledge, still missing. In this project, we tried to overcome these limitations by evaluating the effects of QRPs on replication success and by proposing a formal tool to resize and control the QIA effects on statistical results. This document begins with a brief overview on the replicability crisis, QRPs, and the methods for identifying these practices in the literature (Chapter 1). Chapter 2 introduces a methodology to clarify the effect of some QRPs (cherry picking, questionable interim analyses, questionable inclusion of covariates, and questionable subgroup analyses) on replicability and determinates which replication success metric performs better in presence of these practices. The results show that the metric based on the golden sceptical p-value maintains low values of false positive replication success when QRPs are assumed. Chapter 3 presents a new probabilistic model to identify the incidence of QIA practice in the literature. It provides information about the larger or lower probability of this practice by estimating the ratio between the probability that a published result was obtained on the basis of a QIA practice or, alternatively, by a standard data collection approach. In Chapter 4, we show an application of the probabilistic models in the context of simulation studies. In particular, we illustrate how the new proposal improves some widely used simulation approaches to assess the false positive results in the context of QIA practices. Finally, in Chapter 5 the proposed models are discussed in terms of their utilities, limitations, and possible future extensions.

The aim of this study was to test the applicability of a replication through a publishedresearch article choosing constructs that were of interest to the authors. The one chosen wasa 2020 article from Orgambídez et al., which investigated a model derived from the jobdemands and resources model (JD-R) and Quality of Working Life (QWL) in Spain with a sample of nurses. Our study followed the principles of Orgambídez et al. (2020) of a cross-sectional correlational design but with a Swedish sample in the IT, software and technology field, with 101 participants. Correlational analysis, confirmatory factor analysis and pathanalysis were utilized to test four hypotheses and the mediation effect of work engagementbetween self-efficacy and job satisfaction. Results confirmed all four hypotheses includingthe mediation effect. Opposite to the findings of Orgambídez et al. (2020) though, there wasno direct effect found of self-efficacy on job satisfaction.

Yes / To date, open science, and particularly open data, in Psychology, has focused on quantitative research. This paper aims to explore ethical and practical issues encountered by UK-based psychologists utilising open qualitative datasets. Semi-structured telephone interviews with eight qualitative psychologists were explored using a framework analysis. From the findings, we offer a context-consent meta-framework as a resource to help in the design of studies sharing their data and/or studies using open data. We recommend ‘secondary’ studies conduct archaeologies of context and consent to examine if the data available is suitable for their research questions. This research is the first we know of in the study of ‘doing’ (or not doing) open science, which could be repeated to develop a longitudinal picture or complemented with additional approaches, such as observational studies of how context and consent are negotiated in pre-registered studies and open data. / The author's manuscript has a slightly different title from the published article: A meta-framework for designing open data studies in psychology: ethical and practical issues of open qualitative data sets

<p>This dissertation studies the impact of mutual fund managers' beliefs on fund performance in the first chapter and focuses on the impact of firm behavior on stock performance in the second chapter. </p> <p><br></p> <p>In the first chapter, I utilize the Revealed Preference Theory to recover fund manager belief formation directly from their actual trading activities. By relating stock holdings in a fund's portfolio to past factor returns, I document three facts about managers' belief formation: 1. In contrast to belief extrapolation, a substantial fraction of mutual fund managers act as contrarian investors who expect lower factor returns after a good factor performance; 2. Whether a fund trades in an extrapolative or contrarian way is due to its managers' expectation biases rather than fund style investment strategy, fund catering strategy or fund risk preference; and 3. Contrarian managers generate superior performance, are more experienced investors, charge higher expense ratios, and manage smaller US equity funds. The top (contrarian) managers significantly outperform the bottom (extrapolative) managers by a return of 3.4% per annum after adjusting by FFC4 factor models.</p> <p><br></p> <p>The second chapter is co-authored paper with Yan Liu and Feng Zhang. In this chapter, we systematically replicate the bulk of long-run event studies conducted in the last three decades from 1990 to 2020 using extended samples and four long-run performance measures. The final sample contains 62 papers of long-run event studies and 148 corporate events. Our findings suggest that long-run return anomalies documented in the last three decades are not robust, and firms do not earn long-run abnormal returns following various types of corporate events. Only 2% of the 148 corporate events we replicate earn post-event abnormal returns that are statistically significant at the 5% level based on all the four performance measures, and the fraction further shrinks to 0% at the 1% significance level. Viewed together, our findings suggest that these long-run abnormal returns after corporate events are likely the result of data mining or "p-hacking".</p>

Background: Objectives: To produce and synthesize reliable data, systematic reviews need to adhere to rigorous methodological standards. This living meta-review aims to investigate risk of bias and open science practices in systematic reviews from the educational field published between 2022-2023. The aim of this meta-review is to get a better understanding of the current state of educational research regarding the aforementioned topics.  Methods: Eligibility criteria: We included all systematic reviews of educational interventions, instructions, and methods for all K-12 student populations with experimental or quasi-experimental designs where the outcome variables were academic performance of any kind.  Information sources: We searched through the Education Resources Information Centre (ERIC) for systematic reviews published between 2022-2023. In addition, we also hand searched four scholarly databases. Risk of Bias and FAIR principles: To assess systematic reviews risk of bias and open science practices in systematic reviews two tools were used, ROBIS - Risk of bias in systematic reviews, as well as a FAIR assessment tool.  Results: 44 studies that matched our PICOS were included in this meta-review. Out of these studies four (9%) were deemed as having a low risk of bias. The remaining 40 studies were deemed as either having high risk of bias (89%) or unclear risk of bias (2%).  Among the 44 studies included, only four studies (9%) had their data available, and none of them adhered to all of the items regarding the FAIR principles.  Discussion: This meta-review shows that only a small part of systematic reviews in education can be considered low risk of bias, and an even smaller part can be considered adhering to open science principles. Therefore, much needs to change to adapt to new scientific guidelines.

Esta tese trata do desenvolvimento de materiais de apoio para o ensino de leitura em inglês em cursos técnicos e tecnológicos do Brasil, especialmente no âmbito da formação em Eletrotécnica e Engenharia Elétrica. Os objetivos do trabalho são dois: a) apresentar a Terminografia Didático-Pedagógica (TD-P), uma metodologia inédita para desenvolvimento de materiais didáticos para professores de ESP (English for Specific Purposes - Inglês para Fins Específicos); b) verificar, junto a professores, se a metodologia é útil e replicável. Essa metodologia baseia-se nos preceitos teóricos do ensino do ESP, da Terminologia de perspectiva Textual, da Teoria Sócio-Cognitiva da Terminologia e da Linguística de Corpus. Seguindo o trajeto da metodologia da TD-P, extraíram-se dados lexicais e gramaticais de um corpus composto pelo gênero textual relevante ao ambiente de ensino da Eletrotécnica e Engenharia Elétrica, a fim de construir um protótipo de um glossário idealizado para o trabalho com a área especializada em questão. Para tanto, primeiramente, compilaram-se 30 datasheets de 11 componentes elétricos, somando 21.467 tokens. A coleta de candidatos a termos se deu através das ferramentas AntConc (ANTHONY, 2004) e TermoStat (DROUIN, 2003). Procedeu-se então ao levantamento de necessidades dos aprendizes através da distribuição dos mesmos datasheets a 108 alunos de cursos técnicos e de graduação da área, os quais apontaram as construções lexicais que lhes causavam dúvidas. As informações obtidas foram contrastadas e cotejadas com a análise de um especialista da área de aplicação. Os dados obtidos com a comparação das coletas resultaram em diferenças específicas que auxiliaram a desenvolver o protótipo de ferramenta terminográfica, o GlossElectric. Para a verificação da opinião de outros profissionais sobre a replicabilidade da metodologia da TD-P, foi produzido um vídeo sobre a mesma, e este foi distribuído juntamente com um questionário a 11 professores especialistas em ensino de Inglês para fins específicos. Os resultados da verificação da metodologia apontam para um grau satisfatório de replicabilidade, com uma média de 84% de participantes considerando a metodologia altamente reproduzível em seu ambiente de trabalho. / The current thesis concerns the development of support material for the teaching of reading skills in English in technical courses in Brazil, especially regarding the areas of Electrotechnical/Electrical Engineering. There are two aims for this work: a) introducing the Didatic-Pedagogical Terminography (TD-P), a methodology for developing and building didatic material for ESP (English for Specific Purposes) teachers and professors; b) verify, among ESP professors, if they consider the methodology useful and replicable in other areas. This methodology is based on the theoretical precepts of the teaching of ESP, the Terminology of Textual Perspective, the Socio-Cognitive Theory of Terminology and Corpus Linguistics. Following the methodology’s steps, lexical and gramatical data was extracted from a corpus composed by the textual genre relevant to the teaching environment of Electrotechnical/Electrical Engineering, with the aim of building a prototype of an idealized glossary, in order to work with the mentioned technical area. 30 datasheets from 11 diffrent electrical components were compiled, with the total of 21.467 tokens. The term extraction was carried out using the tools AntConc (ANTHONY, 2004) and TermoStat (DROUIN, 2003). After that, there was a data collection to determine the needs of the learners of the technical area, who need to deal with the datasheets during their professional preparation. The same datasheets used on the term extraction were distributed to 108 students of technical and graduation courses in the Electrotechnical/Electrical Engineering domain. These students were asked to point out the lexical features that caused doubts regarding meaning. The data obtained was contrasted and submitted to the analysis of a specilist in the technical domain. The data obtained from the crossing resulted in specific differences which helped to develop the prototype of the terminographic tool, named GlossElectric. In order to verify the opinion of other professionals regarding the replicability of the TD-P methodology, a video was produced and sent together with a questionnaire to 11 professors who were specialised in ESP. The results of the methodology verification point to a satisfactory degree of replicability, with an average of 84% of participants considering the methodology highly reproducible in their work environment.

Esta tese trata do desenvolvimento de materiais de apoio para o ensino de leitura em inglês em cursos técnicos e tecnológicos do Brasil, especialmente no âmbito da formação em Eletrotécnica e Engenharia Elétrica. Os objetivos do trabalho são dois: a) apresentar a Terminografia Didático-Pedagógica (TD-P), uma metodologia inédita para desenvolvimento de materiais didáticos para professores de ESP (English for Specific Purposes - Inglês para Fins Específicos); b) verificar, junto a professores, se a metodologia é útil e replicável. Essa metodologia baseia-se nos preceitos teóricos do ensino do ESP, da Terminologia de perspectiva Textual, da Teoria Sócio-Cognitiva da Terminologia e da Linguística de Corpus. Seguindo o trajeto da metodologia da TD-P, extraíram-se dados lexicais e gramaticais de um corpus composto pelo gênero textual relevante ao ambiente de ensino da Eletrotécnica e Engenharia Elétrica, a fim de construir um protótipo de um glossário idealizado para o trabalho com a área especializada em questão. Para tanto, primeiramente, compilaram-se 30 datasheets de 11 componentes elétricos, somando 21.467 tokens. A coleta de candidatos a termos se deu através das ferramentas AntConc (ANTHONY, 2004) e TermoStat (DROUIN, 2003). Procedeu-se então ao levantamento de necessidades dos aprendizes através da distribuição dos mesmos datasheets a 108 alunos de cursos técnicos e de graduação da área, os quais apontaram as construções lexicais que lhes causavam dúvidas. As informações obtidas foram contrastadas e cotejadas com a análise de um especialista da área de aplicação. Os dados obtidos com a comparação das coletas resultaram em diferenças específicas que auxiliaram a desenvolver o protótipo de ferramenta terminográfica, o GlossElectric. Para a verificação da opinião de outros profissionais sobre a replicabilidade da metodologia da TD-P, foi produzido um vídeo sobre a mesma, e este foi distribuído juntamente com um questionário a 11 professores especialistas em ensino de Inglês para fins específicos. Os resultados da verificação da metodologia apontam para um grau satisfatório de replicabilidade, com uma média de 84% de participantes considerando a metodologia altamente reproduzível em seu ambiente de trabalho. / The current thesis concerns the development of support material for the teaching of reading skills in English in technical courses in Brazil, especially regarding the areas of Electrotechnical/Electrical Engineering. There are two aims for this work: a) introducing the Didatic-Pedagogical Terminography (TD-P), a methodology for developing and building didatic material for ESP (English for Specific Purposes) teachers and professors; b) verify, among ESP professors, if they consider the methodology useful and replicable in other areas. This methodology is based on the theoretical precepts of the teaching of ESP, the Terminology of Textual Perspective, the Socio-Cognitive Theory of Terminology and Corpus Linguistics. Following the methodology’s steps, lexical and gramatical data was extracted from a corpus composed by the textual genre relevant to the teaching environment of Electrotechnical/Electrical Engineering, with the aim of building a prototype of an idealized glossary, in order to work with the mentioned technical area. 30 datasheets from 11 diffrent electrical components were compiled, with the total of 21.467 tokens. The term extraction was carried out using the tools AntConc (ANTHONY, 2004) and TermoStat (DROUIN, 2003). After that, there was a data collection to determine the needs of the learners of the technical area, who need to deal with the datasheets during their professional preparation. The same datasheets used on the term extraction were distributed to 108 students of technical and graduation courses in the Electrotechnical/Electrical Engineering domain. These students were asked to point out the lexical features that caused doubts regarding meaning. The data obtained was contrasted and submitted to the analysis of a specilist in the technical domain. The data obtained from the crossing resulted in specific differences which helped to develop the prototype of the terminographic tool, named GlossElectric. In order to verify the opinion of other professionals regarding the replicability of the TD-P methodology, a video was produced and sent together with a questionnaire to 11 professors who were specialised in ESP. The results of the methodology verification point to a satisfactory degree of replicability, with an average of 84% of participants considering the methodology highly reproducible in their work environment.