Views of a non-probability sample of corresponding authors with retracted publications in biomedical fields about the impact of different types of retractions on researchers’ careers

Abstract

Echoing Arturo Casadevall and Ferric Fang in their Reforming Science: Methodological and Cultural Reforms, “great human enterprises must undergo periodic cycles of self-examination and renewal to maintain their vigor”. Especially in the last decade, the research culture has undergone such cycles, partially driven by countercultural transformations that have been reshaping assumptions towards reward-deserving achievements. Addressing retractions is among the challenges in this culture. This work builds upon research carried out at the University of California, San Diego (UCSD), which explored the views of 224 reviewers serving on panels for the US National Science Foundation, the National Institutes of Health, among others. We show results of a survey that add to our previous data. It was sent to a population of 1,089 corresponding authors affiliated with institutions from the 20 most productive countries in biomedical fields. We explored how corresponding authors of at least one retracted publication issued between 2013 and 2015 in biomedical journals envisioned the impact of different types of retractions on the careers of the first and corresponding authors. As such impact (if any) is not always immediate, we selected this time frame to ensure that potential respondents would have tangible post-retraction experience.

Key words correction of the literature; research assessment; retractions; research integrity; funding

INTRODUCTION

It is a given that the prevailing research culture is entrenched in traditional views of impact and recognition, relying mostly on original research published in top-tier journals for certifying knowledge and rewarding scientists. This culture is reflected, for example, in scientific awards or funding announcements. It is common that outstanding research findings and achievements are among the main criteria or expectations. Announcing original, novel, outstanding, and even revolutionary research is incorporated in the language of authors in scientific journals (Ball 2015) and applicants in grant proposals, revealing much of how research systems¹ feed into this culture. Millar et al. (2022) found an increase in the prevalence of using promotional language from 1985 to 2020 for successful grant applications to the National Institutes of Health. This result resonates with what Vinkers et al. (2015, p. 3) call “outcome positive bias”. This type of bias is part of the mainstream culture of writing scientific abstracts, according to their lexicographic analysis. The same authors found an increase from 2.0% (1974-80) to 17.5% (2014), regarding the absolute frequency of positive words, with a relative increase of 880% (Vinkers et al. 2015). Among these words conveying positive bias are “assuring, astonishing, bright, creative, encouraging, enormous, excellent… groundbreaking… innovative, inspiring, inventive, novel, phenomenal, prominent, promising… remarkable, robust, spectacular, supportive… unique” (Vinkers et al. 2015, p. 3).

However, echoing Casadevall & Fang’s (2012, p. 891) observation, “[h]istory teaches us that most, if not all, great human enterprises must undergo periodic cycles of self-examination and renewal to maintain their vigor”. Especially in the last decade, the research culture has undergone such cycles, partially driven by countercultural transformations that have been reshaping assumptions towards sound research and reward-deserving achievements. There have been stronger calls for a sharper focus on research integrity, openness, transparency, reproducibility, and credibility (Casadevall & Fang 2012, Ioannidis 2014, Nosek & Lakens 2014, Smaldino & McElreath 2016, Gernsbacher 2018, Soderberg et al. 2021), regardless of the “seal of excellence” of high-impact-factor (IF) journals.

The Hong Kong Principles (HKP) (Moher et al. 2020) illustrate this changing landscape and suggest a turning point for research systems, with transformations in the hard core of indicators traditionally used to assess research performance and career advancement. Each principle adds responsibilities to researchers and policymakers evolving in a research endeavor tightly knotted to new contracts between science and society that demand more transparency and open science practices. These practices align with Gibbons’ (1999, C81) reasoning that society expects science to do more than just produce “reliable knowledge”, communicating its findings as part of the engagement with the public. The HKPs (Box 1) resonate with Gibbons (1999). They both built on the need for “socially robust” scientific knowledge for which transparency should be combined with a participatory mode of interacting with the public.

Box 1. The Hong Kong Principles for assessing researchers. Source: Moher et al. 2020.

Principle 1: Assess researchers on responsible practices from conception to delivery, including the development of the research idea, research design, methodology, execution, and effective dissemination.
Principle 2: Value the accurate and transparent reporting of all research, regardless of the results.
Principle 3: Value the practices of open science (open research)—such as open methods, materials, and data.
Principle 4: Value a broad range of research and scholarship, such as replication, innovation, translation, synthesis, and meta-research.
Principle 5: Value a range of other contributions to responsible research and scholarly activity, such as peer review for grants and publications, mentoring, outreach, and knowledge exchange.

These principles reflect an array of changes that include reshaping the role of the research paper (Shanahan 2015) and all the symbolic capital, in Bordieuan terms, attached to it along the history of science. Traditionally, this type of capital has not been conceived explicitly as related to rigorous, trustworthy research, as such constructs have been taken for granted in the research process. Even in seminal discussions of the Matthew Effect (Merton 1968) and then of the Matilda’s Effect (Rossiter 1993) in science, rewards in the research culture center mostly around prestige and accolades. They promote cumulative advantage that, in turn, leads to higher status, reputation, and recognition.

This skewed distribution of credit and recognition is also noted in the peer review process. For example, Huber et al. (2022) found that author prominence was associated with the probability of reviewers accepting to review a manuscript and, also, on the assessment itself in their reports. Overall, the reward systems of research end up boosting the careers of those already having the lion’s share of credit in their fields. Publishing reliable research regardless of the journal and openly correcting the research record have yet to be captured in the rationale of the social dynamics of rewards in science.

Concerning research funding, Bol et al. (2018) state that grant reviewers often look at past achievements as criteria to judge the potential of applicants to be successful with new grants. They reason that “winners just above the funding threshold accumulate more than twice as much funding during the subsequent eight years as nonwinners with near-identical review scores that fall just below the threshold” (Bol et al. 2018, p. 2). Ribeiro et al. (2023) have shown that criteria shaping grant review include past achievements and that the research question and originality of the proposal are at the core of the process, together with research design and elements that do not encompass, for example, activity to self-correct the literature. The study by Ribeiro et al. (2023) suggests a blind spot in that reviewing process and echoes recommendations by the Global Research Council (GRC 2021). In its 2021 Assessment Report, the GRC notes that “the existing assessment system broadly favours established researchers, and that the established community who may have successfully navigated the current system may be most reluctant to change” (GRC 2021, p. 14). Recognition of this issue from a global organization that embraces funders from countries with quite different research realities and expectations toward success in academia gives a hint of upcoming changes. In 2022, Hatch & Fritch commented that funders have adopted a less conventional approach to assessing research outputs and achievements. Yet, we cannot assume that such a move means that the criteria to define success in academia have already been transformed. Despite the shifts in the landscape of research assessment, as reflected in the HKPs (Moher et al. 2020), and all efforts to revisit the culture of rewards in academia, the research paper, especially those published in top-tier journals, continues to be at the center of the scientific enterprise.

In this arena, an open question is how research systems have responded to the correction of the literature, primarily reliant on research papers when it comes to research assessment and career advancement. To address this question, we have been exploring research assessment-related issues through perceptions of retractions among funders (Ribeiro et al. 2023) and researchers who have experienced retractions. In our previous study, we surveyed experienced panelists for the National Institutes of Health (NIH) and the National Science Foundation (NSF) on their views on the possible role (if any) of retractions in the assessment of researchers when it comes to funding for those agencies. We found that while our respondents recognized the need for and relevance of mechanisms to correct the literature to strengthen the reliability of the research record, they indicated that retractions are yet to be incorporated into the assessment criteria for funding (Ribeiro et al. 2023). And it should take considerable time due to our prevailing research culture. The reviewers surveyed indicated that the lack of visibility of retractions in a researcher’s curriculum vitae is a critical issue (Ribeiro et al. 2023).

To our knowledge, no previous study looked at retracted authors’ perceptions about the role of retractions in researchers’ careers, seeking to identify perceived intersections with the reward systems of science. In this study, we show the results of a survey of a non-probability sample of corresponding authors who experienced retractions, with at least one retraction in their publication record for the 2013-2015 period.

MATERIALS AND METHODS

We explored how corresponding authors of at least one retracted publication issued between 2013 and 2015 in biomedical journals envisioned the impact of different types of retractions on the careers of the first and corresponding authors. As such impact (if any) is not always immediate, we selected this time frame (2013-2015) to ensure that potential respondents could have tangible experience after the time elapsed from the retraction. Our selection assumed that these corresponding authors would offer a unique perspective on the problem. Also, exploring their viewpoints on the role of publications and retractions among authors with retracted papers in light of the culture of rewards in science fills an existing gap in the literature. Based on these criteria, the sample frame was a population of 1,089 corresponding authors affiliated with institutions from the 20 most productive countries in biomedical areas, according to the Scimago Journal & Country Rank (JCR) in 2021. Although not directly correlated, these countries account for a considerable share of publications and retractions (Ribeiro & Vasconcelos 2018, Ribeiro 2023). For this study, we focused on the category “Medicine” and “Biochemistry, Genetics & Molecular Biology” in Scimago JCR to define the scope of biomedical publications. There is an overlap of countries for these two subareas, except for Turkey – the country is among the top 20 in “Medicine” but not in “Biochemistry, Genetics & Molecular Biology” – and Taiwan – the country appears in the top 20 in “Biochemistry, Genetics & Molecular Biology” but not in “Medicine” (Figures 1a and 1b). Thus, the total number of countries is 21, as follows: United States, China, United Kingdom, Germany, Italy, Japan, India, Canada, Australia, France, Spain, Brazil, South Korea, Netherlands, Turkey, Iran, Switzerland, Russia, Sweden, Poland, Taiwan.

Figure 1
a - The top 20 most productive countries in the category “Medicine (miscellaneous)” and their respective publication output, according to Scimago Journal & Country Rank (JCR) (2021). b - The top 20 most productive countries in the category “Biochemistry, Genetics & Molecular Biology (miscellaneous)”, and their respective publication output, according to Scimago Journal & Country Rank (JCR) (2021).

The research protocol was approved by the Research Ethics Committee (CEP) of the Faculty of Dentistry (FO)/Federal University of Rio de Janeiro, under registration CAAE: 67196423.0.0000.0268. Informed consent from respondents was collected using an online consent form. The final version of the survey instrument was defined after the feedback of four researchers with solid experience as authors and supervisors, who had been serving as reviewers or editors for scientific journals in biomedical fields, which led to minor adjustments. For the invitations to the population of corresponding authors selected (n=1,089), their emails were collected from the information available in their publications. We sent the email invitations through the SurveyMonkey platform. Of the 1,089 invitations, 129 “returned” [email inactivity or not receiving invitations via SurveyMonkey] and for three the platform could not recognize as valid email addresses. We did not calculate sample size for the beginning of the survey, as we assumed that a probabilistic sample would not necessarily derive from this study. Given the sensitive topic of having experienced one or more retractions in the research career, our sampling was purposeful and qualitative, with no goals – and even the possibility - to generalize to the entire population (Namey & Trotter 2015). This study was exploratory for its very nature. To our knowledge, the literature does not include previous data collected among authors in the biomedical sciences who experienced retractions and shared their views on the possible impact of retractions and self-retractions for misconduct or honest errors in the first and corresponding authors’ careers.

Among these potential respondents, 318 emails were bounced (29%), and 398 (62%) individuals did not open the email invitation. Among the 639 valid emails, 344 (54%) opened the invitation, and 47 (7% of the population) responded at least to the following statement: “[f]or this survey, participants must be at least 18 years old and have had at least one retracted paper between 2013 and 2015”. Despite this agreement, six (16%) respondents declared that they did not agree with the consent form and only 30 indicated their main research field, with 75% categorized in the biological, biomedical, and clinical sciences. However, respondents who completed only the demographic section were excluded, which led to 25 (4%) valid respondents (Figure 2).

Figure 2
Steps leading to the final number of survey responses.

The survey was open from May 2023 through June 2023 – with six follow-up reminders during this period. No monetary or nonmonetary incentives were offered. The survey instrument, with 22 questions, had a demographic section and a content section: the latter focusing mostly on the role of publications in research performance and of retractions in the career of researchers (see the survey instrument at https://educapes.capes.gov.br/handle/capes/868796).

RESULTS & DISCUSSION

The survey was conducted as part of a larger study (Ribeiro 2023) and aimed to explore the views on the relationship between research assessment and the influence of retractions in the career of scientists. We departed from a sample frame of 1,089 corresponding authors with at least one retraction (irrespective of reason), issued between 2013 and 2015. These retractions were associated with publications in biomedical journals, and their corresponding authors were affiliated with institutions in the 20 most productive countries, according to Scimago JCR.

In Figure 3, we offer an overview of the distribution of the population of corresponding authors (n=1,089) among the 21 most productive countries in terms of publications in biomedical areas. Although there is no evidence that the higher the research output (Figure 1a and 1b), the larger the number of retractions, the two most productive countries in biomedical fields (as per our criteria) account for the highest number of retracted authors. However, we cannot assume that these authors are more willing to share their views on retractions and reward systems as this attitude involves several subjective factors that are out of the scope of this study.

Figure 3
Distribution of our population of (retracted) corresponding authors (n=1,089) for the top 21 most productive countries in biomedical fields (0 to 344), according to Scimago Journal & Country Rank. Note that the final number on the map is 1,107 due to more than one affiliation for some authors.

After having 639 emails delivered successfully, with six follow-up reminders, we obtained a 4% response rate, with 25 valid respondents. This small sample size might be explained by the nature of the issue investigated. As well described in the literature, certain research topics may lead survey participants to feel embarrassed/ashamed and then respond less, with a socially less desirable behavior (Gannon et al. 1971, Green 1991, Lahaut et al. 2002). For example, socially acceptable behavior, such as exercise and good nutrition, may be frequently overreported, whereas undesirable behavior, such as smoking and drinking, may be underreported (Warnecke et al. 1997, Bongers 1998). As Lahaut et al (2002, p.257) report, on the basis of several previous studies, feeling embarassed or even threatened by a topic can influence survey response rates. These authors report that “Van Goor & Stuiver (1995) showed a pattern of overrepresentation of non-response in both ‘extreme’ categories of his outcome variable...”. Addressing “effectiveness of governmental organizations in policy making” in the study was a potentially threatening issue. As for the major topic of our survey, “retractions”, feeling uncomfortable is not unlikely for the invited participants. Career advancement is still primarily reliant on publication in scientific journals, as agreed on by 88% (n=22) of our respondents for this question. These respondents declare that often or almost always these publications should be associated with high-impact-factor journals (Figures 4a and 4b).

Figure 4
a - Respondents’ (n=25) perceptions on the reliance of career advancement on publication in scientific journals. b – Respondents’ (n=25) perceptions on the reliance of career advancement on publication in high-impact scientific journals.

In fact, for 96% (n=25) of respondents, publications in these more prestigious journals would have a more substantial influence on cumulative advantage for recognition and rewards than the mere number of publications. Their views are aligned with the conception of impact itself cultivated in academia (Casadevall 2019). After all, the Matthew effect (Merton 1968), which promotes cumulative advantage earned by getting published in prestigious journals, is still a structuring element in the reward systems of science, which in turn feeds back into the publication culture. As Teixeira da Silva (2021) writes “[t]he Matthew Effect… breeds success from success... [p]restige is driven by resource, which in turn feeds prestige, amplifying advantage and rewards, and ultimately skewing recognition”. This social dynamic is so tightly knotted in the research endeavor that it is quite difficult to disentangle, especially considering that mainstream science has been built up on this recognition and “cycle of credit”, as Latour & Woolgar (1986) conceived it from their anthropological studies. In this “credibility cycle”, credibility earned by researchers through publications influences funding and positions, which keeps their progress through the credibility cycle, with “one form of credit… converted into another” (Latour & Woolgar 1986, p. 208). This mechanism is well-structured in the social organization of science, so much so that it is not uncommon that frustration and suffering are shared by scientists waiting for their papers to be published (Powell 2016). The suffering of scientists trying to respond to the “publish or perish” imperative reported about 20 years ago by De Meis et al. (2003). Though focused on a particular country and moment in time, it reminds us that there continue to be “rites of passage, stress, and burnout” for researchers in the publication arena.

Although it is somewhat contradictory, about 56% of our respondents for this question (n=14) consider that career advancement is mostly reliant on the quality of science, regardless of where it is published (Figure 5).

Figure 5
Respondents’ (n=25) perceptions on the reliance of career advancement on the quality of science per se.

If these respondents equate quality with citations, this percentage is consistent with the response of 65% (n=15) who assigned “almost always” or “often” for citations as a factor influencing cumulative advantage (Figure 6). This result is also consistent with the 36% (n=9) who assigned “almost always” or “often” when asked whether career advancement has been influenced by contributions that go beyond publication in scientific journals (Figure 7).

Figure 6
Respondents’ (n=23) perceptions on the role of citations in cumulative advantage for recognition and rewards in the research career.

Figure 7
Respondents’ (n=25) perceptions when asked whether career advancement has been influenced by contributions that go beyond publication in scientific journals.

Recognizing the relationship between career advancement and the publication record, these corresponding authors surveyed would probably agree that “[a] retraction is an important mechanism for the reliability of the scientific record”, which was the case for 78% (n=18) of these researchers. This result is consistent with that in our previous study (Ribeiro et al. 2023) with grant reviewers, with 90% declaring “definitely” (66%, n=111) or “very” (24%, n=41) when asked whether “[m]echanisms for correcting the scientific literature are a way to strengthen the reliability of the research record”. When these corresponding authors were asked about the impact of self-retractions on the career of the first author and of the corresponding author, patterns of response were similar, though stronger for the corresponding author (65% for strong and considerable), compared to that for the first author (61%) (Figures 8a and 8b).

Figure 8
a – Respondents’ (n=23) perceptions on the impact of self-retractions for honest error on the career of the first author. b – Respondents’ (n=23) perceptions on the impact of self-retractions for honest error on the career of the corresponding author.

Explicitly addressing the negative impacts of self-retractions for misconduct and honest error on the career of researchers when it comes to funding decisions, the views of these respondents (n=22) were that self-retractions for honest error would have a positive impact (50%) compared to those for misconduct (23%) (Figures 9a and 9b).

Figure 9
a – Respondents’ (n=22) perceptions on the (negative or another) impact of self-retractions for honest error in funding decisions. b – Respondents’ (n=22) perceptions on the (negative or another) impact of self-retractions for misconduct in funding decisions.

These results reinforce that although self-retractions seem to be taken in a more positive light in the research community (Lu et al. 2013, Fanelli 2016, Nature Human Behaviour 2021, Ribeiro et al. 2022), self-retracting for misconduct is an attitude that may affect the career of those in leadership positions in scientific publications. For these corresponding authors surveyed, who experienced at least one retraction, when publishers or others are those initiating the retraction, the negative impact is even higher than that for a self-retraction for misconduct. As can be seen in Figure 10, 77% of these respondents view that it affects the careers of researchers negatively.

Figure 10
Respondents’ (n=22) perceptions on the (negative or another) impact of retractions for misconduct initiated by publishers or others in funding decisions.

These findings add to previous results showing that retractions are not a neutral factor in the career of scientists (Azoulay, 2015; 2017). Teixeira da Silva & Al-Khatib (2021) remind us that the outcome is similar or even identical for the career of scientists with retractions for honest errors or misconduct. As the authors sum up previous findings, “the chance of being offered a job is diminished; collaborating with colleagues becomes more difficult; the citation rate to their articles, i.e., non-retracted articles, drops to 10% (Azoulay et al. 2015) a finding that was earlier reported by Lu et al (2013)” (Teixeira da Silva & Al-Khatib 2021, p. 255).

Our respondents change their perceptions when editors or others initiate a retraction for honest error, although the positive impact would be much smaller (23%, Figure 11) than that for a self-retraction when it comes to funding, for example (50%, Figure 9a).

Figure 11
Respondents’ (n=22) perceptions on the (negative or another) impact of retractions for honest error initiated by publishers or others in funding decisions.

Most of these corresponding authors (73%, n=16) responding to the statement (n=22) “agreed” or “strongly agreed” with the idea by Besançon et al. (2022, p.3) that “…institutions and journals should promote corrections and retractions as much as they promote new research findings.” This finding makes us believe that having undergone the process of retracting their own work or having it retracted does not necessarily lead to a negative attitude toward retractions, suggesting a slightly less stigmatized perception of this type of correction, especially of self-retractions for honest error. It also resonates with a growing awareness in academia that this type of correction should be encouraged, when necessary, in the post-publication process as part of the self-regulation of science (Nature Human Behaviour 2021, Ribeiro et al. 2022, Barbour et al. 2017, Enserink 2017).

CONCLUSIONS

Overall, our results suggest that for these researchers publishing in the biomedical sciences, recognition for publications in high-impact-factor journals continues to be the main source of credit and rewards in the research system, although the number of publications, irrespective of journal, has a relevant role in gaining such recognition. When it comes to the reliability of the literature, most respondents agree that a retraction has an important role to play and that “…institutions and journals should promote corrections and retractions as much as they promote new research findings” (Besançon et al. 2022). Yet, about 30% of the respondents perceive those self-retractions for honest errors as having a mild or no impact on the career of the first and the corresponding author. Note, however, that, according to the perceptions of the surveyed authors, any negative impact that a retraction could have, be for misconduct or honest error, would be asymmetrical between the first and corresponding authors. One possible explanation might be that the corresponding authors would be “shielded” by a “reverse Matthew Effect”, which Jin et al. (2019, p. 492) described in the context of “credit sharing after damaging events”.

In a nutshell, the viewpoints on self-retractions among the corresponding authors surveyed are that retractions for honest error would have a more positive impact on the career of both first and corresponding authors than those for misconduct. These results reflect a trend in recent debates encouraging that authors should be proactive in self-correcting their work (Fanelli 2016, Teixeira da Silva & Al-Khatib 2021, Fanelli et al. 2022). Yet, these results reinforce that addressing retractions and self-retractions, irrespective of the reason, continues to be challenging in the publication system (Teixeira da Silva & Al-Khatib 2021) or, broadly, in the prevailing research culture. In light of the “reverse Matthew effect” described by Jin et al. (2019), our study also reinforces that the impact of retractions for honest errors on the career of the first and corresponding authors should be investigated further to avoid exacerbating such an effect. Investigating the implications of retractions in the career of researchers in the biomedical fields and beyond is an evident demand at a time of marked breakthroughs in how we communicate and assess science.

Finally, from a methodological perspective, experimenting with approaches that can yield larger sample sizes to explore the impact of retractions on one’s research career is timely. In our study, we pre-tested the survey instrument with a few experienced reviewers/editors, with positive feedback on the proposal. We tried to mitigate biased responses when designing the survey and, at the same time, nonresponse rates by not asking corresponding authors about their own experience with self-retracting or having a paper retracted. We also avoided respondent fatigue by keeping the survey instrument as short as possible. We sent extra follow-up reminders to increase the sample size without making direct reference to the invitee’s retracted publications. All these efforts notwithstanding, only a small fraction of these individuals accepted to participate. This is a socially relevant and qualitative significant finding worth noting. The resulting small sample size underscores that retractions are a sensitive topic among individuals who experienced retracted publications. Perhaps, having to revive the retraction experience through responding to a survey addressing the topic is so emotionally demanding that might lead to reluctance to participate.

Limitations

The main limitations related to this study is the sample size, comprising 4% (n=25) of the population, which is smaller than the minimum necessary for a probability sample (Yamane 1967, Israel 1992). We tried to avoid high nonresponse rates, mainly due to the sensitive nature of retractions, with survey questions that did not approach the influence of retractions on the respondent’s own career. We also distinguished retractions for misconduct and those for honest error, with the understanding that there would be different views and impacts, as seen in the previous results of our survey with grant reviewers (Ribeiro et al. 2023). Apart from these constraints, these corresponding authors who experienced at least one retraction might be biased towards a socially desirable rendering of the publication system and of retractions, and their answers should be taken with a grain of salt. Additionally, the survey instrument captures only some aspects of the reward systems of science and of the influence of retractions in the career of authors in biomedical fields.

• 1
  ¹ As described by the National Research Council (2014) “the systems of research and innovation provide almost everything needed to ensure the continued generation, flow, and use of knowledge. But external variables—such as investment and infrastructure, intellectual environment, management, motivations, and incentives—can enhance or hinder the ultimate success of the research enterprise.”
• 2
  (https://www.ncbi.nlm.nih.gov/books/NBK253891/)

REFERENCES

• AZOULAY P, BONATTI A & KRIEGER JL. 2015. The career effects of scandal: Evidence from scientific retractions. NBER Working Paper No. 21146. National Bureau of Economic Research, Cambridge, MA. Available at: http://jkrieger.scripts.mit.edu/docs/w21146.pdf
• AZOULAY P, BONATTI A & KRIEGER JL. 2017. The career effects of scandal: Evidence from scientific retractions. Res Policy 46: 1552-1569.
• BALL, P. 2015. ‘Novel, amazing, innovative’: Positive words on the rise in science papers. Nature doi:10.1038/nature.2015.19024.
• BARBOUR V, BLOOM T, LIN J & MOYLAN E. 2017. Amending published articles: Time to rethink retractions and corrections? F1000Research 6: 1960.
• BESANÇON L, BIK E, HEATHERS J & MEYEROWITZ-KATZ G. 2022. Correction of scientific literature: Too little, too late! PLoS Biol 20(3): e3001572. doi: 10.1371/journal.pbio.3001572.
• BOL T, DE VAAN M & VAN DE RIJT A. 2018. The Matthew effect in science funding. PNAS 115: 4887–4890. doi.org/10.1073/pnas.171955711.
  » https://doi.org/10.1073/pnas.171955711.
• BONGERS IBM. 1998. Problem drinking among the general population: a public health issue? Dissertation. Addiction Research Institute (IVO), Rotterdam.
• CASADEVALL A. 2019. Duke University’s huge misconduct fine is a reminder to reward rigour. Nature 568: 7-7.
• CASADEVALL A & FANG FC. 2012. Reforming science: Methodological and cultural reforms. Infect Immun 80: 891-896.
• DE MEIS L, VELLOSO A, LANNES D, CARMO MS & DE MEIS C. 2003. The growing competition in Brazilian science: rites of passage, stress and burnout. Braz J Med Biol Res 36(9): 1135-114.
• ENSERINK M. 2017. How to avoid the stigma of a retracted paper? don’t call it a retraction. Science. doi:10.1126/science.aan6937.
• FANELLI D. 2016. Set up a ‘self-retraction’ system for honest errors. Nature 531: 415-415.
• FANELLI D, WONG J & MOHER D. 2022. What difference might retractions make? An estimate of the potential epistemic cost of retractions on meta-analyses. Account Res 29(7): 442-459.
• GANNON MJ, NOTHERN JC & CARROLL SJ. 1971. Characteristics of nonrespondents among workers. J Appl Psychol 55(6): 586-588.
• GERNSBACHER MA. 2018. Writing empirical articles: Transparency, reproducibility, clarity, and memorability. Adv Meth Pract Psychol Sci 1(3): 403-414.
• GIBBONS M. 1999. Science’s new social contract with society. Nature 402(S6761): C81-C84.
• GRC - GLOBAL RESEARCH COUNCIL. 2021. Responsible research assessment. https://globalresearchcouncil.org/fileadmin/documents/GRC_Publications/GRC_RRA_Conference_Summary_Report.pdf
  » https://globalresearchcouncil.org/fileadmin/documents/GRC_Publications/GRC_RRA_Conference_Summary_Report.pdf
• GREEN KE. 1991. Reluctant respondents: Differences between early, late, and nonresponders to a mail survey. J Exp Edu 59(3): 268-276.
• HATCH A & FRITCH R. 2022. Cross-funder action to improve the assessment of researchers for grant funding. https://sfdora.org/2022/01/19/cross-funder-action-to-improve-the-assessment-of-researchers-for-grant-funding/
  » https://sfdora.org/2022/01/19/cross-funder-action-to-improve-the-assessment-of-researchers-for-grant-funding/
• HUBER J, INOUA S, KERSCHBAMER R, KÖNIG-KERSTING C, PALAN S & SMITH VL. 2022. Nobel and novice: Author prominence affects peer review. PNAS 119(41).
• IOANNIDIS JPA. 2014. How to make more published research true. PLoS Med 11(10): e1001747.
• ISRAEL GD. 1992. Determining sample size. Program evaluation and organizational development. Florida Co-operative Extension Service. IFAS, University of Florida. PEOD-6.
• JIN GZ, JONES B, LU SF & UZZI B. 2019. The reverse Matthew effect: Consequences of retraction in scientific teams. Rev Econ Stat 101(3): 492-506.
• LAHAUT VM, JANSEN HA, VAN DE MHEEN D, GARRETSEN HF. 2002. Non-response bias in a sample survey on alcohol consumption. Alcohol Alcohol 37(3): 256-260.
• LATOUR B & WOOLGAR S. 1986. Laboratory life: the construction of scientific facts. Princeton University Press.
• LU SF, JIN GZ, UZZI B & JONES B. 2013. The retraction penalty: Evidence from the Web of Science. Sci Rep 3: 3146.
• MERTON RK. 1968. The Matthew effect in science- the reward and communication systems of science are considered. Science 159(3810): 56-63.
• MILLAR N, BATALO B, & BUDGELL B. 2022. Trends in the use of promotional language (Hype) in abstracts of successful National Institutes of Health Grant Applications, 1985-2020. JAMA Network Open 5(8): e2228676.
• MOHER D, BOUTER L, KLEINERT S, GLASZIOU P, SHAM MH, BARBOUR V, CORIAT AM, FOEGER N & DIRNAGL U. 2020. The Hong Kong Principles for assessing researchers: Fostering research integrity. PLoS Biol 18(7): e3000737.
• NAMEY EE & TROTTER RT. 2015. Qualitative research methods in public health research methods. Sage Research Methods. Thousand Oaks, CA: Sage. https://doi.org/10.4135/9781483398839
  » https://doi.org/10.4135/9781483398839
• NATIONAL RESEARCH COUNCIL. 2014. Furthering America’s research enterprise. In: Celeste RF, Griswold A & Straf ML (Eds), Committee on Assessing the Value of Research in Advancing National Goals, Division of Behavioral and Social Sciences and Education. Washington, DC: The National Academies Press.
• NATURE HUMAN BEHAVIOUR. 2021. Breaking the stigma of retraction. Nat Hum Behav 5: 1591-1591.
• NOSEK BA & LAKENS D. 2014. Registered reports: A method to increase the credibility of published results [Editorial]. Soc Psychol 45(3): 137-141.
• POWELL K. 2016. Does it take too long to publish research? Nature News 530(7589): 148.
• RIBEIRO MD. 2023. The influence of retractions on the career of scientists from the most productive countries in the biomedical sciences. PhD Thesis, Federal University of Rio de Janeiro.
• RIBEIRO MD & VASCONCELOS SMR. 2018. Retractions covered by Retraction Watch in the 2013-2015 period: Prevalence for the most productive countries. Scientometrics 114: 719-734.
• RIBEIRO MD, KALICHMAN M & VASCONCELOS SMR. 2022. Scientists should get credit for correcting the literature. Nat Hum Behav 7(4): 472.
• RIBEIRO MD, KALICHMAN MW, VASCONCELOS SMR. 2023. Retractions and rewards in science: An open question for reviewers and funders. Sci Eng Ethics 29(4).
• ROSSITER MW. 1993. The Matthew Matilda Effect in Science. Soc Stud Sci 23(2): 325-341.
• SHANAHAN DR. 2015. A living document: reincarnating the research article. Trials 16(1).
• SMALDINO PE & MCELREATH R. 2016. The natural selection of bad science. R Soc Open Sci 3(9): 160384.
• SODERBERG CK, ERRINGTON TM, SCHIAVONE SR, BOTTESINI J, THORN FS, VAZIRE S, ESTERLING KM, & NOSEK BA. 2021. Initial evidence of research quality of registered reports compared with the standard publishing model. Nat Hum Behav 5(8): 990-997.
• TEIXEIRA DA SILVA JA. 2021. The Matthew effect impacts science and academic publishing by preferentially amplifying citations, metrics and status. Scientometrics 126(6): 5373-5377.
• TEIXEIRA DA SILVA JA & AL-KHATIB A. 2021. Ending the retraction stigma: Encouraging the reporting of errors in the biomedical record. Res Ethics 17(2): 251-259.
• VAN GOOR H & STUIVER B. 1995. Succes en falen van beleid en non-respons: een empirisch onderzoek naar het terugzenden van schriftelijke enquêtes door Nederlandse gemeenten. Sociol Gids 42: 388-406.
• VINKERS CH, TIJDINK JK, & OTTE WM. 2015. Use of positive and negative words in scientific PubMed abstracts between 1974 and 2014: retrospective analysis. BMJ 351: h6467. doi: 10.1136/bmj.h6467.
• WARNECKE RB, JOHNSON TP, CHÁVEZ N, SUDMAN S, O’ROURKE DP, LACEY L & HORM J. 1997. Improving question wording in surveys of culturally diverse populations. Ann Epidemiol 7: 334-342.
• YAMANE T. 1967. Statistics an introductory analysis, 2nd ed., New York: Harper and Row.

Publication Dates

• Publication in this collection
  23 Sept 2024
• Date of issue
  2024

History

• Received
  24 Jan 2024
• Accepted
  26 May 2024