Eleven strategies for making reproducible research the norm

In recent years, awareness of the importance of reproducible research and open science has grown in the research community. The importance of conducting robust, transparent, and open research has especially been highlighted by the reproducibility crisis, or credibility revolution (Baker, 2016; Errington et al., 2021; Vazire, 2018). Reproducible and open science practices increase the likelihood that research will yield trustworthy results, and facilitate reuse of methods, data, code, and software (Chan et al., 2014; Diaba-Nuhoho and Amponsah-Offeh, 2021; Downs, 2021; Ioannidis et al., 2014). Across fields, definitions of ‘reproducible’ and ‘open’ may vary. While some fields use the terms interchangeably, in other fields ‘reproducible’ includes elements of scientific rigor and research quality, whereas ‘open’ simply refers to making research outputs publicly accessible. Overall, these practices seek to improve the transparency, trustworthiness, reusability, and accessibility of scientific findings for the research community and society (Barba, 2018; Claerbout and Karrenbach, 1992; Nosek et al., 2022; Parsons et al., 2022; Wolf, 2017). Examples of specific practices include sharing of protocols, data and code, publishing open access, implementing practices such as blinding and randomization to reduce the risk of bias, engaging patients in designing and conducting research, using reporting guidelines to improve reporting, and using CRediT authorship statements to specify author contributions. Despite these developments, reproducible research and open science practices remain uncommon in many fields (Blanco et al., 2019; Grant et al., 2013; Hardwicke et al., 2022; Hardwicke et al., 2020; Page and Moher, 2017).

According to a survey by the European University Association (EUA) for 2020–2021, 59% of the 272 European institutions surveyed rated open science’s strategic importance at the institutional level as very high or high (Morais et al., 2021). The strategic importance of open science has also been recognized by policy-makers, for example by the UNESCO Recommendations on Open Science (UNESCO, 2021). Unfortunately, these values are not reflected in the current research assessment system. ‘Classic’ research assessment criteria, such as the Journal Impact Factor or the h-index, are still being used to assess the contribution of individual researchers. The use of these biased metrics should be discouraged, however, as they ignore the value of other research outputs (e.g. protocols, data, code) and are not useful for assessing the impact and quality of individual research contributions (https://sfdora.org/read/). Initiatives such as COARA seek to reform research assessment criteria to recognize a broad range of activities that contribute to high quality research (https://coara.eu/). These reforms are essential to incentivize reproducible research and open science practices.

In addition to shifting incentives, effective education and training programs that teach reproducible research and open science skills have not yet been implemented across research fields. Researchers in various disciplines are discussing whether these concepts apply, and how they might be implemented. To explore these ideas, German Reproducibility Network members organized a virtual brainstorming event (see Box 1) to discuss strategies for making reproducible research and open science training the norm at research institutions in Germany and beyond.

Virtual unconference format

In March 2022, 96 participants, consisting mostly of members of initiatives and organizations belonging to the German Reproducibility Network (GRN) and other researchers based in Germany, took part in the virtual brainstorming event. Participants came from a variety of professional backgrounds (e.g. academic researchers, administrators, library and information science professionals), career stages (from graduate students to senior group leaders), and disciplines (e.g. psychology, biomedical sciences). The virtual brainstorming event unconference format has been explained previously (Holman et al., 2021). Supplementary file 1 provides details of this specific event. This paper shares lessons learned from two days of intensive discussions, through virtual networking events, virtual meetings, and asynchronous conversations on an online discussion board.

The first section of this paper provides a brief overview of eleven strategies that were derived from the event. Members of the research community can implement these strategies by taking action in their roles as instructors, researchers, supervisors, mentors, members of curriculum or hiring and evaluation committees, or as part of institutional leadership, research support or administrative teams. The section also highlights actions that institutions can take to support these activities, by allocating resources and monitoring impact. The second section of this paper lists a few tips for implementing several strategies. Cited resources provide additional insights for those interested in pursuing specific strategies. While making reproducible and open science training the norm might involve major changes at institutions, this journey starts with small steps towards reproducible and open science practices. Changing norms will require a broad coalition; hence, we hope that this piece inspires others to join this effort, while encouraging those who are already engaged to think creatively about opportunities to enhance the impact of their work.

from Hacker News https://ift.tt/94hPxqW