Instructions

Prototyping – Building tools or documentation collaboratively with others. This can be open source code, documentation, tools and technologies used in data-driven science, scientific software, discovery tools (e.g., ways of searching code, hosting services, etc.) Community Building – These might include mentorship programs, workshops, meet-ups or other events that promote community engagement. Curriculum – Materials might include text, visuals, and video for online or in-person training. All materials must be open source, shared under a Creative Commons license of CC-0 or CC-BY, when licensing is required. Proposals must address one of three key areas: Prototyping, Community Building, or Curriculum. The brief application will require that they reflect an understanding of the work we are currently doing and articulate their own efforts within the needs of the broader community.
All projects should specifically reflect priorities that enhance our broader community efforts toward open innovation, efficiency in regards to practicing open science (lower barriers, ease of use & integration, etc), and reproducibility (transparent research methods & results). Questions!!

Questions

1. What institution are you based at currently, if any?
University of California San Francisco (Sam) and Oregon Health & Science University (Daniela).
2. Have you previously received a grant from Mozilla Foundation? If so, list the amount and purpose of the grant.
No.
3. Proposal Title:
Preprint Journal Club Starter Kit
4. Provide a brief description of your project or event. (50 words)
We are passionate about Open Science and we believe preprints can help to promote this. The Preprint Journal Club Starter Kit project aims to provide resources to encourage and facilitate the hosting of preprint journal clubs at different research institutions around the world.
5. Total proposal budget in US Dollars (USD).
$2,000
6. How much money in USD are you requesting from the Mozilla Foundation?
$2,000
7. List your partners, if any, and the role/s they will play. (250 Words)
Samantha Hindle and Daniela Saderi will be the primary leaders of this project. Samantha is an Assistant Professional Researcher at the University of California San Francisco and Daniela is a Ph.D. candidate at Oregon Health & Science University. They are both ASAPbio Ambassadors and dedicated Open Science advocates.
Samantha and Daniela will be responsible for developing the Preprint JC Starter Kit and distributing it among a group of researcher volunteers (alpha and beta testers) that will be asked to use the Preprint JC materials to start their own Preprint JC at their institutions. Furthermore, they will act as point of contact for the volunteers and for the other members of the community of ASAPbio Ambassadors. In fact, we just recently started a dedicated Slack channel where we have almost daily conversations on how to increase awareness of preprints in the academic community.
In addition, we are counting on Jessica Polka and Prachee Avasthi as primary advisors of the project. Jessica is the Director of ASAPbio, a non profit organization born from the need of changing the culture around preprints in academia. Prachee is an Assistant Professor at the University of Kansas Medical Center, and is an ASAPbio Ambassador. She has led a Prerprint JC course for her students the past Fall and has generously shared her material and her experience with us.
8. On which Internet health issue(s) does your proposal focus? (Select all that apply.)
Open Innovation
9. Are you requesting support for an event (e.g. workshop, training, hack-a-thon, or other event) or a project (e.g. open software, curriculum, or other project)?
Project
10. Describe the issue/problem you are trying to address. (100 words)
Publishing in the biological sciences has become a lengthier process over the past 30 years and can take 6-12 months from journal submission to publication. Preprints, free and immediately available versions of scientific manuscripts that have not yet undergone peer review, are considered a solution to this problem. Our project aims to promote the wide usage of preprints by providing incentives to incorporate preprints into Journal Club (JC) practices. According to our Mozsprint survey, 86% expressed an interest in hosting a Preprint JC and 83% felt that writing a preprint review would provide beneficial training for Early Career Researchers (ECRs).
11. List key project activities (what will you do), outputs (what will be produced through your activities, e.g. products, publications, number of people trained) and outcomes (impact of your project on your beneficiaries during the grant period). (1000 words)
Key activities: Phase I: Alpha testing. We will engage five ASAPbio ambassadors that are already hosting Preprint JCs to be our alpha testers. We will ask them to commit to one Preprint JC using our Starter Kit. The materials provided will include a sample email to send out to the department announcing the Preprint JC, a guide to discovering preprints on the web, Preprint JC Guidelines for the host, a Preprint Review Worksheet, a Preprint JC flyer template, and a Preprint info sheet containing an infographic description of preprints. At the end of the JC, we will ask our alpha testers to complete a feedback survey that will detail their experience as hosts of a Preprint JC, their opinion on our Starter Kit, and what they think we should change/improve.
Phase II: Beta-testing. We will enroll 20 researchers from diverse research areas and countries as beta testers. This grant would allow us to provide each beta tester with $75 to host two successful JCs (starting on October 1st) by buying snacks and coffee/drinks. In our experience, refreshments are key to attract ECRs to institutional events. The beta testers will be asked to: a) Host two Preprint JCs in their own department following our Preprint JC Starter Kit; b) Write a review for each discussed preprint following our Review Worksheet; c) Email the reviews to each preprint’s corresponding author(s) and upload them online (platform TBD); b) Fill out our Beta Tester Feedback Survey; d) Document attendees’ reactions and experiences with discussing a preprint and providing feedback to the authors by having them fill out our short Preprint JC Participant Survey. To provide a baseline to compare the data to, we will also ask either the same alfa and beta testers (if appropriate) or other randomly selected researchers that normally host conventional JCs to tell us about their own success level and obstacles.
Outputs: Our main outputs from this project will be to collect quantifiable data from each phase. We expect to collect: a) One set of feedback from our 5 alpha testers; b) One set of feedback from the two Preprint JC run by our 20 beta testers; c) One set of feedback from all Preprint JC attendees combined; d) One set of feedback from the preprint corresponding authors (possibly via another short survey that we are hoping the authors will be willing to engage with); e) Forty Preprint reviews posted online either as comments to the discussed preprint (e.g., comment in bioRxiv) or using servers such as Authorea (they have expressed some interest on Twitter), The Winnower (host of Prachee’s students reviews), or Publons.
Outcomes: Our project will provide incentives for other researchers to host Preprint JCs in their own institutions, which will promote the awareness of preprints and their benefits across the world. As we aim to enroll beta testers that are widely distributed across the globe, and from a variety of career stages, we hope to gain a rich dataset via our user and preprint author surveys. This will allow us to scientifically assess the usefulness of Preprint JCs and their associated reviews.
12. Provide key indicators you plan to use to measure project outcomes and source of data. (500 words)
Our main project goal is to initiate and maintain Preprint JCs in host institutions across the globe. As we plan to collect quantifiable feedback throughout the two phases of our projects, we will be able to gauge the effectiveness of our guidelines and improve them based on survey results. As another important goal of our project is to expose Early Career Researchers (ECRs) to the peer review process, our surveys will also gather feedback from the Preprint JCs’ attendees. We will ask them if and how the Preprint JC has changed their understanding and opinion about preprints and if they found it a useful exercise to learn how to write a scientific review.
A final key indicator for our project outcomes will be the results of our Preprint Corresponding Author surveys, which will indicate whether reviews from Preprint JCs were beneficial and therefore integrated into their manuscript prior to publication.
13. Explain who will benefit from the project. (50 words)
Early career researchers (ECRs) will be the primary beneficiaries as they will become familiarized with preprints as well as learn how to write a constructive peer review. The ASAPbio community will also benefit from this project as it will provide an opportunity to engage in a preprint collaboration.
14. Describe the geographic location(s) in which this project will be implemented. (100 words)
We are currently trying to recruit alpha and beta-testers from different research institutions around the world. We are hoping to take advantage of the global Mozilla community to reach a diverse community of scientists. As a result of the Mozilla Global Sprint, we already have a list of potential beta-testers from our community opinion survey respondents.
15. List any risks or challenges that may affect the overall success of your project, and note how Mozilla and/or others can help you to overcome these challenges. (250 words)
We don’t foresee risks associated with our project, but we do recognize the potential challenges associated with getting academics to change their attitudes towards scientific publishing – particularly without a change in incentives. Despite the increasing evidence that shows how posting a preprint can improve the quality of the research – as the authors potentially get feedback from their peers prior to submission for publication, as well as increase the visibility of the work, preprints are not widely used by academics. We hope that the funding provided to this project will help to incentivize ECRs to integrate preprints into their JC practices and make an example than senior researchers will follow. We intend to provide a $25 gift card as a further incentive for our beta testers to complete the project.
16. Is this a new project or a continuation? If new, please describe your qualifications to initiate the activity. If continued, please describe your accomplishments to date. Feel free to include links to articles and documents online which highlight your recent work. (1000 words)
The Preprint JC, led by Daniela Saderi and Samantha Hindle, is a new project built from the work initiated by ASAPbio. Samantha and Daniela are both ASAPbio Ambassadors (http://asapbio.org/asapbio-ambassadors), which is a large and diverse community of people dedicated to spreading the word about preprints in their own institutions and scientific communities. The ASAPbio Ambassador community, along with the broader Mozilla community, will provide a rich source of potential beta-testers for our project.
Samantha is an Assistant Professional Researcher at the University of California San Francisco and, since ASAPbio’s inception, has closely coordinated with the ASAPbio Founder Ron Vale and Director Jessica Polka. Samantha has co-hosted Preprint Town Hall meetings at her home institution and has represented ASAPbio at Open Science meetings. Samantha also has substantial experience in writing manuscript reviews and is well placed to develop resources for this project.
Daniela is a Ph.D candidate in Neuroscience at Oregon Health & Science University and has been leading a conventional (postprint) Systems Neuroscience Journal Club for four years. Under Daniela’s encouragement, in the past few months the group (including researchers from all career stages) has begun to discuss preprints with the goal of sending comments to the authors and the whole community via online repositories. This practice has been a hugely positive experience for both students and preprint authors, who have expressed thanks for the comments.
17. Mozilla works in the open. How will you document and share your project progress with the community. (250 words)
The PreprintJC project is an open project and all our resources are accessible through the repository at https://github.com/SamanthaHindle/preprint_JournalClub. We aim to release the results of our surveys through our repository after our beta-testing stage of our project is complete. We will also create a repository file that will house links to all 40 of our beta-tester reviews. This file will not only document the success of our project, but will provide an educational resource in itself: a rich set of reviews that ECRs can read and learn from.
18. How will you continue work on this project beyond this funding period? (250 words)
First of all, our Starter Kit as well as our surveys’ questions will be shared openly on our GitHub repo (https://github.com/SamanthaHindle/preprint_JournalClub) so that anyone interested will be able to contribute and/or fork the project and start their own. Following the completion of this funding period we will be well poised (and quite excited) to enter Phase III of our Preprint JC project. This will involve spreading the word about our Preprint JCs and distributing our resources to facilitate their occurrence. The data collected in Phase I and II will provide us with a pretty good picture of the success level and the barriers associated with Preprint JCs. In addition we will write a template email that the beta testers will be able to use to contact the Heads of their Department. We will also write a blog post (or even an article to post as preprint!) with the results of our program. We hope to show to the world the successful adoption of Preprint JCs and highlight the positive experience with peer reviewing from ECRs. We will ask ASAPbio network and Mozilla Science Lab community to help us distribute our project.