How to preregister a study on as predicted

Summary

Emily demonstrates the process of pre-registering a study on askpredicted.org, a platform promoting open science and reproducible research. She guides viewers through entering author details, hypotheses, data collection plans, and analysis methods. Emily emphasizes the importance of specificity to prevent data manipulation and suggests documenting secondary predictions and demographics. She also shares her experience with platform sampling inaccuracies and how to anonymize pre-registrations for peer review.

Takeaways

• 📝 Pre-registration is a part of the open science movement, promoting transparency and reproducibility in research.
• 🔍 To pre-register on askpredicted.org, researchers must answer a set of questions about their study before collecting data.
• 👥 Researchers need to provide names, email addresses, and institutions of all authors involved in the study.
• 🚫 Ideally, no data should have been collected before pre-registration to ensure the integrity of the research process.
• 📜 Hypotheses should be explicitly stated, aligning with the planned tests to prevent data manipulation.
• 📊 Researchers should pre-register the dependent variable (DV), number of conditions, and specific analyses such as t-tests, ANOVA, or regression.
• 🔍 Outliers and exclusion criteria, like attention checks, must be specified in the pre-registration to avoid bias.
• 🔢 Sample size and the method of determining it, such as the number of participants recruited per condition, should be clearly stated.
• 📈 Pre-registration helps in specifying secondary predictions and exploratory questions, which are not essential to the main hypothesis.
• 📑 After completing the pre-registration, researchers can preview, edit, and accept it before it's sent to all authors for final acceptance.
• 🔒 Once all authors accept the pre-registration, it becomes part of the public record, enhancing the credibility of the study.

Q & A

What is the purpose of pre-registration in research?

Pre-registration is part of a movement to promote open science, open data, and reproducible research. It helps ensure transparency and prevent data manipulation by specifying the research plan before data collection begins.

What is 'askpredicted.org' and how does it relate to pre-registration?

'askpredicted.org' is a platform created by researchers at Wharton where researchers can create new pre-registrations and answer a set of questions to formalize their research plan and hypotheses.

Why should data collection be avoided before pre-registering a study?

Ideally, data collection should not occur before pre-registration because the process assumes it is the first time data is being collected, ensuring the integrity and originality of the research findings.

What information is required when pre-registering a study on askpredicted.org?

Researchers need to provide the names, email addresses, and institutions of the authors involved in the study, specify whether data has been collected, and detail the hypotheses, conditions, analyses, and any exclusions or outliers.

Why is it important to be explicit about the analyses and covariates in a pre-registration?

Being explicit about the analyses and covariates helps reduce the chance of p-hacking, which is manipulating data to achieve statistically significant results. It also ensures that the research is reproducible and transparent.

What is the significance of specifying outliers or exclusions in a pre-registration?

Specifying outliers or exclusions is important because it sets the criteria for participant inclusion or exclusion in the study. This transparency helps others understand the sample selection process and prevents bias.

How should the sample size be determined and specified in a pre-registration?

The sample size should be determined based on the research design and specified in the pre-registration, including how participants will be recruited and any potential oversampling to account for platform inaccuracies.

What is the process of making a pre-registration anonymous for peer review?

To make a pre-registration anonymous for peer review, researchers can use the 'make anonymous pdf' feature on askpredicted.org, which removes identifying information, making it suitable for blind review.

How can researchers recycle the text from a pre-registration for their final paper?

Researchers can copy and paste the text from their pre-registration into a blank word document, where they can further develop their answers to the pre-registration questions. This text can later be used in the write-up of the study and the paper.

What is the role of attention checks in a pre-registered study?

Attention checks are used to ensure that participants are engaging with the study material attentively. Researchers should specify in the pre-registration if they plan to exclude participants based on attention check results.

How does the process of accepting a pre-registration work among the authors?

Once a pre-registration is completed and previewed by the lead researcher, it must be accepted by all authors involved. This acceptance is typically done through a personal or institutional email, and once all authors have accepted, the pre-registration is finalized and can be made public.