Replication

This page includes several resources that researchers can use to help in the planning, carrying out, and disseminating a replication study

1. Identifying a study to replicate

  • The journal Language Teaching publishes a regular feature that recommends replication of particular studies, including suggestions for the replication approach (e.g., close, approximate, conceptual) as well as suggestions for variable modification. These papers can be used to identify potential studies for replication. Recent examples include McManus (2022), Omidian et al. (2023), and Roehr-Brackin (2022)

  • The book Doing Replication Research in Applied Linguistics by Porte and McManus (2019) is a hands-on guide for how to do replication research in the field of applied linguistics. Not only does the book discuss why researchers should carry out replication studies, but it also guides the reader through the entire replication research process: From seeking out a suitable study for replication, through deciding on the most valuable form of replication approach, to its execution, discussion, and writing up for publication. This book includes the following chapters:
    • Introduction: Why replication research matters
    • Finding a study to replicate: Background research
    • Planning your replication research project
    • What kind of replication should you do? From the inside, looking out: initial critique and internal replication
    • What kind of replication should you do? From the outside, looking in
    • Executing and writing up your replication study: Research questions and methodology
    • Executing and writing up your replication study: Analysis, Results, Discussion and Conclusion
    • Disseminating your research

  • In Psychology, Isager et al. (2021) have proposed a decision model for replication study selection that can be used to help with decisions about which studies to consider for replication.

2. Designing a replication study

  • The book by Porte and McManus (2019) includes guidelines that can inform how researchers decide to design a replication study. The discussion here focuses on common variables to modify in a replication, reading for awareness-raising, and reading with targeted questions that can all be used to inform the decision-making process about what type of replication study to conduct, why, and how.

  • A systematic and narrative review of replication in second language research with recommendations for the field by Marsden, Morgan-Short, Thompson, and Abugaber (2018). This article presents a comprehensive synthesis of replication practices in the field of second language research based on a synthesis of 67 self-identified replication studies published between 1973 and 2016 in Social Science Citation Index journals.

  • A paper by Brandt and colleagues (2014) titled “The replication recipe: What makes for a convincing replication” published in the Journal of Experimental Social Psychology includes a useful 36-question guide that can be used to plan a replication study (see also McManus, 2023). These simple questions encourage researcher to think critically about several factors, related to the design of the replication study, among other things.
  • The IRIS database (Instruments and data for research in language studies) is a collection of instruments, materials, stimuli, data, and data coding and analysis tools used for research into languages, including first, second-, and beyond, and signed language learning, multilingualism, language education, language use, and language processing. These materials are freely accessible and searchable, easy to upload (for contributions) and download (for use). Researchers can use this resource to locate a study’s data collection materials, data, and/or coding protocol that can be used in the replication.

3. Other resources

  • Reporting standards for replication studies in applied linguistics (McManus, 2024). This paper outlines basic expectations in the reporting of replication studies in the field of applied linguistics.

  • Journal article reporting standards for quantitative research in psychology (Appelbaum et al., 2018). This paper describes reporting standards for replication studies

  • The book chapter “Replication research in instructed SLA” (2022) introduces researchers with interests in instructed SLA to the replication research process (e.g., what replication research is, common types of replication research, why carrying out replication is important). In addition, specific guidelines are provided for carrying out and reporting replication studies. Recommendations for future replications in instructed SLA are suggested as well as ways in which researchers can integrate replication into future programs of research.

  • The paper “Are replication studies infrequent because of negative attitudes” (2022) reports on recent findings from a survey of attitudes and practices in the field of second language research. 354 researchers were surveyed about their attitudes toward replication and their practices conducting replication studies. The analyses indicated that replications were evaluated as relevant and valuable to the field. Claims that replication studies lack originality/innovation were not supported. However, dissemination issues were identified: half of published replication studies lacked explicit labeling and one quarter of completed replications were unpublished.