Cover not available

In:Ethical Issues in Applied Linguistics Scholarship
Edited by Peter I. De Costa, Amr Rabie-Ahmed and Carlo Cinaglia
[Research Methods in Applied Linguistics 7] 2024
► pp. 10–27

Get fulltext from our e-platform
JBE logo with link
PDF logo with download linkDownload Book PDF
EPUB logo with download linkDownload Book EPUB
authorization required

Chapter 1
A taxonomy of questionable research practices in quantitative humanities

 | Northern Arizona University
 | Northern Arizona University
 | Indiana State University
 | Uppsala University
 | University of South Florida
 | Northern Arizona University
Published online: 21 November 2024
DOI logo with link
https://doi.org/10.1075/rmal.7.01plo
Abstract
A growing body of research has begun to address ethical issues in the context of Applied Linguistics (e.g., De Costa, 2016; Isbell et al., 2022). One of the messages inherent in this line of inquiry is that ethical concerns are embedded throughout the research cycle from study conceptualization to realization, dissemination, application, and beyond (see Bernstein et al., this volume). With this concern in mind, the present study sought to catalog and develop a taxonomy of what are often referred to as ‘questionable research practices’ (QRPs; Steneck, 2006) and related decisions that come into play in the conduct of quantitative Applied Linguistics research. These include practices such as selective reporting and obscuring of methodological details to limit criticism. Using existing taxonomies developed in neighboring disciplines as a starting point (e.g., Tauginienė et al., 2019), we employed the Delphi method to elicit responses on potential QRPs in an iterative fashion from an expert panel as well as from peer scholars. The analyses of these data resulted in a domain-specific taxonomy that laid the groundwork for a large-scale survey that assessed the prevalence and perceived severity of ethical issues and QRPs found specifically in quantitative Applied Linguistics research (Larsson et al., 2023). The results are also used to inform materials for methodological training in research ethics in Applied Linguistics and related disciplines (see De Costa et al., 2021; Wood et al., 2024, in press).
Article outline
  • Introduction
  • Literature review
  • Method
  • The expert panel
  • QRP item generation and revision
  • Results
  • Discussion
  • Conclusion and future directions
  • Notes
  • References
References (41)
References
Al-Marzouki, S., Roberts, I., Marshall, T., & Evans, S. (2005). The effect of scientific misconduct on the results of clinical trials: A Delphi survey. Contemporary Clinical Trials, 26(3), 331–337. Google Scholar logo with link to Google Scholar
Andringa, S., & Godfroid, A. (2020). Sampling bias and the problem of generalizability in applied linguistics. Annual Review of Applied Linguistics, 40, 134–142. Google Scholar logo with link to Google Scholar
Crossley, S., Marsden, E., Ellis, N., Kormos, J., Morgan-Short, K., Thierry, G. (2020). Introduction of methods showcase articles in Language Learning. Language Learning, 70, 5–10. Google Scholar logo with link to Google Scholar
Crowther, D., Kim, S., Lee, J., Lim, J., & Loewen, S. (2021). Methodological synthesis of cluster analysis in second language research. Language Learning, 71, 99–130.Google Scholar logo with link to Google Scholar
De Costa, P. I. (Ed.). (2016). Ethics in applied linguistics research: Language researcher narratives. Routledge.Google Scholar logo with link to Google Scholar
De Costa, P. I., Sterling, S., Lee, J., Li, W., & Rawal, H. (2021). Research tasks on ethics in applied linguistics. Language Teaching, 54, 58–70. Google Scholar logo with link to Google Scholar
Farangi, M. R., & Nejadghanbar, H. (2024). Investigating questionable research practices among Iranian applied linguists: Prevalence, severity, and the role of artificial intelligence tools. System, 125, 103427.Google Scholar logo with link to Google Scholar
Fanelli, D. (2009). How many scientists fabricate and falsify research? A systematic review and meta-analysis of survey data. PLoS One, 4(5), e5738.Google Scholar logo with link to Google Scholar
Gass, S., Loewen, S., & Plonsky, L. (2021). Coming of age: The past, present, and future of quantitative SLA research. Language Teaching, 54, 245–258. Google Scholar logo with link to Google Scholar
Gass, S., & Plonsky, L. (2020). Introducing the SSLA methods forum. Studies in Second Language Acquisition, 42, 667–669. Google Scholar logo with link to Google Scholar
Gass, S. M., Plonsky, L., & Huntley, E. (2022). Taking the Long view: A bibliometric analysis. In A. Benati & J. W. Schwieter (Eds.), Second language acquisition as shaped by the scholarly legacy of Professor Michael Long (pp. 9–27). John Benjamins. Google Scholar logo with link to Google Scholar
Gries, S. Th. (2021). (Generalized linear) mixed effects modeling: A learner corpus example. Language Learning, 71, 757–798. Google Scholar logo with link to Google Scholar
Hall, J., & Martin, B. R. (2019). Towards a taxonomy of research misconduct: The case of business school research. Research Policy, 48, 414–427. Google Scholar logo with link to Google Scholar
Hu, Y., & Plonsky, L. (2021). Statistical assumptions in L2 research: A systematic review. Second Language Research, 37, 171–184. Google Scholar logo with link to Google Scholar
Isbell, D., Brown, D., Chen, M., Derrick, D., Ghanem, R., Gutiérrez Arvizu, M. N., Schnur, E., Zhang, M., & Plonsky, L. (2022). Misconduct and questionable research practices: The ethics of quantitative data handling and reporting in applied linguistics. Modern Language Journal, 106, 172–195. Google Scholar logo with link to Google Scholar
Jordan, S. R. (2013). Conceptual clarification and the task of improving research on academic ethics. Journal of Academic Ethics, 11, 243–256. Google Scholar logo with link to Google Scholar
Kumar, M. N. (2008). A review of the types of scientific misconduct in biomedical research. Journal of Academic Ethics, 6, 211–228. Google Scholar logo with link to Google Scholar
Larson-Hall, J. (2017). Moving beyond the bar plot and the line graph to create informative and attractive graphics. Modern Language Journal, 101, 244–270. Google Scholar logo with link to Google Scholar
Larsson T., Plonsky, L., & Hancock, G. R. (2021). On the benefits of structural equation modeling for corpus linguists. Corpus Linguistics and Linguistic Theory, 17, 683–714.Google Scholar logo with link to Google Scholar
Larsson, T., Plonsky, L., Sterling, S., Kytö, M., Yaw, K., & Wood, M. (2023). On the prevalence and perceived severity of questionable research practices. Research Methods in Applied Linguistics, 2, 100064. Google Scholar logo with link to Google Scholar
Loewen, S., Lavolette, E., Spino, S., Papi, M., Schmidtke, J., Sterling, S., & Wolff, D. (2014). Statistical literacy among applied linguists and second language acquisition researchers. TESOL Quarterly, 48, 360–388. Google Scholar logo with link to Google Scholar
Marsden, E., Morgan-Short, K., Thompson, S., & Abugaber, D. (2018). Replication in second language research: Narrative and systematic reviews and recommendations for the field. Language Learning, 68, 321–391. Google Scholar logo with link to Google Scholar
Marsden, E., & Plonsky, L. (2018). Data, open science, and methodological reform in second language acquisition research. In A. Gudmestad, & A. Edmonds (Eds.), Critical reflections on data in second language acquisition (pp. 219–228). John Benjamins. Google Scholar logo with link to Google Scholar
Norouzian, R., de Miranda, M. A., & Plonsky, L. (2018). The Bayesian revolution in second language research: An applied approach. Language Learning, 68, 1032–1075. Google Scholar logo with link to Google Scholar
Norris, J. M., Plonsky, L., Ross, S. J., & Schoonen, R. (2015). Guidelines for reporting quantitative methods and results in primary research. Language Learning, 65, 470–476. Google Scholar logo with link to Google Scholar
Plonsky, L. (2023). Sampling and generalizability in Lx research: A second order synthesis. Languages, 8, 75. Google Scholar logo with link to Google Scholar
(2024). Study quality as an intellectual and ethical imperative: A proposed framework. Annual Review of Applied Linguistics. Google Scholar logo with link to Google Scholar
Plonsky, L., Brown, D., Chen, M., Ghanem, R., Gutiérrez Arvizu, M. N., Isbell, D. R., & Zhang, M. (2024). “Significance sells”: Applied linguists’ view on questionable research practices. Research Methods in Applied Linguistics, 3, 100099. Google Scholar logo with link to Google Scholar
Porte, G., & McManus, K. (2019). Doing replication research in applied linguistics. Routledge.Google Scholar logo with link to Google Scholar
Resnik, D. B., Neal, T., Raymond, A., & Kissling, G. E., (2015). Research misconduct definitions adopted by U.S. research institutions. Accountability in Research, 22, 14–21.Google Scholar logo with link to Google Scholar
Rodríguez-Lifante, A., & Pereira, M. M. B. (2021). The Delphi method in applied linguistics in the light of a theoretical and critical analysis. Revista Brasileira de Linguística Aplicada, 21, 271–293. Google Scholar logo with link to Google Scholar
Steneck, N. H. (2006). Fostering integrity in research: Definitions, current knowledge, and future directions. Science and Engineering Ethics, 12(1), 53–74. Google Scholar logo with link to Google Scholar
Sterling, S., & Gass, S. (2017). Exploring the boundaries of research ethics: Perceptions of ethics and ethical behaviors in classroom-based applied linguistics research. System, 70, 50–62. Google Scholar logo with link to Google Scholar
Sterling, S., Plonsky, L., Larsson, T., Kytö, M., & Yaw, K. (2023). Introducing and illustrating the Delphi method for applied linguistics research. Research Methods in Applied Linguistics, 2, 100040. Google Scholar logo with link to Google Scholar
Sudina, E. (2023). Scale quality in second-language anxiety and WTC: A methodological synthesis. Studies in Second Language Acquisition, 47, 1427–1455. Google Scholar logo with link to Google Scholar
Tauginienė, L., Gaižauskaitė, I., Razi, S., Glendinning, I., Sivasubramaniam, S., Marino, F., Cosentino, M., Anohina-Naumeca, A., & Kravjar, J. (2019). Enhancing the taxonomies relating to academic integrity and misconduct. Journal of Academic Ethics, 17, 345–361. Google Scholar logo with link to Google Scholar
Turoff, M., & Linstone, H. A. (2002). The Delphi method: Techniques and applications. Addison-Wesley.Google Scholar logo with link to Google Scholar
Wood, M., Larsson, T., Plonsky, L., Sterling, S., Kytö, M., & Yaw, K. (2024). Addressing questionable research practices in applied linguistics: A practical guide. Applied Linguistics Press.Google Scholar logo with link to Google Scholar
Wood, M., Sterling, S., Larsson, T., Plonsky, L., Kytö, M., & Yaw, K. (in press). Researchers training researchers: Ethics training in Applied Linguistics. TESOL Quarterly.
Yaw, K., Plonsky, L., Larsson, T., Sterling, S., & Kytö, M. (2023). Timeline: Research ethics in applied linguistics. Language Teaching, 56, 478–494. Google Scholar logo with link to Google Scholar
Cited by (11)

Cited by 11 other publications

Babaii, Esmat
2025. The knowledge lost in information. Journal of Second Language Studies DOI logo
Dobrin, Lise M.
2025. Training for ethical linguistics: a model for building “responsible conduct of research” into department culture. Linguistics 63:2 ► pp. 557 ff. DOI logo
D’Addario, Michael
2025. A taxonomy of standardized terms for generative AI use in the composition classroom. Discover Education 4:1 DOI logo
Fernández-González, Carlos & Mónica Ledo
2025. Grit and L2 grit research in SLA (2013–2025). Studies in Second Language Acquisition 47:3 ► pp. 828 ff. DOI logo
McClelland, Neil & Jenifer Larson-Hall
2025. Why you should stop using the ideal L2 self and the L2 motivational self-system to measure motivation (Reaction to Al-Hoorie, Hiver & In’nami, 2024). Studies in Second Language Acquisition ► pp. 1 ff. DOI logo
Park, Hae In, Megan Solon & Kwangmin Lee
2025. Understanding proficiency assessment practices in SLA research. Studies in Second Language Acquisition ► pp. 1 ff. DOI logo
Plonsky, Luke, Scott Sterling, Katherine Yaw, Tove Larsson & Merja Kytö
2025. Expanding the scope of questionable research practices in applied linguistics. Journal of Second Language Studies DOI logo
Plonsky, Luke & Ekaterina Sudina
2025. Self-citation practices in applied linguistics. Journal of Second Language Studies DOI logo
Sterling, Scott, Kate Yaw, Luke Plonsky, Tove Larsson & Merja Kytö
2025. Investigating researcher perceptions of Questionable Research Practices. Journal of Second Language Studies DOI logo
Vitta, Joseph P., Aaron Hahn, Derek Canning, Daniel R. Isbell, Ali H. Al-Hoorie & Christopher Nicklin
2025. Conceptualization and frequency of sampling (mal)practices in L2 inferential quantitative research. Journal of Second Language Studies DOI logo
Wood, Margaret, Scott Sterling, Tove Larsson, Luke Plonsky, Merja Kytö & Kate Yaw
2025. Researchers Training Researchers: Ethics Training in Quantitative Applied Linguistics. TESOL Quarterly 59:3 ► pp. 1817 ff. DOI logo

This list is based on CrossRef data as of 20 november 2025. Please note that it may not be complete. Sources presented here have been supplied by the respective publishers. Any errors therein should be reported to them.

Mobile Menu Logo with link to supplementary files background Layer 1 prag Twitter_Logo_Blue