Article published In: Questionable Research Practices in Applied Linguistics
Edited by Luke Plonsky
[Journal of Second Language Studies 8:2] 2025
► pp. 409–422
The knowledge lost in information
Questionable research practices from the perspective of a manuscript reviewer
Published online: 22 April 2025
https://doi.org/10.1075/jsls.00046.bab
https://doi.org/10.1075/jsls.00046.bab
Abstract
The current position paper is a brief account of the pitfalls I have found in the research manuscripts I have reviewed over a period of twenty years. As a review panel member of several academic journals, I encountered several problems in the submissions I have been asked to review, both minor ones and those beyond repair. In this paper, I intend to report my observations with a focus on what I may call unsophisticated and simplistic treatment of the findings. To put it briefly, while some submissions are rightly rejected due to sloppy data collection, biased sampling, or erroneous use of statistics, there are papers that succeed in following the strict methodological do’s and don’ts of research but fail to make sense of the bulk of the collected data, leading to fixation at the lower levels of Data-Information-Knowledge-Wisdom (DIKW) hierarchy. I will try to address this issue, postulating that inadequate practice of critical thinking and other higher-order thinking skills such as analytical reasoning, evaluation, and inference could be partly responsible for this caveat. The paper ends with suggestions for educating would-be researchers not only by teaching the principles of conducting research but also by encouraging creativity, critical evaluation of information, and a genuine search for knowledge. Such qualities may not readily lend themselves to objective measurement and can hardly be translated into numerical indices by which research impact is estimated but they seem to add to the meaningfulness of research findings in the field.
Article outline
- 1.Introduction
- 2.Questionable research practices
- Cases 1, 2, and 3
- Cases 4, 5, and 6
- Cases 7, 8, and 9
- 3.Moving beyond data and information
- 4.Conclusion
- Acknowledgments
References
References (32)
Aksnes, D., Langfeldt, L., & Wouters, P. (2019). Citations, citation indicators, and research quality: An overview of basics, concepts and theories. SAGE Open, January-March 2019: 1–17.
Baltes, P., & Smith, J. (2008). The fascination of wisdom: Its nature, ontogeny, and function. Perspectives of Psychological Science, 3 (1), 56–64.
Baškarada, S., & Koronios, A. (2013). Data, Information, Knowledge, Wisdom (DIKW): A semiotic theoretical and empirical exploration of the hierarchy and its quality dimension. Australasian Journal of Information Systems, 18 (1), 5–24. [URL].
Bassey, M. (1995). Creating education through research: A global perspective of educational research for the 21st century. Kirklington Moor Press.
Bornmann, L., & Leydesdorff, L. (2014). Scientometrics in a changing research landscape. EMBO Reports, 15 (12), 1228–1232.
Carlisle, J. (2015). Continuing the DIKW Hierarchy Conversation. Midwest MWAIS 2015 Proceedings. 81. [URL]
Chaffey, D., & Wood, S. (2005). Business information management: Improving performance using information systems. Prentice Hall.
Curtis, G., & Cobham, D. (2005). Business information systems: Analysis, design and practice. Prentice Hall.
Egbert, J. (2007). Quality analysis of journals in TESOL and applied linguistics. TESOL Quarterly, 41, (1), 157–171.
Faintuch, J., & Faintuch, S. (2022). Past and current status of scientific, academic research fraud. In J. Faintuch, & S. Faintuch (Eds.), Integrity of scientific research fraud, misconduct and fake news in the academic, medical and social environment (pp. 3–8). Springer.
Fumerton, R. (2020). Is scientific knowledge special? Plus ça change, plus c’est la même chose. In K. McCain, & K. Kampourakis (Eds.), What is scientific knowledge: An introduction to contemporary epistemology of science (pp. 132–143). Routledge.
Isaac, T., & Chalmers, H. (2023). Reducing ‘avoidable research waste’ in applied linguistics research: Insights from healthcare research. Language Teaching, 1–18.
Isbell, D., Brown, D., Chen, M., Derrick, D., Ghanem, R., Gutiérrez Arvizu, M. N., Schnur, E., Zhang, M., & Plonsky, L. (2022). Misconduct and questionable research practices: The ethics of quantitative data handling and reporting in applied linguistics. Modern Language Journal, 1061, 172–195.
Larsson, T., Plonsky, L., Sterling, S., Kytö, M., Yaw, K., & Wood, M. (2023). On the prevalence and perceived severity of questionable research practices. Research Methods in Applied Linguistics, 21, 100064.
Laudon, K., & Laudon, J. (2014). Management information systems: Managing the digital firm. Pearson Prentice Hall.
Lee, I. (2014). Publish or perish: The myth and reality of academic publishing. Language Teaching, 47 (2), 250–261.
Levitt, H., & Piazza-Bonin, E. (2016). Wisdom and psychotherapy: Studying expert therapists’ clinical wisdom to explicate common processes. Psychotherapy Research, 26(1), 31–47.
Mausethagen, S., Prøitz, T., & Skedsmo, G. (2018). Teachers’ use of knowledge sources in ‘result meetings’: Thin data and thick data use. Teachers and Teaching, 24(1), 37–49.
Montoneri, B. (2020). Fake news and fake research, from the cave to the light: Critical reflection and literature review. In B. Montoneri (Ed.), Academic misconduct and plagiarism: Case studies from universities around the world (pp. 167–198). Lexington Books.
Murtonen, M., & Salmento, H. (2019). Broadening the theory of scientific thinking for higher education. In M. Murtonen, & K. Balloo, (Eds.), Redefining scientific thinking for higher education (pp. 3–29). Palgrave Macmillan.
Nederhof, A., Luwel, M., & Moed, H. (2001). Assessing the quality of scholarly journals in Linguistics: An alternative to citation-based journal impact factors. Scientometrics, 51 (1), 241–265. [URL]
Nygaard, L. (2017). Publishing and perishing: an academic literacies framework for investigating research productivity. Studies in Higher Education, 42 (3), 519–532.
Pearlson, K. & Saunders, C. (2004). Managing and using information systems: A strategic approach. Wiley.
Plonsky, L., Larsson, T., Sterling, S., Kytö, M., Yaw, K., & Wood, M. (2024). A taxonomy of questionable research practices in quantitative humanities. In P. De Costa, A. Rabie-Ahmed, & C. Cinaglia (Eds.) Ethical issues in applied linguistics scholarship (pp. 10–27). John Benjamins.
