Article published In: Translation, Cognition & Behavior
Vol. 5:1 (2022) ► pp.1–26
Considerations of ecological validity in cognitive translation and interpreting studies
Published online: 8 July 2022
https://doi.org/10.1075/tcb.00061.mel
https://doi.org/10.1075/tcb.00061.mel
Abstract
Renewed interest in research methods used in cognitive translation and interpreting studies (CTIS) has led to
increased reflection on the nature of research and experimental design, internal and external validity, and the type and nature of
experimental tasks. Of particular concern is the extent to which valid generalization can be made from empirical studies of
translation, interpreting, and their associated cognitive behaviors as objects of study. This article traces the definition and
history of ecological validity in the extant literature on research methods and philosophy of science, emphasizing three
considerations: the experimental setting, the stimuli under investigation, and the behavioral response of participants.
Subsequently, we discuss potential misunderstandings or misapplications of appeals to ecological validity, including mundane
realism, naturalistic tasks, and various data collection methods. A final section describes key points to consider for research in
CTIS. Throughout, the argument considers tradeoffs among various categories of validity as well as the importance of aligning
experimental design with research goals. This extensive engagement with a singular aspect of validity is provided to encourage
deeper reflection and better communication around the topic of ecological validity.
Article outline
- 1.Introduction
- 2.Definitions of ecological validity
- 2.1Alternative conception of ecological validity
- 3.Balancing risks to ecological validity
- 3.1Naturalistic tasks
- 3.2Mundane realism
- 3.3Data collection methods
- 4.Establishing ecological validity
- 4.1Define the relevant environment for experimentation and generalization
- 4.2Align the stimuli, variables, and measurement with the research purpose
- 4.3Consider influences and constraints on participant responses
- 5.Conclusion
References
References (89)
Alves, Fabio. 2015. “Translation
Process Research at the Interface: Paradigmatic, theoretical, and methodological issues in dialogue with cognitive science,
expertise studies, and psycholinguistics.” In Psycholinguistic and
Cognitive Inquiries into Translation and Interpreting. Edited by A. Ferreira, and J. W. Schwieter, 17–40. Amsterdam: John Benjamins.
Araújo, Duarte, Keith Davids, and Pedro Passos. 2007. “Ecological
Validity, Representative Design, and Correspondence between Experimental Task Constraints and Behavioral Setting: Comment on
Rogers, Kadar, and Costall (2005).” Ecological
Psychology 19 (1), 69–78.
Banaji, Mahzarin R., and Robert G. Crowder. 1989. “The
Bankruptcy of Everyday Memory.” American
Psychologist 44 (9), 1185–1193.
Baraldi, Claudio, and Christopher D. Mellinger. 2016. “Observations.” In Researching
Translation and Interpreting. Edited by C. V. Angelelli, and B. J. Baer, 257–268. New York: Routledge.
Berkowitz, Leonard, and Edward Donnerstein. 1982. “External
Validity is More than Skin Deep: Some answers to criticisms of laboratory
experiments.” American
Psychologist 37 (3), 245–257.
Blanco-Elorrieta, Esti, and Liina Pylkkänen. 2018. “Ecological
Validity in Bilingualism Research and the Bilingual Advantage.” Trends in Cognitive
Sciences 22 (12), 1117–1126.
Bolaños Medina, Alicia. 2016. “Translation
Psychology within the Framework of Translator Studies: New research
perspectives.” In From the Lab to the Classroom and Back Again:
Perspectives on Translation and Interpreting Training. Edited by C. Martín de León, and V. González-Ruíz, 59–100. Peter Lang.
Bracht, Glenn H., and Gene V. Glass. 1968. “The
External Validity of Experiments.” American Educational Research
Journal 5 (4), 437–474.
Bronfenbrenner, Urie. 1977. “Toward
an Experimental Ecology of Human Development.” American
Psychologist 32 (7), 513–531.
Brunswik, Egon. 1943. “Organismic
Achievement and Environmental Probability.” Psychological
Review 50 (3), 255–272.
. 1955. “Representative
Design and Probabilistic Theory in a Functional Psychology.” Psychological
Review 62 (3), 193–217.
Calder, Bobby J., Lynn W. Phillips, and Alice M. Tybout. 1981. “Designing
Research for Application.” Journal of Consumer
Research 81, 197–207.
Campbell, Donald T., and Julian C. Stanely. 1963. Experimental
and Quasi-experimental Designs for Research. Cengage.
Carlsmith, J. Merrill, Phoebe C. Ellsworth, and Eliot Aronson. 1976. Methods
of Research in Social Psychology. Addison-Wesley.
Chaytor, Naomi, and Maureen Schmitter-Edgecombe. 2003. “The
Ecological Validity of Neuropsychological Tests: A review of the literature on everyday cognitive
skills.” Neuropsychology
Review 13 (4), 181–197.
Chow, Siu L. 1987. “Science, Ecological Validity,
and Experimentation.” Journal for the Theory of Social
Behavior 17 (2), 181–194.
Cook, Thomas D., and Donald T. Campbell. 1979. Quasi-experimentation:
Design & analysis issues for field settings. Houghton Mifflin.
Dipboye, Robert L., and Michael F. Flanagan. 1980. “Reply
to Willems and Howard.” American
Psychologist 351, 388–390.
Dobbins, Gregory H., Irving M. Lane, and Dirk D. Steiner. 1988. “A
Note on the Role of Laboratory Methodologies in Applied Behavioural Research: Don’t throw out the baby with the bath
water.” Journal of Organizational
Behavior 9 (3), 281–286.
Downie, Jonathan. 2021. “Interpreting
is Interpreting: Why we need to leave behind interpreting settings to discover comparative interpreting
studies.” Translation and Interpreting
Studies 16 (3), 325–346.
Dunlosky, John, Sara Bottiroli, and Marissa Hartwig. 2009. “Sins
Committed in the Name of Ecological Validity: A call for representative design in education
science.” In Handbook of Metacognition in
Education. Edited by D. J. Hacker, J. Dunlosky, and A. C. Graesser, 430–440. New York: Routledge.
Ehrensberger-Dow, Maureen, and Gary Massey. 2020. “Translation
Workplace-based Research.” In The Routledge Handbook of Translation
and Technology. Edited by M. O’Hagan, 354–369. New York: Routledge.
Franzen, Michael D., and Karen L. Wilhelm. 1996. “Conceptual
Foundations of Ecological Validity in Neuropsychological
Assessment.” In Ecological Validity of Neuropsychological
Testing. Edited by R. J. Sbordone, and C. J. Long, 91–112. New York: St. Lucie Press.
García, Adolfo M. 2019. The Neurocognition of Translation and
Interpreting. John Benjamins.
Gerber, Leah, Jim Hlavac, Irwyn Shepherd, Paul McIntosh, Alex Avella Archila, and Hyein Cho. 2021. “Stepping
into the Future: Virtual reality training for community interpreters working in the area of family
violence.” Journal of Specialised
Translation 361, 252–275.
Gibson, James J. 1957. “Survival in a World of Probable
Objects. Review of perception and the representative design of psychological experiment by E.
Brunswik.” Contemporary
Psychology 21, 33–35.
Gile, Daniel. 2001. “Consecutive
vs. Simultaneous: Which is more accurate?” Interpretation
Studies 11, 8–20.
. 2016. “Experimental
Research.” In Researching Translation and
Interpreting. Edited by C. V. Angelelli, and B. J. Baer, 220–228. New York: Routledge.
. 2018. “Research
into Translation as a Specialism: An analysis and recommendations.” Journal of Specialised
Translation 301, 23–39.
Hale, Sandra, Jane Goodman-Delahunty, Natalie Martschuk, and Stephen Doherty. 2021. “The
Effects of Mode on Interpreting Performance in a Simulated Police Interview.” Translation and
Interpreting Studies.
Hammond, Kenneth R. 1998. “Ecological Validity:
Then and Now.” Available at [URL]
Hammond, Kenneth R., and Thomas R. Stewart. 2001. “Introduction.” In The
Essential Brunswik: Beginnings, Explications, Applications. Edited by K. R. Hammond, and T. R. Stewart, 3–11. New York: Oxford University Press.
Heilmann, Arndt, Tatiana Serbina, Daniel Couto Vale, and Stella Neumann. 2019. “Shorter
than a Text, Longer than a Sentence: Source text length for ecologically valid translation
experiments.” Target 31 (1), 98–124.
Highhouse, Scott. 2009. “Designing
Experiments that Generalize.” Organizational Research
Methods 12 (3), 554–566.
Hoc, Jean-Michel. 2001. “Towards
Ecological Validity of Research in Cognitive Ergonomics.” Theoretical Issues in Ergonomic
Science 2 (3), 278–288.
Holleman, Gijs A., Ignace T. C. Hooge, Chantal Kemner, and Roy S. Hessels. 2020. “The
“Real-World Approach” and its Problems: A critique of the term ecological validity.” Frontiers
in Psychology 111, 721.
House, Juliane. 2013. “Towards
a New Linguistic-Cognitive Orientation in Translation
Studies.” Target 25 (1), 46–60.
Jääskeläinen, Riitta. 2012. “Translation
Psychology.” In Handbook of Translation
Studies. Edited by Y. Gambier, and L. van Doorslaer, 191–197. Amsterdam: John Benjamins.
. 2017. “Verbal
Reports.” In Handbook of Translation and
Cognition. Edited by J. W. Schwieter, and A. Ferreira, 213–231. Hoboken, NJ: Wiley.
Jakobsen, Arnt L. 2020. “Translation Technology Research
with Eye Tracking.” In The Routledge Handbook of Translation and
Technology. Edited by M. O’Hagan, 398–416. New York: Routledge.
Kihlstrom, John F. 2021. “Ecological Validity and
‘Ecological Validity.’” Perspectives on Psychological
Science 16 (2), 466–471.
Korpal, Pawel. 2015. “Eye-tracking
in Translation and Interpreting Studies: The growing popularity and methodological
problems.” In Accessing Audiovisual
Translation. Edited by L. Bogucki, and M. Deckert, 199–212. Frankfurt: Peter Lang.
Kothgassner, Oswald D., and Anna Felnhofer. 2020. “Does
Virtual Reality Help to Cut the Gordian Knot between Ecological Validity and Experimental
Control?” Annals of the International Communication
Association 44 (3), 210–218.
Kvavilashvili, Lia, and Judi A. Ellis. 2004. “Ecological
Validity and the Real-life/Laboratory Controversy in Memory Research: A critical (and historical)
review.” History and Philosophy of
Psychology 6 (1), 59–80.
Ladouce, Simon, David I. Donaldson, Paul A. Dudchenko, and Magdalena Ietswaart. 2017. “Understanding
Minds in Real-world Environments: Toward a mobile cognition approach.” Frontiers in Human
Neuroscience 101, 694.
Lewkowicz, David J. 2001. “The Concept of Ecological
Validity: What are its limitations and is it bad to be
invalid?” Infancy 2 (4), 437–450.
Lucas, Jeffrey W. 2003. “Theory-testing, Generalization,
and the Problem of External Validity.” Sociological
Theory 21 (3), 236–253.
Lund, Thorleif. 2021. “A
Revision of the Campbellian Validity System.” Scandinavian Journal of Educational
Research 65 (3), 523–535.
Lynch, John G., Jr. 1999. “Theory
and External Validity.” Journal of the Academy of Marketing
Science 27 (3), 367–376.
Malamatidou, Sofia. 2018. Corpus
Triangulation: Combining Data and Methods in Corpus-based Translation
Studies. London: Routledge.
Marín García, Álvaro. 2019. “The
Opportunities of Epistemic Pluralism for Cognitive Translation Studies.” Translation,
Cognition &
Behavior 2 (2), 165–185.
Matt, G. E., A. Brewer, and M. Sklar. 2010. “External
validity.” In International Encyclopedia of
Education, 3rd Ed. Edited by P. Peterson, E. Baker, and B. McGaw, 521–527. Elsevier Science.
McGrath, Joseph E., and David Brinberg. 1983. “External
Validity and the Research Process: A comment on the Calder/Lynch dialogue.” Journal of Consumer
Research 10 (1), 115–124.
Mellinger, Christopher D. 2015. “On the Applicability of
Internet-mediated Research Methods to Investigate Translators’ Cognitive Behavior.” Translation
&
Interpreting 7 (1), 59–71.
2020. “Positionality in Public Service
Interpreting Research.” FITISPos International
Journal 71, 92–109.
Mellinger, Christopher D., and Thomas A. Hanson. 2017. Quantitative
Research Methods in Translation and Interpreting Studies. New York: Routledge.
. 2020. “Methodological
Considerations for Survey Research: Validity, Reliability, and Quantitative
Analysis.” Linguistica Antverpiensia, New Series – Themes in
Translation Studies 191, 172–190.
Mellinger, Christopher D., Nicoletta Spinolo, Maureen Ehrensberger-Dow, and Sharon O’Brien. Forthcoming. “Designing
Naturalistic Tasks.” In Innovative Data Collection Methods in
Cognitive Translation and Interpreting Studies. Edited by A. Rojo and R. Muñoz Martín. Amsterdam: John Benjamins.
Muñoz Martín, Ricardo. 2010. “On
Paradigms and Cognitive Translatology.” In Translation and
Cognition. Edited by G. M. Shreve and E. Angelone, 169–188. Amsterdam: John Benjamins.
. 2016. “Of
Minds and Men–Computers and Translators.” Poznań Studies in Contemporary
Linguistics 52 (2), 351–381.
. 2017. “Looking
Toward the Future of Cognitive Translation Studies.” In The Handbook
of Translation and Cognition. Edited by J. W. Schwieter, and A. Ferreira, 554–572. Hoboken, NJ: Wiley.
Neguţ, Alexandra, Silviu-Andrei Matu, Florin Alin Sava, and Daniel David. 2016. “Virtual
Reality Measures in Neuropsychological Assessment: A meta-analytic review.” The Clinical
Neuropsychologist 30 (2), 165–184.
O’Brien, Sharon. 2013. “The
Borrowers: researching the cognitive aspects of
translation.” Target 25 (1), 5–17.
Orne, Martin T. 1962. “On the Social Psychology of the
Psychological Experiment: With particular reference to demand characteristics and their
implications.” American
Psychologist 17 (11), 776–783.
Parsons, Thomas D. 2016. Clinical Neuropsychology and
Technology: What’s New and How We Can Use It. Cham, Switzerland: Springer.
Peecher, Mark E., and Ira Solomon. 2001. “Theory
and Experimentation in Studies of Audit Judgments and Decisions: Avoiding common research
traps.” International Journal of
Auditing 5 (3), 193–203.
Risku, Hanna, Regina Rogl, and Jelena Milošević. 2020. “Researching
Workplaces.” In The Bloomsbury Companion to Language Industry
Studies. Edited by E. Angelone, M. Ehrensberger-Dow, and G. Massey, 37–62. London: Bloomsbury.
Rojo López, Ana M., and Pawel Korpal. 2020. “Through
your Skin to your Heart and Brain: A Critical Evaluation of Physiological Methods in Cognitive Translation and Interpreting
Studies.” Linguistica Antverpiensia, New Series: Themes in Translation
Studies 191, 191–217.
Saldanha, Gabriela, and Sharon O’Brien. 2013. Research
Methodologies in Translation
Studies. London: Routledge.
Schmuckler, Mark A. 2001. “What is Ecological Validity? A
dimensional
analysis.” Infancy, 2 (4), 419–436.
Shadish, William R., Thomas D. Cook, and Donald T. Campbell. 2002. Experimental
and Quasi-experimental Designs for Generalized Causal
Inference. Cengage.
Shreve, Gregory M., and Erik Angelone. 2010. “Translation
and Cognition: Recent developments.” In Translation and
Cognition. Edited by G. M. Shreve, and E. Angelone, 1–13. Amsterdam: John Benjamins.
Spooner, Donna M., and Nancy A. Pachana. 2006. “Ecological
Validity in Neuropsychological assessment: A case for greater consideration in research with neurologically intact
populations.” Archives of Clinical
Neuropsychology 211, 327–337.
Steiner, Erich. 2021. “Translation,
Equivalence, and Cognition.” In The Routledge Handbook of Translation
and Cognition. Edited by F. Alves, and A. L. Jakobsen, 344–359. New York: Routledge.
Stoffregen, Thomas A. 1993. “‘Natural,’ ‘Real,’ and the Use
of Non-physical Displays in Perception-Action Research.” ISEP
Newsletter 61, 4–9.
Stoffregen, Thomas A., Benoit Bardy, L. J. Smart, and Randy J. Pagulayan. 2003. “On
the Nature and Evaluation of Fidelity in Virtual
Environments.” In Virtual and Adaptive Environments: Applications,
Implications, and Human Performance Issues. Edited by L. J. Hettinger, and M. W. Haas, 111–128. Mahwah, NJ: Lawrence Erlbaum.
Sun, Sanjun, Tian Li, and Xiaoyan Zhou. 2020. “Effects
of Thinking Aloud on Cognitive Effort in Translation.” Linguistica
Antverpiensia, New Series – Themes in Translation
Studies 191, 132–151.
Tiselius, Elisabet. 2021. “Informed
Consent: An Overlooked Part of Ethical Research in Interpreting
Studies.” InContext 1 (1), 83–100.
Wachtel, Paul L. 1980. “Investigation and its
Discontents: Some Constraints on Progress in Psychological Research.” American
Psychologist 35 (5), 399–408.
Weng, Yu, and Binghan Zheng. 2020. “A
Multi-Methodological Approach to Studying Time-Pressure in Written Translation: Manipulation and
measurement.” Linguistica Antverpiensia, New Series: Themes in
Translation Studies 191, 218–236.
Winer, Russell S. 1999. “Experimentation in the 21st
Century: The importance of external validity.” Journal of the Academy of Marketing
Science 27 (3), 349–358.
Cited by (22)
Cited by 22 other publications
Korpal, Paweł & Christopher D. Mellinger
Mellinger, Christopher D., Nicoletta Spinolo, Maureen Ehrensberger-Dow & Sharon O’Brien
2025. Designing studies with naturalistic tasks. In Research Methods in Cognitive Translation and Interpreting Studies [Research Methods in Applied Linguistics, 10], ► pp. 49 ff.
O'Brien, Sharon & Christopher D. Mellinger
Valdez, Susana, Leticia Pablos Robles & Karin van den Berg
2025. The reception of translated vaccination information. Target. International Journal of Translation Studies 37:2 ► pp. 213 ff.
Zong, Xueyan, Lei Song & Shanshan Yang
Bø, Vibeke
2024. Investigating interpreter-mediated interaction through the lens of depictions, descriptions, and indications. Translation and Interpreting Studies 19:3 ► pp. 335 ff.
Freiwald, Jonas, Zoë Miljanović, Arndt Heilmann & Stella Neumann
Gu, Chonglong & Binhua Wang
2024. From “Within” to “Beyond” in interpreting studies. Babel. Revue internationale de la traduction / International Journal of Translation / Revista Internacional de Traducción 70:6 ► pp. 783 ff.
Kuchinsky, Stefanie E., Ian Phillips & Rebecca E. Bieber
Ma, Xingcheng & Dechao Li
2024. Effect of word order asymmetry on the cognitive load of English–Chinese sight translation. Translation and Interpreting Studies 19:1 ► pp. 105 ff.
Su, Wenchao, Defeng Li & Jing Ning
Zou, Deyan & Jiahao Guo
Barnett, Michael D., Danielle R. Hardesty, Rebekah A. Griffin & Thomas D. Parsons
Božović, Petar
2023. How should metaphors be rendered in audiovisual translation?. Translation and Interpreting Studies 18:3 ► pp. 471 ff.
Cai, Rendong, Jiexuan Lin & Yanping Dong
Chang, Vincent Chieh-Ying & I-Fei Chen
Jia, Huihuang
Michael, Carl
Su, Wenchao
Zou, Longhui, Michael Carl & Devin Gilbert
Chmiel, Agnieszka & Nicoletta Spinolo
2022. Testing the impact of remote interpreting settings on interpreter experience and performance. Translation, Cognition & Behavior 5:2 ► pp. 250 ff.
This list is based on CrossRef data as of 5 december 2025. Please note that it may not be complete. Sources presented here have been supplied by the respective publishers. Any errors therein should be reported to them.
