Cover not available

Article published In: The Mental Lexicon
Vol. 17:2 (2022) ► pp.178–212

Get fulltext from our e-platform
JBE logo with link
PDF logo with download linkDownload PDF
EPUB logo with download linkDownload EPUB

A note on the modeling of the effects of experimental time in psycholinguistic experiments

 | University of Tübingen
 | University of Bristol
 | University of Edinburgh
 | University of Tübingen
Open Access logo
Available under the Creative Commons Attribution (CC BY) 4.0 license.

For any use beyond this license, please contact the publisher at rights@benjamins.nl.

Published online: 4 April 2022
DOI logo with link
https://doi.org/10.1075/ml.21012.baa
Abstract
Thul, R., Conklin, K., and Barr, D. J. (2020). Using gamms to model trial-by-trial fluctuations in experimental data: More risks but hardly any benefit. Open Science Framework. downloaded October 2020. Google Scholar logo with link to Google Scholar called attention to problems that arise when chronometric experiments implementing specific factorial designs are analysed with the generalized additive mixed model (GAMM), using factor smooths to capture trial-to-trial dependencies. From a series of simulations incorporating such dependencies, they conclude that GAMMs are inappropriate for between-subject designs. They argue that in addition GAMMs come with too many modeling possibilities, and advise using the linear mixed model (LMM) instead. As clarified by the title of their paper, their conclusion is: “Using GAMMs to model trial-by-trial fluctuations in experimental data: More risks but hardly any benefit”.
We address the questions raised by Thul, R., Conklin, K., and Barr, D. J. (2020). Using gamms to model trial-by-trial fluctuations in experimental data: More risks but hardly any benefit. Open Science Framework. downloaded October 2020. Google Scholar logo with link to Google Scholar, who clearly demonstrated that problems can indeed arise when using factor smooths in combination with factorial designs. We show that the problem does not arise when using by-smooths. Furthermore, we have traced a bug in the implementation of factor smooths in the mgcv package, which will have been removed from version 1.8–36 onwards.
To illustrate that GAMMs now produce correct estimates, we report simulation studies implementing different by-subject longitudinal effects. The maximal LMM emerges as slightly conservative compared to GAMMs, and GAMMs provide estimated coefficients that can be less variable across simulation runs. We also discuss two datasets where time-varying effects interact with numerical predictors in a theoretically informative way.
Article outline
  • 1.Introduction
  • 2.Sine waves
    • 2.1Sine waves with varying amplitudes
    • 2.2Sine waves with varying phase
  • 3.Power and Type-I error rate
    • 3.1Sine waves with varying absolute amplitude
    • 3.2Sine waves with varying phase
    • 3.3Simulations with the parameters of Thul et al. (2021)
    • 3.4From sine waves to random time-varying effects
  • 4.Interactions with time in experimental data
  • 5.Discussion
  • 6.Open practices statement
  • Acknowledgements
  • Notes
  • Mathematical notation
  • References
References (24)
References
Baayen, R. H. (2010). The directed compound graph of English. an exploration of lexical connectivity and its processing consequences. In Olsen, S., editor, New impulses in word-formation (Linguistische Berichte Sonderheft 17), pages 383–402. Buske, Hamburg.Google Scholar logo with link to Google Scholar
Baayen, R. H., van Rij, J., de Cat, C., and Wood, S. N. (2017a). Autocorrelated errors in experimental data in the language sciences: Some solutions offered by generalized additive mixed models. In Speelman, D., Heylen, K., and Geeraerts, D., editors, Mixed Effects Regression Models in Linguistics, pages 49–69. Springer, Berlin.Google Scholar logo with link to Google Scholar
Baayen, R. H., Vasishth, S., Bates, D., and Kliegl, R. (2017b). The cave of shadows. Addressing the human factor with generalized additive mixed models. Journal of Memory and Language, 941:206–234. Google Scholar logo with link to Google Scholar
Barr, D. J., Levy, R., Scheepers, C., and Tily, H. J. (2013). Random effects structure for confirmatory hypothesis testing: Keep it maximal. Journal of Memory and Language, 68(3):255–278. Google Scholar logo with link to Google Scholar
Bates, D. M., Kliegl, R., Vasishth, S., and Baayen, R. H. (2015). Parsimonious mixed models. arXiv.org.Google Scholar logo with link to Google Scholar
Chuang, Y.-Y. (2017). The effect of phonetic variation on word recognition in Taiwan Mandarin. PhD thesis, National Taiwan University, Taipei.
Chuang, Y.-Y., Fon, J., Papakyritsis, I., and Baayen, R. H. (2021). Analyzing phonetic data with generalized additive mixed models. In Ball, M. J., editor, Manual of Clinical Phonetics. Routledge. Google Scholar logo with link to Google Scholar
Francis, G. (2012). Publication bias and the failure of replication in experimental psychology. Psychonomic Bulletin & Review, 19(6):975–991. Google Scholar logo with link to Google Scholar
Harm, M. W. and Seidenberg, M. S. (2004). Computing the meanings of words in reading: Cooperative division of labor between visual and phonological processes. Psychological Review, 1111:662–720. Google Scholar logo with link to Google Scholar
Hastie, T. and Tibshirani, R. (1990). Generalized Additive Models. Chapman & Hall, London.Google Scholar logo with link to Google Scholar
Keuleers, E., Lacey, P., Rastle, K., and Brysbaert, M. (2012). The British Lexicon Project: Lexical decision data for 28,730 monosyllabic and disyllabic English words. Behavior Research Methods, 441:287–304. Google Scholar logo with link to Google Scholar
Matuschek, H., Kliegl, R., Vasishth, S., Baayen, R. H., and Bates, D. M. (2017). Balancing Type I Error and Power in Linear Mixed Models. Journal of Memory and Language. Google Scholar logo with link to Google Scholar
Mock, P., Tibus, M., Ehlis, A.-C., Baayen, H., and Gerjets, P. (2018). Predicting adhd risk from touch interaction data. In Proceedings of the 20th ACM International Conference on Multimodal Interaction, pages 446–454. Google Scholar logo with link to Google Scholar
Open Science Collaboration (2015). Estimating the reproducibility of psychological science. Science, 349(6251):aac4716. Google Scholar logo with link to Google Scholar
Pinheiro, J. C. and Bates, D. M. (2000). Mixed-effects models in S and S-PLUS. Statistics and Computing. Springer, New York. Google Scholar logo with link to Google Scholar
Segalowitz, N. S. and Segalowitz, S. J. (1993). Skilled performance, practice, and the differentiation of speed-up from automatization effects: Evidence from second language word recognition. Applied Psycholinguistics, 14(03):369–385. Google Scholar logo with link to Google Scholar
Thul, R., Conklin, K., and Barr, D. J. (2020). Using gamms to model trial-by-trial fluctuations in experimental data: More risks but hardly any benefit. Open Science Framework. downloaded October 2020. Google Scholar logo with link to Google Scholar
(2021). Using gamms to model trial-by-trial fluctuations in experimental data: More risks but hardly any benefit. Journal of Memory and Language, 1201:104247. Google Scholar logo with link to Google Scholar
van Rij, J., Hendriks, P., van Rijn, H., Baayen, R. H., and Wood, S. N. (2019). Analyzing the time course of pupillometric data. Trends in hearing, 231:2331216519832483. Google Scholar logo with link to Google Scholar
Westfall, J., Kenny, D. A., and Judd, C. M. (2014). Statistical power and optimal design in experiments in which samples of participants respond to samples of stimuli. Journal of Experimental Psychology: General, 143(5):2020. Google Scholar logo with link to Google Scholar
Wieling, M. (2018). Analyzing dynamic phonetic data using generalized additive mixed modeling: a tutorial focusing on articulatory differences between l1 and l2 speakers of english. Journal of Phonetics, 701:86–116. Google Scholar logo with link to Google Scholar
Wood, S. N. (2013). On p-values for smooth components of an extended generalized additive model. Biometrika, 1001:221–228. Google Scholar logo with link to Google Scholar
(2016). Just Another Gibbs Additive Modeller: Interfacing JAGS and mgcv. arXiv preprint arXiv:1602.02539v1.Google Scholar logo with link to Google Scholar
(2017). Generalized Additive Models. Chapman & Hall/CRC, New York. Google Scholar logo with link to Google Scholar
Cited by (7)

Cited by seven other publications

Chen, Yingzhao & Danni Zhang
2025. Does data collection mode matter for word learning trajectory: processing variability in lab and web environments. Research Methods in Applied Linguistics 4:3 ► pp. 100262 ff. DOI logo
Cho, Sun-Joo, Sarah Brown-Schmidt, Sharice Clough & Melissa C. Duff
2025. Comparing Functional Trend and Learning among Groups in Intensive Binary Longitudinal Eye-Tracking Data using By-Variable Smooth Functions of GAMM. Psychometrika 90:2 ► pp. 628 ff. DOI logo
Widmann, Andreas, Björn Herrmann & Florian Scharf
2025. Pupillometry is sensitive to speech masking during story listening: A commentary on the critical role of modeling temporal trends. Journal of Neuroscience Methods 413 ► pp. 110299 ff. DOI logo
Winn, Matthew B.
2024. The Effort of Repairing a Misperceived Word Can Impair Perception of Following Words, Especially for Listeners With Cochlear Implants. Ear & Hearing 45:6 ► pp. 1527 ff. DOI logo
Heitmeier, Maria, Yu-Ying Chuang & R. Harald Baayen
2023. How trial-to-trial learning shapes mappings in the mental lexicon: Modelling lexical decision with linear discriminative learning. Cognitive Psychology 146 ► pp. 101598 ff. DOI logo
Ke, Sihui (Echo), Rui Jin & Keiko Koda
2023. Adult L2 learners’ morphological sensitivity in a morphosyllabic language. The Mental Lexicon 18:3 ► pp. 446 ff. DOI logo
Johns, Michael A. & Paola E. Dussias
2022. Comparing Single-Word Insertions and Multi-Word Alternations in Bilingual Speech: Insights from Pupillometry. Languages 7:4 ► pp. 267 ff. DOI logo

This list is based on CrossRef data as of 27 november 2025. Please note that it may not be complete. Sources presented here have been supplied by the respective publishers. Any errors therein should be reported to them.

Mobile Menu Logo with link to supplementary files background Layer 1 prag Twitter_Logo_Blue