Cover not available

Article published In: Study Abroad Research in Second Language Acquisition and International Education
Vol. 9:2 (2024) ► pp.217–244

Get fulltext from our e-platform
JBE logo with link
PDF logo with download linkDownload PDF
EPUB logo with download linkDownload EPUB
authorization required

Introducing regression discontinuity design to applied linguistics

The case of Japanese government study-abroad scholarships

ORCiD logo with link | Waseda University
 | Sophia University
 | Keio University
 | University of Melbourne
 | Kansai University
Published online: 13 May 2024
DOI logo with link
https://doi.org/10.1075/sar.21020.sas
Abstract
This report presents regression discontinuity design (RDD) as a powerful analytical tool for use in applied linguistics showcased through our study of the impact of Japanese government study-abroad (SA) scholarships. RDD enables the estimation of causal effects in scenarios where a true experiment is not feasible by exploiting a naturally occurring cutoff point for treatment assignment. Because RDD may be novel to most readers in applied linguistics, this report provides a detailed step-by-step explanation of the standard RDD procedures our study exemplifies. The results section is crafted to reflect the conventional presentation style of RDD findings. Furthermore, the report’s concluding segment suggests scenarios within SA-related language learning research that could benefit from RDD application by enhancing the nuanced and precise interpretation of data in applied linguistics contexts.
Article outline
  • 1.Introduction
  • 2.Past studies of impacts of SA experiences with special focus on methodology
  • 3.Regression discontinuity design (RDD)
    • 3.1Background
    • 3.2Standard RDD procedure
      • 3.2.1Step 1: Defining the assignment variable and cutoff
      • 3.2.2Step 2: Identifying the control group and ensuring continuity
      • 3.2.3Step 3: Choosing the bandwidth
      • 3.2.4Step 4: Graphical analysis and subsequent statistical testing
      • 3.2.5Step 5: Assessing the density of the assignment variable
      • 3.2.6Step 6: Determining robustness through multiple bandwidths
      • 3.2.7Step 7: Supplementing RDD with other statistical methods
      • 3.2.8Step 8: Interpreting reduced-form models
      • 3.2.9Step 9: Stage 1 for conducting the instrumental variable (IV) approach to estimate local average treatment effect (LATE)
      • 3.2.10Step 10: IV approach Stage 2 to estimating LATE
      • 3.2.11Step 11: Analyzing heterogeneous treatment effects
  • 4.Illustrative study: Effects of SA scholarships
    • 4.1Background
    • 4.2Outcome (dependent) variables
    • 4.3Research questions
    • 4.4Method
      • 4.4.1Participants
      • 4.4.2Outcome (dependent) variables
        • 4.4.2.1Perceived (Communication) Competence in the SA country’s main language
        • 4.4.2.2International Posture
      • 4.4.3Data collection procedure
      • 4.4.4Analyses
    • 4.5Results
    • 4.5.1Numbers of SA months between pre-SA and post-SA surveys
    • 4.5.2Perceived Competence in the SA country’s main language
    • 4.5.3International Posture
    • 4.5.4Regression results: Reduced-form models, IV-LATE, and heterogeneity of treatment effects
  • 4.6Discussion
  • 5.Conclusion and suggestions for possible application of RDD to applied linguistics studies
  • Notes
  • References
References (37)
References
Angrist, J. D., & Pischke, J. S. (2010). The credibility revolution in empirical economics: How better research design is taking the con out of econometrics. Journal of Economic Perspectives, 24(2), 3–30. Google Scholar logo with link to Google Scholar
Bachman, L. F. (1990). Fundamental considerations in language testing. Oxford University Press.Google Scholar logo with link to Google Scholar
Calonico, S., Cattaneo, M. D., & Titiunik, R. (2014). Robust nonparametric confidence intervals for regression-discontinuity designs. Econometrica, 82(4), 2295–2326. Google Scholar logo with link to Google Scholar
Cattaneo, M. D., Idrobo, N., & Titiunik, R. (2019). A practical introduction to regression discontinuity designs: Foundations. Cambridge University Press. Google Scholar logo with link to Google Scholar
De Poli, S., Vergolini, L., & Zanini, N. (2018). The impact of a study abroad program on learning abilities and personality traits: Evidence from a randomization. Applied Economics Letters, 25(8), 562–566. Google Scholar logo with link to Google Scholar
Di Pietro, G. D. (2015). Do study abroad programs enhance the employability of graduates? Education Finance and Policy, 10(2), 223–243. Google Scholar logo with link to Google Scholar
Dörnyei, Z. (2014). Researching complex dynamic systems: Retrodictive qualitative modeling in the language classroom. Language Teaching, 47(1), 80–91. Google Scholar logo with link to Google Scholar
Geeslin, K. L., & Schmidt, L. B. (2018). Study abroad and L2 learner attitudes. In C. Sanz & A. Morales-Font (Eds.), The Routledge handbook of study abroad research and practice (pp. 387–405). Routledge. Google Scholar logo with link to Google Scholar
Gelman, A., & Imbens, G. (2019). Why high-order polynomials should not be used in regression discontinuity designs. Journal of Business & Economic Statistics, 37(3), 447–456. Google Scholar logo with link to Google Scholar
Gertler, P. J., Martínez, S., Premand, P., Rawlings, L. B., & Vermeersch, C. M. J. (2016). Impact evaluation in practice (2nd ed.). World Bank Group.Google Scholar logo with link to Google Scholar
Grey, S. (2018). Quantitative approaches for study abroad research. In C. Sanz & A. Morales-Font (Eds.), The Routledge handbook of study abroad research and practice (pp. 48–57). Routledge. Google Scholar logo with link to Google Scholar
Higuchi, Y., Nakamuro, M., Roever, C., Sasaki, M., & Yashima, T. (2023). Impact of studying abroad on language skill development: Regression discontinuity evidence from Japanese university students. Journal of the Japanese and International Economies, 701, 101284. Google Scholar logo with link to Google Scholar
Imbens, G. W., & Kalyanaraman, K. (2012). Optimal bandwidth choice for the regression discontinuity estimator. Review of Economic Studies, 79(3), 933–959. Google Scholar logo with link to Google Scholar
Imbens, G. W. & Lemieux, T. (2008). Regression discontinuity designs: A guide to practice. Journal of Econometrics, 142(2), 615–635. Google Scholar logo with link to Google Scholar
Kato, M., & Suzuki, K. (2019). Effective or self-selective: Random assignment demonstrates short-term study abroad effectively encourages further study abroad. Journal of Studies in International Education, 23(4), 411–428. Google Scholar logo with link to Google Scholar
Kawaguchi, D. (2016). Fewer school days, more inequality. Journal of the Japanese and International Economies, 391, 35–52. Google Scholar logo with link to Google Scholar
Lee, D. S., & Lemieux, T. (2010). Regression discontinuity designs in economics. Journal of Economic Literature, 48(2), 281–355. Google Scholar logo with link to Google Scholar
MacIntyre, P. D., & Charos, C. (1996). Personality, attitudes, and affect as predictors of second language communication. Journal of Language and Social Psychology, 15(1), 3–26. Google Scholar logo with link to Google Scholar
McCrary, J. (2008). Manipulation of the running variable in the regression discontinuity design: A density test. Journal of Econometrics, 142(2), 698–714. Google Scholar logo with link to Google Scholar
Ministry of Education, Culture, Sports, Science, and Technology, Japan (MEXT). (2022a). About Tobitate! [Leap for Tomorrow] Study Abroad Initiative. Retrieved February 24, 2022, from [URL]
. (2022b). Tobitate! [Leap for Tomorrow] Ryugaku [Study Abroad] Japan. Retrieved February 24, 2022, from [URL]
Oosterbeek, H., & Webbink, D. (2011). Does studying abroad induce a brain drain? Economica, 78(310), 347–366. Google Scholar logo with link to Google Scholar
Parey, M., & Waldinger, F. (2010). Studying abroad and the effect on international labor market mobility: Evidence from the introduction of ERASMUS. Economic Journal, 121(551), 194–222. Google Scholar logo with link to Google Scholar
Robinson, J. P. (2011). Evaluating criteria for English learner reclassification: A causal-effects approach using a binding-score regression discontinuity design with instrumental variables. Educational Evaluation and Policy Analysis, 33(3), 267–292. Google Scholar logo with link to Google Scholar
Sanz, C. & Morales-Front, A. (2018). Issues in study abroad research and practice. In C. Sanz & A. Morales-Font (Eds.), The Routledge handbook of study abroad research and practice (pp. 1–16). Routledge. Google Scholar logo with link to Google Scholar
Shadish, W. R., Cook, T. D., & Campbell, D. T. (2002). Experimental and quasi-experimental designs for generalized causal inference. Houghton Mifflin.Google Scholar logo with link to Google Scholar
Taguchi, N. (2008). The role of the learning environment in the development of pragmatic comprehension: A comparison of gains between EFL and ESL learners. Studies in Second Language Acquisition, 30(4), 423–452. Google Scholar logo with link to Google Scholar
Thistlethwaite, D. L., & Campbell, D. T. (1960). Regression-discontinuity analysis: An alternative to the ex post facto experiment. Journal of Educational Psychology, 51(6), 309–317. Google Scholar logo with link to Google Scholar
Tullock, B., & Ortega, L. (2017). Fluency and multilingualism in study abroad: Lessons from a scoping review. System, 711, 7–21. Google Scholar logo with link to Google Scholar
Urquiola, M., & Verhoogen, E. (2009). Class-size caps, sorting, and the regression discontinuity design. American Economic Review, 99(1), 179–215.
Waibel, S., Rüger, H., Ette, A., & Sauer, L. (2017). Career consequences of transnational educational mobility: A systematic literature review. Educational Research Review, 201, 81–98. Google Scholar logo with link to Google Scholar
Xiao, F. (2015). Adult second language learners’ pragmatic development in the study-abroad context: A review. Frontiers: The Interdisciplinary Journal of Study Abroad, 25(1), 132–149. Google Scholar logo with link to Google Scholar
Xu, Y. (2019). Changes in interlanguage complexity during study abroad: A meta-analysis. System, 801, 199–211. Google Scholar logo with link to Google Scholar
Yashima, T. (2002). Willingness to communicate in a second language: The Japanese EFL context. The Modern Language Journal, 86(1), 54–66. Google Scholar logo with link to Google Scholar
(2009). International posture and the ideal L2 self in the Japanese EFL context. In Z. Dörnyei, & E. Ushioda (Eds.), Motivation, language identity, and the L2 self (pp. 144–163). Multilingual Matters. Google Scholar logo with link to Google Scholar
Yashima, T., & Zenuk-Nishide, L. (2008). The impact of learning contexts on proficiency, attitudes, and L2 communication: Creating an imagined international community. System, 36(4), 566–585. Google Scholar logo with link to Google Scholar
Cited by (2)

Cited by two other publications

Fernández, Julieta & Rachel Shively
2025. Methodologies in Spanish-language study abroad research. Journal of Spanish Language Teaching 12:2 ► pp. 123 ff. DOI logo
Lindstromberg, Seth
2025. Eight reasons not to test for baseline group equivalence in a parallel groups pretest-posttest study. Research Methods in Applied Linguistics 4:3 ► pp. 100254 ff. DOI logo

This list is based on CrossRef data as of 1 december 2025. Please note that it may not be complete. Sources presented here have been supplied by the respective publishers. Any errors therein should be reported to them.

Mobile Menu Logo with link to supplementary files background Layer 1 prag Twitter_Logo_Blue