Cover not available

In:Implicit and Explicit Learning of Languages
Edited by Patrick Rebuschat
[Studies in Bilingualism 48] 2015
► pp. 213–246

Get fulltext from our e-platform
JBE logo with link
PDF logo with download linkDownload Book PDF
authorization required

Implicit learning of non-adjacent dependencies

A graded, associative account

 | Nanyang Technological University
 | Université Libre de Bruxelles
 | Cornell University
 | University of Warwick
 | Université Libre de Bruxelles
Published online: 24 September 2015
DOI logo with link
https://doi.org/10.1075/sibil.48.10onn
Language and other higher-cognitive functions require structured sequential behavior including non-adjacent relations. A fundamental question in cognitive science is what computational machinery can support both the learning and representation of such non-adjacencies, and what properties of the input facilitate such processes. Learning experiments using miniature languages with adult and infants have demonstrated the impact of high variability (Gómez, 2003) as well as nil variability (Onnis, Christiansen, Chater, & Gómez (2003; submitted) of intermediate elements on the learning of nonadjacent dependencies. Intriguingly, current associative measures cannot explain this U shape curve. In this chapter, extensive computer simulations using five different connectionist architectures reveal that Simple Recurrent Networks (SRN) best capture the behavioral data, by superimposing local and distant information over their internal ‘mental’ states. These results provide the first mechanistic account of implicit associative learning of non-adjacent dependencies modulated by distributional properties of the input. We conclude that implicit statistical learning might be more powerful than previously anticipated.
References (60)
References
Allen, J., & Seidenberg, M.S. (1999). The emergence of grammaticality in connectionist networks. In B. MacWhinney (Ed.), The emergence of language (pp. 115–151). Mahwah, NJ: Lawrence Erlbaum Associates.Google Scholar logo with link to Google Scholar
Botvinick, M., & Plaut, D.C. (2006). Short-term memory for serial order: A recurrent neural network model. Psychological Review, 113, 201–233. Google Scholar logo with link to Google Scholar
. (2004). Doing without schema hierarchies: A recurrent connectionist approach to normal and impaired routine sequential action. Psychological Review, 111, 395–429. Google Scholar logo with link to Google Scholar
Brakel, P., & Frank, S.L. (2009). Strong systematicity in sentence processing by simple recurrent networks. In N.A. Taatgen, H. van Rijn, J. Nerbonne & L. Schomaker (Eds.), Proceedings of the 31st Annual Conference of the Cognitive Science Society (pp. 1599–1604). Austin, TX: Cognitive Science Society.Google Scholar logo with link to Google Scholar
Chater, N., & Conkey, P. (1992). Finding linguistic structure with recurrent neural networks. In Proceedings of the 14th Annual Conference of the Cognitive Science Society (pp. 402–407). Hillsdale, New Jersey: Psychology Press.
Chomsky, N. (1959). A review of BF Skinner's Verbal Behavior. Language, 35(1), 26–58. Google Scholar logo with link to Google Scholar
Christiansen, M.H., Allen, J., & Seidenberg, M.S. (1998). Learning to segment speech using multiple cues: A connectionist model. Language and Cognitive Processes, 13, 221–268. Google Scholar logo with link to Google Scholar
Christiansen, M.H., & Chater, N. (1999). Toward a connectionist model of recursion in human linguistic performance. Cognitive Science, 23, 157–205. Google Scholar logo with link to Google Scholar
Christiansen, M.H., Conway, C.M., & Curtin, S. (2000). A connectionist single-mechanism account of rule-like behavior in infancy. In L.R. Gleitman & A.K. Joshi (Eds.), The Proceedings of the 22nd Annual Conference of the Cognitive Science Society (pp. 83–88). Philadelphia, PA: University of Pennsylvania.Google Scholar logo with link to Google Scholar
Christiansen, M.H., & MacDonald, M.C. (2009). A usage-based approach to recursion in sentence processing. Language Learning, 59(Suppl. 1), 126–161. Google Scholar logo with link to Google Scholar
Cleeremans, A., Servan-Schreiber, D., & McClelland, J.L. (1989). Finite state automata and simple recurrent networks. Neural Computation, 1, 372–381. Google Scholar logo with link to Google Scholar
Destrebecqz, A., & Cleeremans, A. (2003). Temporal factors in sequence learning. In Luis Jiménez (Ed.), Attention and implicit learning. Amsterdam: John Benjamins. Google Scholar logo with link to Google Scholar
Cottrell, G.W., & Plunkett, K. (1995). Acquiring the mapping from meanings to sounds. Connection Science, 6, 379–412. Google Scholar logo with link to Google Scholar
Dell, G.S., Juliano, C., & Govindjee, A. (1993). Structure and content in language production: A theory of frame constraints in phonological speech errors. Cognitive Science, 17, 149–195. Google Scholar logo with link to Google Scholar
Dienes, Z. (1992). Connectionist and memory-array models of artificial grammar learning. Cognitive Science, 23, 53–82. Google Scholar logo with link to Google Scholar
Dulany, D.E., Carlson, R.A., & Dewey, G.I. (1984). A case of syntactical learning and judgement: How conscious and how abstract? Journal of Experimental Psychology: General, 113, 541–555. Google Scholar logo with link to Google Scholar
Elman, J.L. (1990). Finding structure in time. Cognitive science, 14(2), 179–211. Google Scholar logo with link to Google Scholar
. (1991). Distributed representations, simple recurrent networks, and grammatical structure. Machine Learning, 7, 195–224. Google Scholar logo with link to Google Scholar
Estes, K., Evans, J., Alibali, M., & Saffran, J. (2007). Can infants map meaning to newly segmented words? Psychological Science, 18(3), 254. Google Scholar logo with link to Google Scholar
Farkaš, I., & Crocker, M.W. (2008). Syntactic systematicity in sentence processing with a recurrent self-organizing network. Neurocomputing, 71(7), 1172–1179. Google Scholar logo with link to Google Scholar
Frank, M.C., Goldwater, S., Griffiths, T.L., & Tenenbaum, J.B. (2010). Modeling human performance in statistical word segmentation. Cognition, 117(2), 107–125. Google Scholar logo with link to Google Scholar
Frank, S.L. (in press). Getting real about systematicity. In P. Calvo & J. Symons (Eds.), Systematicity and cognitive architecture: Conceptual and empirical issues 25 years after Fodor & Pylyshyn's challenge to connectionism. Cambridge, MA: The MIT Press.
Frinken, V., Fischer, A., Manmatha, R., & Bunke, H. (2012). A novel word spotting method based on recurrent neural networks. IEEE Transactions on, Pattern Analysis and Machine Intelligence, 34(2), 211–224. Google Scholar logo with link to Google Scholar
Gaskell, M.G., Hare, M., & Marslen-Wilson, W.D. (1995). A connectionist model of phonological representation in speech perception. Cognitive Science, 19, 407–439. Google Scholar logo with link to Google Scholar
Gibson, F.P., Fichman, M., & Plaut, D.C. (1997). Learning in dynamic decision tasks: Computational model and empirical evidence. Organizational Behavior and Human Decision Processes, 71, 1–35. Google Scholar logo with link to Google Scholar
Gómez, R. (2002). Variability and detection of invariant structure. Psychological Science, 13, 431–436. Google Scholar logo with link to Google Scholar
Harm, M.W., & Seidenberg, M.S. (1999). Phonology, reading acquisition, and dyslexia: Insights from connectionist models. Psychological Review, 106, 491–528. Google Scholar logo with link to Google Scholar
Hinoshita, W., Arie, H., Tani, J., Okuno, H.G., & Ogata, T. (2011). Emergence of hierarchical structure mirroring linguistic composition in a recurrent neural network. Neural Networks, 24(4), 311–320. Google Scholar logo with link to Google Scholar
Johnstone, T. & Shanks, D.R. (2001). Abstractionist and processing accounts of implicit learning. Cognitive Psychology, 42, 61–112. Google Scholar logo with link to Google Scholar
Jordan, M.I. (1986). Attractor dynamics and parallelism in a connectionist sequential machine. In Proceedings of the Eighth Annual Conference of the Cognitive Science Society . Hillsdale, NJ: Lawrence Erlbaum Associates.
Kinder, A. & Shanks, D.R. (2001). Amnesia and the declarative/procedural distinction: A recurrent network model of classification, recognition, and repetition priming. Journal of Cognitive Neuroscience, 13, 648–669. Google Scholar logo with link to Google Scholar
Kirov, C., & Frank, R. (2012). Processing of nested and cross-serial dependencies: An automaton perspective on SRN behaviour. Connection Science, 24(1), 1–24. Google Scholar logo with link to Google Scholar
Lashley, K.S. (1951). The problem of serial order in behavior. In L.A. Jeffress (Ed.), Cerebral mechanisms in behavior (pp. 112–146). New York, NY: Wiley.Google Scholar logo with link to Google Scholar
Luce, R.D. (1963). Detection and recognition. In R.D. Luce, R.R. bush, & E. Galanter (Eds.), Handbook of mathematical psychology. New York, NY: Wiley.Google Scholar logo with link to Google Scholar
Maraqa, M., Al-Zboun, F., Dhyabat, M., & Zitar, R.A. (2012). Recognition of Arabic Sign Language (ArSL) using recurrent neural networks. Journal of Intelligent Learning Systems and Applications, 4(1), 41–52. Google Scholar logo with link to Google Scholar
Maskara, A., & Noetzel, A. (1992). Forced simple recurrent neural network and grammatical inference. In Proceedings of the Fourteenth Annual Conference of the Cognitive Science Society (pp. 420–425). Hillsdale, NJ: Lawrence Erlbaum Associates.
Miikkulainen, R., & Mayberry III, M.R. (1999). Disambiguation and grammar as emergent soft constraints. In B. MacWhinney (Ed.), Emergence of language, 153–176. Mahwah, NJ: Lawrence Erlbaum Associates.Google Scholar logo with link to Google Scholar
Misyak, J.B., & Christiansen, M.H. (2012). Statistical learning and language: An individual differences study. Language Learning, 62, 302–331. Google Scholar logo with link to Google Scholar
Misyak, J.B., Christiansen, M.H. & Tomblin, J.B. (2010a). On-line individual differences in statistical learning predict language processing. Frontiers in Psychology, Sept.14. Google Scholar logo with link to Google Scholar
. (2010b). Sequential expectations: The role of prediction- based learning in language. Topics in Cognitive Science, 2, 138–153. Google Scholar logo with link to Google Scholar
Moss, H.E., Hare, M.L., Day, P., & Tyler, L.K. (1994). A distributed memory model of the associative boost in semantic priming. Connection Science, 6, 413–427. Google Scholar logo with link to Google Scholar
Munakata, Y., McClelland, J.L., & Siegler, R.S. (1997). Rethinking infant knowledge: Toward an adaptive process account of successes and failures in object permanence tasks. Psychological Review, 104, 686–713. Google Scholar logo with link to Google Scholar
Onnis, L., Christiansen, M.H., Chater, N., & Gómez, R. (submitted). Statistical learning of non-adjacent relations. Submitted manuscript.
. (2003). Reduction of uncertainty in human sequential learning: Preliminary evidence from Artificial Grammar Learning. In R. Alterman & D. Kirsh (Eds.), Proceedings of the 25th Annual Conference of the Cognitive Science Society. Boston, MA: Cognitive Science Society.Google Scholar logo with link to Google Scholar
Onnis, L., Monaghan, P., Christiansen, M.H., & Chater, N. (2004). Variability is the spice of learning, and a crucial ingredient for detecting and generalizing in nonadjacent dependencies. In Proceedings of the 26th annual conference of the Cognitive Science Society (pp. 1047–1052). Mahwah, NJ: Lawrence Erlbaum.
Pacton, S., Perruchet, P., Fayol, M., & Cleeremans, A. (2001). Implicit learning out of the lab: The case of orthographic regularities. Journal of Experimental Psychology: General, 130, 401–426. Google Scholar logo with link to Google Scholar
Perruchet, P., & Pacteau, C. (1990). Synthetic grammar learning: Implicit rule abstraction or explicit fragmentary knowledge? Journal of Experimental Psychology: General, 119, 264–275. Google Scholar logo with link to Google Scholar
Perruchet, P., & Pacton, S. (2006). Implicit learning and statistical learning: One phenomenon, two approaches. Trends In Cognitive Sciences, 10(5), 233–238. Google Scholar logo with link to Google Scholar
Plaut, D.C., & Kello, C.T. (1999). The emergence of phonology from the interplay of speech comprehension and production: A distributed connectionist approach. In B. MacWhinney (Ed.), The emergence of language (pp. 381–415). Mahwah, NJ: Lawrence Erlbaum Associates.Google Scholar logo with link to Google Scholar
Redington, M., & Chater, N. (2002). Knowledge representation and transfer in artificial grammar learning (AGL). In R.M. French & A. Cleeremans (Eds.), Implicit learning and consciousness: An empirical, philosophical, and computational consensus in the making. Hove: Psychology Press.Google Scholar logo with link to Google Scholar
Rohde, D.L.T., & Plaut, D.C. (1999). Language acquisition in the absence of explicit negative evidence: How important is starting small? Cognition, 72, 67–109. Google Scholar logo with link to Google Scholar
Saffran, J.R., Aslin, R.N., & Newport, E.L. (1996). Statistical learning by 8-month-old infants. Science, 274, 1926–1928. Google Scholar logo with link to Google Scholar
Saffran, J. (2001). Words in a sea of sounds: The output of infant statistical learning. Cognition, 81, 149–169. Google Scholar logo with link to Google Scholar
Servan-Schreiber, D., Cleeremans, A. & McClelland, J.L. (1991). Graded state machines: The representation of temporal dependencies in simple recurrent networks. Machine Learning, 7, 161–193. Google Scholar logo with link to Google Scholar
Si, Y., Xu, J., Zhang, Z., Pan, J., & Yan, Y. (2012). An improved Mandarin voice input system using recurrent neural network language model. In Computational Intelligence and Security (CIS), Eighth International Conference on IEEE (pp. 242–246).
Socher, R., Manning, C.D., & Ng, A.Y. (2010). Learning continuous phrase representations and syntactic parsing with recursive neural networks. In Proceedings of the NIPS-2010 Deep Learning and Unsupervised Feature Learning Workshop . Hilton: Cheakmus.
Sutskever, I., Martens, J., & Hinton, G. (2011). Generating text with recurrent neural networks. In Proceedings of the 2011 International Conference on Machine Learning (ICML-2011).
Tabor, W. (2011). Recursion and recursion-like structure in ensembles of neural elements. In H. Sayama, A. Minai, D. Braha, & Y. Bar-Yam (Eds.), Unifying themes in complex systems. Proceedings of the VIII International Conference on Complex Systems (pp. 1494–1508). Berlin: Springer.
Takac, M., Benuskova, L, & Knott, A. (2012). Mapping sensorimotor sequences to word sequences: A connectionist model of language acquisition and sentence generation. Cognition, 125, 288–308. Google Scholar logo with link to Google Scholar
Vokey, J.R., & Brooks, L.R. (1992). Salience of item knowledge in learning artificial grammar. Journal of Experimental Psychology: Learning, Memory, and Cognition, 20, 328–344. Google Scholar logo with link to Google Scholar
Cited by (3)

Cited by three other publications

Deocampo, Joanne A., Tricia Z. King & Christopher M. Conway
2019. Concurrent Learning of Adjacent and Nonadjacent Dependencies in Visuo-Spatial and Visuo-Verbal Sequences. Frontiers in Psychology 10 DOI logo
Katan, Pesia, Shani Kahta, Ayelet Sasson & Rachel Schiff
2017. Performance of children with developmental dyslexia on high and low topological entropy artificial grammar learning task. Annals of Dyslexia 67:2 ► pp. 163 ff. DOI logo
Andringa, Sible & Patrick Rebuschat
2015. NEW DIRECTIONS IN THE STUDY OF IMPLICIT AND EXPLICIT LEARNING. Studies in Second Language Acquisition 37:2 ► pp. 185 ff. DOI logo

This list is based on CrossRef data as of 1 december 2025. Please note that it may not be complete. Sources presented here have been supplied by the respective publishers. Any errors therein should be reported to them.

Mobile Menu Logo with link to supplementary files background Layer 1 prag Twitter_Logo_Blue