Conceptual grounding in simulation studies of language acquisition

In order to understand the evolutionary pathway to the capability for language, we must first clearly understand the functional capabilities that the child brings to the task of language acquisition. Behavioral studies provide insight into infants’ ability to extract statistical and distributional structure directly from the auditory signal, and their capabilities to construct relations between this structure and the structure extracted from perceptual systems. At the interface of these two processes lies a conceptual scene representation that can be accessed by both, and that importantly provides a means for the two systems to constructively interact.

Recent studies have begun to make progress in simulating infants’ capabilities to extract statistical structure (e.g. word segmentation and lexical categorization) directly from the speech sound sequence. The current research examines how this structure interacts with perceptual structure at the level of the conceptualized scene. In particular we demonstrate how the grounding of words and sentences in conceptualized visual scenes permits the system to construct the appropriate relations between words and their referents, and sentences and theirs (structured conceptualizations of scenes representing agents, objects and actions) in the initial phases of acquisition of syntactic structure. These studies simulate behavioral observations of the trajectory of infants’ linguistic acquisition of concrete nouns, followed by concrete verbs and then more abstract nouns and verbs, in parallel with the development of first simple and then more complex syntactic structures. The relevance of these results to infant language acquisition behavior will be discussed. While this research yields interesting new results in characterizing the grounding of language in conceptualized scenes, it also identifies serious limitations of the current methods that will be discussed, along with the associated future extensions.