Short description: Learning bimodal distributions of auditory, visual, and tactile signals: is category learning specialized for speech?

Abstract:

The evolution of linguistic cognition is a notoriously difficult problem: as all learning mechanisms are intertwined with human development and behavior, it is hard to tease apart which aspects of language are the result of cultural processes and which are evolved cognitive traits (De Boer, 2015). We explore whether humans have specializations related to language, and specifically, whether statistical learning of categories is specialized for language-like signals. While it is well-known that statistical learning is not restricted to language learning, the way the mechanism operates in different domains may not be the same, and may be affected by perceptual and cognitive constraints (Conway & Christiansen, 2005).

We present an experiment in which participants learn, categorize, and reproduce signals in the acoustic, visual, and tactile modalities. Participants are trained on bimodal distributions of tones, images and buzzes with some variation in duration, resulting in a `long' and `short' category (Maye et al., 2002). After training, participants rate individual signals on a 6 point scale from `definitely short' to `definitely long'. The production task consists of creating 3 signals for both categories by pressing the mouse button. The signal is presented as long as the button is pressed. If there is indeed a specialization for learning language-like signals, then we expect that participants reveal: a) more certainty in the categorization task in the auditory and visual conditions, and b) better estimation of the peaks in the distributions, resulting in more reliable reproductions of the categories in the auditory and visual conditions. Alternatively, in normal hearing adults there may be a linguistic training effect, for instance from distinctions between long and short vowels in their native language, in which case better performance in the auditory condition compared to the other conditions is expected, following earlier perception studies (Jones et al., 2009).

A within subjects design provided us with categorization and production data from 29 participants in all modalities. Results from a logistic regression reveal three interesting trends. First, statistical learning of categories and (reliably) reproducing them is possible in all domains, including the tactile modality. Second, and most importantly, the categorization and production processes are remarkably similar in the tactile and auditory domains (odds ratios 0.99 and 1.10; 95\% confidence intervals 0.84-1.15 and 0.94-1.28, respectively), but not in the visual domain (OR = 1.30; CIs = 1.12-1.53), suggesting that there is no cognitive specialization for learning language-like signals, nor that there is an effect from previous language experience. Finally, comparing the categorization and production data reveals an interesting tension: on the one hand, we demonstrate that across all three modalities participants were able to induce representations of distinct categories that respect the statistics of the training distributions. On the other hand, we consistently find more variation in duration among participants' productions than was present in the training distributions. This pattern of variation differs between modalities, with more variation in the visual modality. We fitted a Bayesian model of inference for Gaussian distributions to the data in order to investigate whether this variation reflects a meaningful component of the learning process, and to provide a quantitative characterization of the pattern of differences in how participants formed categories in this experiment. The model correctly predicts the overall pattern of categorization behaviour that we find in the empirical data.

Our study further explores the extent to which statistical learning is domain general, as it is becoming increasingly clear that we cannot approach it as a unitary mechanism (e.g. Frost et al., 2015). Teasing apart differences between modalities will help us find out how the perceptional biases of each modality potentially affect these learning mechanisms, which is necessary for understanding whether certain forms of learning are specialized for language.

Download Paper (pdf)

Citation:

van der Ham S., Thompson B. and de Boer B. (2016). Catergory Learning In Audition, Touch, And Vision. In S.G. Roberts, C. Cuskley, L. McCrohon, L. Barceló-Coblijn, O. Fehér & T. Verhoef (eds.) The Evolution of Language: Proceedings of the 11th International Conference (EVOLANG11). Available online: http://evolang.org/neworleans/papers/38.html