Short description: Word order variability influences interpretation of verbs in silent gesture. Bayesian Inference explains how.

Abstract:

Silent gesture, a lab methodology in which adult hearing participants describe simple events using only their hands, has shown to be valuable for investigating the origins of word order (the ordering of Subject, Object and Verb) in language. In particular, recent experiments using this methodlogy have uncovered word order biases in silent gesture production: people prefer SOV order when they describe extensional transitive events (boy-pail-swing; Goldin-Meadow et al., 2008), but prefer SVO order for events with different semantic properties, such as intensional events (man-think of-ball, witch-build-house; Schouwstra & de Swart, 2014). However, a core underexplored question is whether these biases also feature in silent gesture intepretation. In our presentation we describe experimental and computational analyses of silent gesture interpretation: first, we examine the influence of word order variation on silent gesture interpretation; second, we develop a Bayesian computational account of the results that allows us to directly compare production and interpretation behaviour.

In our experiment, we recorded silent gestures of ambiguous actions (e.g., a gesture that could mean build as well as climb). For each ambiguous action, we composed ambiguous gesture strings describing transitive events: one in SVO and one in SOV order (e.g., witch-climb/build-house and witch-house-climb/build). We predicted that participants would be more likely to interpret SVO ordered videos as intensional events than as extensional events, and vice versa. Forty one Dutch and forty Turkish participants watched twelve videos (6 in SVO and 6 in SOV order; we used two different versions to make sure that all participants saw each video in only one order), and were asked, after each video, tochoose an interpretation from two line drawings of the two target events (one intensional and one extensional). We found that, independently of native language, for SOV sequences, an extensional interpretation was given significantly more often (M=.711, SE=.019) than for SVO sequences (M=.569, SE=.020).

When people interpret silent gesture, they use word order as a key to the semantic distinction between intensional and extensional events: their preferences are semantically conditioned, like in silent gesture production. However, a key aspect of our findings is that this effect is much weaker in interpretation than in production. We propose a computational account for this finding, based on the idea that silent gesture interpretation can be understood as inductive inference under uncertainty. A Bayesian model allows us to specify the influence of word order biases – or prior beliefs – on judgements about the unseen intentions of another gesturer – or posterior beliefs which are updated after observing data. We specify two versions of the model: one in which participants are assumed to rely solely on their own biases during interpretation, as in production (model M1); and another which assumes participants account rationally for the uncertainty that surrounds the speaker’s usage of word order – consistent with the idea that the learner entertains emerging linguistic rules in this context (model M2). Our analysis is in two parts: first, we estimate an appropriate prior from production experiments and contrast the predictions of the two interpretation models (and their fit to the empirical data) under this prior (see figure 1); second, we fit both interpretation models to the entire dataset of productions and interpretations. Both analyses support model M2 over M1, suggesting that silent gesture interpretation is underpinned by domain-independent computational principles that balance word order biases with communicative uncertainty.

Download Paper (pdf)

Citation:

Thompson B., Schouwstra M. and de Swart H. (2016). Interpreting Silent Gesture. In S.G. Roberts, C. Cuskley, L. McCrohon, L. Barceló-Coblijn, O. Fehér & T. Verhoef (eds.) The Evolution of Language: Proceedings of the 11th International Conference (EVOLANG11). Available online: http://evolang.org/neworleans/papers/94.html