Abstract:

Researchers that assume discrete, symbolic models of syntax, tend to reject gradualist accounts in favor of a 'saltationist' view, where the crucial machinery for learning and processing syntactic structure emerges in a single step (as a side-effect of known or unknown brain principles or as a macro-mutation). A key argument for the last position is that the vast amount of work on describing the syntax of different natural languages makes use of symbolic formalisms like minimalist grammar, HPSG, categorical grammar etc., and that it is difficult to imagine process of any sort that moves gradually from a state without to a state with their key components. The most often repeated version of this argument concerns 'recursion' or 'merge', which is said to be an all or nothing phenomenon where you can't have just a little bit of it.

In my talk I will review this argument in the light of recent developments in computational linguistics where a class of models has become popular that we can call 'vector grammars', that include the Long-Short Term Memory networks, Gated Recurrent Networks and Recursive Neural Networks (e.g., Socher et al., 2010). These models have over the last two years swept to prominence in many areas of NLP and shown state of the art results on many benchmark tests including constituency parsing, dependency parsing, sentiment analysis and question classification.

Download Paper (pdf)

Citation:

Zuidema W. (2016). Deep Learning Models Of Language Processing And The Evolution Of Syntax. In S.G. Roberts, C. Cuskley, L. McCrohon, L. Barceló-Coblijn, O. Fehér & T. Verhoef (eds.) The Evolution of Language: Proceedings of the 11th International Conference (EVOLANG11). Available online: http://evolang.org/neworleans/papers/151.html