Stochastic optimization of a probabilistic language model

Abstract

The system described generates a language model for use in the APRIL natural-language parser. The APRIL language model takes the form of a set of typical productions (pairings of mother label with sequence of daughter labels) for each non-terminal category; the system analyses an input by seeking a labelled tree over the words of the input which offers the best fit between each production in the tree and some production of the language model. Previous versions of APRIL used hand-designed language models, but this became a serious research bottleneck. This paper presents a system which uses stochastic optimization to reduce a large set of observed productions to a set of “prototype productions” which are few in number but which nevertheless successfully typify the large observed set.