Feature Optimization for Constituent Parsing via Neural Networks

Zhiguo Wang, Haitao Mi, Nianwen Xue


Abstract

The performance of discriminative constituent parsing relies crucially on feature engineering, and effective features usually have to be carefully selected through a painful manual process. In this paper, we propose a method that automatically learns a set of optimal features. Specifically, we build a feedforward neural network model, which takes as input a few primitive units (words, POS tags and certain contextual tokens) from the local context, induces the feature representation in the hidden layer and makes parsing predictions in the output layer. The network simultaneously learns the feature representation and the prediction model parameters using a back propagation algorithm. By pre-training the model on a large amount of automatically parsed data, our model achieves impressive improvements. Evaluated on the standard data sets, our final performance reaches 86.6% (F_1) for Chinese, which outperforms all state-of-the-art parsers, and 90.7% (F_1) for English, which surpasses all the previous neural network based parsers. On cross-domain data sets, our parser outperforms the Berkeley Parser by 3.4 percentage points on average for Chinese and 2.4 percentage points for English.