Reasons for over-fitting and how to fix it

Usually, over-fitting is caused by the ability to learn complex hypothesis, and aggregated by the noise in the training data.

To fix over-fitting, we can: (1) add more training data, (2) simply the training model, (3) prune the data, (4) use gold annotation to test the model got, stop training when error goes up.