Evaluation
Important Notions
- Empirical evaluation of accuracy:
- Independent test data
- Supervised or unsupervised
- Gold standard evaluation
- Descriptive statistics/measures of accuracy:
- Accuracy rate (percent correct): 1/n ∑ix_i
- Recall: true positives / (true positives + false negatives)
- Precision: true positives / (true positives + false positives)
- Logprob: 1/n - ∑i log P(xi)
- Statistical inference:
- Confidence intervals
- Hypothesis testing (significance)
Slides for lecture 10
Suggested Reading
- Manning, C. D. & Schütze, H. (1999) Foundations of
Statistical Natural Language Processing. MIT Press.
Sections 6.2.3, 10.6.1., 12.1.8, 15.1.2.