Semi-Supervised Learning and Domain Adaptation in Natural Language Processing
This book introduces basic supervised learning algorithms applicable to natural language processing (NLP) and shows how the performance of these algorithms can often be improved by exploiting the marginal distribution of large amounts of unlabeled data. One reason for that is data sparsity, i.e., the limited amounts of data we have available in NLP. However, in most real-world NLP applications our labeled data is also heavily biased. This book introduces extensions of supervised learning algorithms to cope with data sparsity and different kinds of sampling bias. This book is intended to be both readable by first-year students and interesting to the expert audience. My intention was to introduce what is necessary to appreciate the major challenges we face in contemporary NLP related to data sparsity and sampling bias, without wasting too much time on details about supervised learning algorithms or particular NLP applications. I use text classification, part-of-speech tagging, and dependency parsing as running examples, and limit myself to a small set of cardinal learning algorithms. I have worried less about theoretical guarantees ("this algorithm never does too badly") than about useful rules of thumb ("in this case this algorithm may perform really well"). In NLP, data is so noisy, biased, and non-stationary that few theoretical guarantees can be established and we are typically left with our gut feelings and a catalogue of crazy ideas. I hope this book will provide its readers with both. Throughout the book we include snippets of Python code and empirical evaluations, when relevant.
What people are saying - Write a review
We haven't found any reviews in the usual places.
Other editions - View all
20 Newsgroups 25 randomly accuracy Adversarial learning applications assume averaged perceptron bag-of-words base learner Bayesian network binary Chapter clustering algorithms co-training compute condensed nearest neighbor Consider correlated decision boundaries dependency tree distribution bias document classification domain adaptation EMNLP estimate example Gaussian hierarchical clustering Ifwe implementation importance weighting introduce k-nearest KL divergence labeled data points learning under bias linear classifier linguistic machine learning macro-average meta-analysis Naive Bayes classifier Natural Language Processing nearest neighbor classifier nearest neighbor editing nearest neighbor search non-spam numpy ofthe outlier output variables parameters parsers perceptron performance POS tagging probability problem instances Python code random effects model range(X.shape[0 sampling bias Section semi-supervised learning algorithms SkLearn source data spam emails spam filtering structure prediction structured perceptron subset supervised learning algorithms support vector machines syntactic target data target distribution target domain techniques training data Treebank tri-training typically update variance weight functions words