Transfer Learning for Low-Resource Part-of-Speech Tagging

Authors

Jeffrey Zhou, Yale University
Neha Verma, Yale University

Abstract

Neural network approaches to Part-of-Speech tagging, like other supervised neural network tasks, benefit from larger quantities of labeled data. However, in the case of low-resource languages, additional methods are necessary to improve the performances of POS taggers. In this paper, we explore transfer learning approaches to improve POS tagging in Afrikaans using a neural network. We investigate the effect of transferring network weights that were originally trained for POS tagging in Dutch. We also test the use of pretrained word embeddings in our POS tagger, both independently and in conjunction with the transferred weights from a Dutch POS tagger. We find a marginal increase in performance due to transfer learning with the Dutch POS tagger, and a significant increase due to the use of either unaligned or aligned pretrained embeddings. Notably, there is little difference in performance when using either unaligned or aligned embeddings, even when utilizing cross-lingual transfer learning.

Recommended Citation

Zhou, Jeffrey and Verma, Neha (2020) "Transfer Learning for Low-Resource Part-of-Speech Tagging," The Yale Undergraduate Research Journal: Vol. 1: Iss. 1, Article 19.
Available at: https://elischolar.library.yale.edu/yurj/vol1/iss1/19