no data, no problem

Transfer Learning, Weak Supervision & Noisy Labels for Low-Resource NLP

Overview

  • Modern machine learning approaches often require large amounts of labeled training data. We study how one can train such models in low-resource scenarios.
  • This includes transfer learning and distant supervision for African low-resource languages.
  • Distant and weak supervision allow to leverage insights from experts efficently and label large amounts of unlabeled data automatically. However, this labeling tends to contain errors. We propose methods to model the label noise and leverage these labels more effectively.

WeaSuL Workshop

Weak and distant supervision is a popular topic in machine learning, computer vision and NLP both from a theoretic and applied/industry perspective. To bring together researchers from these different perspectives and to help new people into the field, we organize the WeaSuL workshop at ICLR’21.

Workshop website

Visual Guide

As companion to our survey, we published a more applied and visual guide for low-resource NLP. It is available on towards data science

Publications

  1. Dawei Zhu, Xiaoyu Shen, Michael A. Hedderich, and Dietrich Klakow
    In Proceedings of the 17th Conference of the European Chapter of the Association for Computational Linguistics (EACL), 2023
  2. Dawei Zhu, Michael A. Hedderich, Fangzhou Zhai, David Ifeoluwa Adelani, and Dietrich Klakow
    In Proceedings of the ICLR 2022 Workshop AfricaNLP, 2022
  3. Dawei Zhu, Michael A. Hedderich, Fangzhou Zhai, David Adelani, and Dietrich Klakow
    In Proceedings of the Third Workshop on Insights from Negative Results in NLP, 2022
  4. Michael A. Hedderich, Lukas Lange, Heike Adel, Jannik Strötgen, and Dietrich Klakow
    In Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT), 2021
  5. Michael A. Hedderich, Dawei Zhu, and Dietrich Klakow
    In Thirty-Fifth AAAI Conference on Artificial Intelligence, 2021
  6. Michael A. Hedderich, Lukas Lange, and Dietrich Klakow
    In ICML 2021 Workshop on Practical Machine Learning For Developing Countries, 2021
  7. Michael A. Hedderich, David Adelani, Dawei Zhu, Jesujoba Alabi, Udia Markus, and 1 more author
    In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), 2020
  8. David Ifeoluwa Adelani, Michael A. Hedderich, Dawei Zhu, Esther Berg, and Dietrich Klakow
    2020
  9. Debjit Paul, Mittul Singh, Michael A. Hedderich, and Dietrich Klakow
    In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Student Research Workshop, Jun 2019
  10. Lukas Lange, Michael A. Hedderich, and Dietrich Klakow
    In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), Jun 2019
  11. Michael A. Hedderich, and Dietrich Klakow
    In Proceedings of the Workshop on Deep Learning Approaches for Low-Resource NLP, Jun 2018