Training a Neural Network in a Low-Resource Setting on Automatically Annotated Noisy Data
ACL-Workshop Deep Learning Approaches for Low Resource Natural Language Processing (DeepLo)
July 2018, oral presentation
Manually labeled corpora are expensive to create and often not available for low-resource languages or domains. Automatic labeling approaches are an alternative way to obtain labeled data in a quicker and cheaper way. However, these labels often contain more errors which can deteriorate a classifier's performance when trained on this data. We propose a noise layer that is added to a neural network architecture. This allows modeling the noise and train on a combination of clean and noisy data. We show that in a low-resource NER task we can improve performance by up to 35% by using additional, noisy data and handling the noise.
author = "Hedderich, Michael A. and Klakow, Dietrich",
title = "Training a Neural Network in a Low-Resource Setting on Automatically Annotated Noisy Data",
booktitle = "Proceedings of the Workshop on Deep Learning Approaches for Low-Resource NLP",
year = "2018",
publisher = "Association for Computational Linguistics",
pages = "12--18",
location = "Melbourne",
url = "http://aclweb.org/anthology/W18-3402"
Tensorflow Neural Language Model Toolkit
While not being the original author, I'm the current mantainer of TF-NNLM-TK, a Tensorflow neural language
model toolkit, at the Spoken Language Systems group. Being used both by business and research groups it
functions as a stand-alone language model and as a language model component for more
For more information, please visit the Github page. If you have any questions, you can submit an
issue there or contact me directly.
Local Multiplayer Game
"Grab some friends, jump on the couch and get ready for this fast-paced and colorful arena game. In the world of these cats, there can only be one winner!"
Semantic Embeddings for Paraphrase Detection
and Reverse Dictionaries Using Memory Networks
and External Knowledge from WordNet
This thesis examines two tasks in the context of creating sentence embeddings with
neural networks. We first look at reverse dictionaries which retrieve a target word from
a large vocabulary based on the word’s description or definition. In the second task, we
detect paraphrases, i.e. sentences that have the same meaning. In both cases, our specific
network architectures create vector representations of the sentences using established
techniques like LSTMs. We enhance these representations by adding a memory network
component that allows integrating information from an external knowledge source. This
information can be directly selected by the neural network. In our case, we incorporate
additional linguistic knowledge from WordNet (like synonyms or hypernyms) about the
words the network processes. Extensive experiments are performed and evaluated for
both tasks. As part of the thesis, we give an introduction to basic and more advanced
techniques from the areas of word representations and neural networks as well as a review
of the related literature. We also reproduce existing work on reverse dictionaries and we
create large paraphrase corpora based on existing video description data. Our analysis
of word-sense ambiguity in the datasets can be used as a basis for future work.
If you are interested in the PDF of the thesis, please contact me.
"Welcome to your new taxi business. Your job is to control the taxis in your city. To be successful you have to keep your passengers requests in mind. Businessmen are always in a hurry, be quick enough to catch them. Tourists like to see the city’s highlights and everyone likes a ride in a fancy car. But be aware to not get stuck in the rush hour.
Invest your money wisely: Hire new drivers, buy new cars and upgrade your fleet. If you are good enough, you might even be able to buy a limousine to transport your clients in style. Gain some extra money by completing challenges and try to beat the highscores."
Mages is a 2D multiplayer arena game in a fantasy setting that offers a combination of fast gameplay and tactical resource managment. It was developed and published by Little Factory. You can download the prototype at www.little-factory-games.com/Mages.
of up to 6 Users in the Browser Using Kinect and XML3D
Authors: Michael Hedderich, Magdalena Kaiser, Dushyant Mehta and Guillermo Reyes
This project can and has been used as basis for combining other tracking hardware with XML3D. Our work on the Kinect and retargeting part can also be used for other forms of tracking and visualization.
You can find the documented code on GitHub.
We have created a presentation that explains the structure of the system, technical details of our implementation, open problems, etc. If you still have questions or remarks, do not hesitate to contact us.
Seminar Deep Learning
Seminar talk about
Venugopalan, Xu, Donahue, Rohrbach, Mooney, Saenko: Translating Videos to Natural Language Using Deep Recurrent Neural Networks, NAACL HLT (2015).
All images and figures I created for the presentation are licensed under Creative Commons.
Construction of a Virtual Reality Environment
Based on Position Tracking Combined with a Head-Mounted Display
Virtual reality offers fascinating possibilities to experience computer generated worlds. We present a virtual reality system that enables the user to naturally walk through the virtual environment. He or she can crouch, jump, turn around, look upwards and downwards, etc. The user wears an active marker which is tracked by a camera and a special tracking software. The information about its position and rotation is sent to a head-mounted device which has a smartphone as a display. The smartphone renders a stereoscopic scene from the user’s perspective.
We developed a first prototype that demonstrates the feasibility of our concept. Our implementation aims at museums or exhibitions that want to bring virtual worlds to life. Nevertheless, this technique can be extended to a variety of applications from scientific visualization to augmented reality. In this thesis we explain all the parts in detail. As background a short description of the theory of stereoscopic projection is given. We conclude with an extensive evaluation explaining the problems that still exist and showing promising improvements for future work.