Publications

Lessons Learned from a Citizen Science Project for Natural Language Processing
Many Natural Language Processing (NLP) systems use annotated corpora for training and evaluation. However, labeled data is often costly …
Jan-Christoph Klie, Ji-Ung Lee, Kevin Stowe, Gözde Şahin, Nafise Sadat Moosavi, Luke Bates, Dominic Petrak, Richard Eckart de Castilho, Iryna Gurevych
Cite DOI URL
Annotation Error Detection: Analyzing the Past and Present for a More Coherent Future
Annotated data is an essential ingredient in natural language processing for training and evaluating machine learning models. It is …
Jan-Christoph Klie, Bonnie Webber, Iryna Gurevych
Cite DOI URL
Analyzing Dataset Annotation Quality Management in the Wild
Jan-Christoph Klie, Richard Eckart de Castilho, Iryna Gurevych
Cite arXiv
Annotation Curricula to Implicitly Train Non-Expert Annotators
Annotation studies often require annotators to familiarize themselves with the task, its annotation scheme, and the data domain. This …
Ji-Ung Lee, Jan-Christoph Klie, Iryna Gurevych
Cite DOI URL
From Zero to Hero: Human-In-The-Loop Entity Linking in Low Resource Domains
Entity linking (EL) is concerned with disambiguating entity mentions in a text against knowledge bases (KB). It is crucial in a …
Jan-Christoph Klie, Richard Eckart de Castilho, Iryna Gurevych
Cite DOI URL
A Multi-Platform Annotation Ecosystem for Domain Adaptation
This paper describes an ecosystem consisting of three independent text annotation platforms. To demonstrate their ability to work in …
Richard Eckart de Castilho, Nancy Ide, Jin-Dong Kim, Jan-Christoph Klie, Keith Suderman
Cite DOI URL
Integrating Knowledge-Supported Search into the INCEpTION Annotation Platform
Annotating entity mentions and linking them to a knowledge resource are essential tasks in many domains. It disambiguates mentions, …
Beto Boullosa, Richard Eckart de Castilho, Naveen Kumar, Jan-Christoph Klie, Iryna Gurevych
Cite DOI URL
The INCEpTION Platform: Machine-Assisted and Knowledge-Oriented Interactive Annotation
We introduce INCEpTION, a new annotation platform for tasks including interactive and semantic annotation (e.g., concept linking, fact …
Jan-Christoph Klie, Michael Bugert, Beto Boullosa, Richard Eckart de Castilho, Iryna Gurevych
Cite URL
Multimodal Frame Identification with Multilingual Evaluation
An essential step in FrameNet Semantic Role Labeling is the Frame Identification (FrameId) task, which aims at disambiguating a …
Teresa Botschen, Iryna Gurevych, Jan-Christoph Klie, Hatem Mousselly-Sergieh, Stefan Roth
Cite DOI URL