Skip to content

AlpacaTag: An Active Learning-based Crowd Annotation Framework for Sequence Tagging (ACL 2019 Demo)

Notifications You must be signed in to change notification settings

INK-USC/AlpacaTag

Folders and files

NameName
Last commit message
Last commit date

Latest commit

ce04045 · Jan 9, 2021
Dec 11, 2020
Dec 16, 2020
Dec 18, 2020
Jul 30, 2019
Dec 17, 2020
Dec 18, 2020
Dec 18, 2020
Jan 30, 2020
Jul 24, 2019
Mar 24, 2020
Mar 24, 2020
Dec 14, 2020
Dec 16, 2020
Jul 24, 2019
Jun 18, 2019

Repository files navigation

AlpacaTag

AlpacaTag is an active learning-based crowd annotation framework for sequence tagging, such as named-entity recognition (NER).

Website      Documenations      Paper      Poster

The UI framework of AlpacaTag is based on the awesome work of Doccano, while the distinctive advantages of AlpacaTag are three-fold:

  • Active intelligent recommendation: dynamically suggesting annotations and sampling the most informative unlabeled instances with a back-end active learned model.

  • Automatic crowd consolidation: enhancing real-time inter-annotator agreement by merging inconsistent labels from multiple annotators.

  • Real-time model deployment: users can deploy their models in downstream systems while new annotations are being made.

  • Overall Workflow

Documentations

Citation

@InProceedings{acl19alpaca, 
     author = {Bill Yuchen Lin and Dongho Lee and Frank F. Xu and Ouyu Lan and Xiang Ren}, 
     title = {AlpacaTag: An Active Learning-based Crowd Annotation Framework for Sequence Tagging.}, 
     booktitle = {Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics (ACL), Demo Track},
     year = {2019} 
}

Demo

Back-end Model APIs

Crowd Consolidation

Performance Study