Skip to content

Latest commit

 

History

History
109 lines (75 loc) · 3.48 KB

README.md

File metadata and controls

109 lines (75 loc) · 3.48 KB

Deep Learning for NLP

Understanding Classic Deep Learning NLP Methods and their Implementations. Do Something Useful in a Long Time Feature. Slow but Valuable.

Terms

  • NLI: Natural Language Inference
  • NLU: Natural Language Understanding
  • NLG: Natural Language Generation
  • QA: Question Answering
  • MRC: Machine Reading Comprehension

Code Reading

Valuable Blogs

Important Tools

  • stanfordnlp/stanza

    • The Stanford NLP Group's official Python NLP library. It contains support for running various accurate natural language processing tools on 60+ languages and for accessing the Java Stanford CoreNLP software from Python. website
    • pytorch, 2020
    • 66 languages(including Chinese); from raw text and pipeline without error progation; high perfermance;
    • language-agnostic

  • spaCy

    • 10 languages(not including Chinese);
    • python

  • coreNLP

    • 6 languages(including Chinese);
    • Java, 2014

Important Board

  1. MS MARCO

    • KeyPhrase Extraction(10/18/2019) ranked by F1 @3 on Eval
      • OpenKP (OpenKeyPhrase) is a large scale, open-domain keyphrase extraction dataset,
      • which was first released in the paper Open Domain Web Keyphrase Extraction Beyond Language Modeling at EMNLP-IJCNLP 2019.
    • Passage Retrieval(10/26/2018-Present) ranked by MRR on Eval
    • Q&A Task(03/01/2018-Present)
    • Q&A + Natural Language Generation Task(03/01/2018-Present)
    • MS MARCO V1 Leaderboard(12/01/2016-03/31/2018)

  2. GLUE: General Language Understanding Evaluation benchmark

Important Links

  1. thunlp/RCPapers
  2. Overview of Modern Deep Learning Techniques Applied to Natural Language Processing
  3. Hankcs

Important Agencies

  1. [Google AI]
    • Transformer
    • Bert
  2. Open AI
    • GPT1.0/2.0
  3. MSRA
    • MT-DNN

Courses

  1. 2019 Stanford, CS224U: Natural Language Understanding

  2. 2019 Standfor, CS224N: Natural Language Processing with Deep Learning

Problems

  • language modeling

  • sequence labeling

    • POS tagging (part-of-speech tagging)
    • Chunking
    • NER ( named entity recognition)
    • Semantic role labeling

  • machine translation

  • sentence matching/paraphrase

  • text classification

Research dire

Services

  • ShannonAI/service-streamer
    • ServiceStreamer is a middleware for web service of machine learning applications. Queue requests from users are sampled into mini-batches.
    • ServiceStreamer can significantly enhance the overall performance of the system by improving GPU utilization.
    • A good and fast deep learning deploy frames.

NLP Corpus