Tracking issue for the Transformer model

[FL03]

The transformer model was introduced in 2017 with the release of the Attention is all you need paper. Over the last few years, the popularity of the Transformer model has grown exponentially due to its abilities in natural language processing (NLP) workloads.

Transformers are mathematically simply, primarily relying on dot-product / matrix multiplication algorithms.

Issues

• Scaled Dot-Product Attention #68
• Multi-Head Attention #69
• Implement the Decoder for the Transformer model #82
• Implement an Encoder layer for the Transformer model #83
• Implement the piece-wise feed-forward neural network for the Transformer model #84

Pull Requests

Resources

• Attention is all you need
• The Annotated Transformer