[hotfix][minor] Poolformer does not need projections (#349)

* skip the projection layer for poolformer * changelog
facebookresearch · Jul 3, 2022 · 6c003f1 · 6c003f1
1 parent 7fdb90d
commit 6c003f1
Show file tree

Hide file tree

Showing 2 changed files with 2 additions and 0 deletions.
diff --git a/CHANGELOG.md b/CHANGELOG.md
@@ -9,6 +9,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 ### Fixed
 - Removed dupliacated biases in the FusedMLP layers [#317]
 - Rotary embeddings respecting input types [#326]
+- Poolformer style instantiating useless projection layers [#349]
 
 ### Added
 - Four blocksparsity layouts from DeepSpeed [#320]

diff --git a/xformers/components/attention/pooling.py b/xformers/components/attention/pooling.py
@@ -58,6 +58,7 @@ def __init__(
 
         # This "attention" (token mixing) skips the multihead attention altogether
         self.requires_skip_multi_head = True
+        self.requires_input_projection = False
 
         # This operator does not really handle q,k,v
         self.requires_same_k_q_dimensions = True