Code

[train_engine] support fsdp#2412

Code

Merged

xingchensong merged 26 commits intomainwenet-e2e/wenet:mainfrom Mddct-fsdpwenet-e2e/wenet:Mddct-fsdpCopy head branch name to clipboardApr 7, 2024

+290-55

Commits

Commits on Mar 15, 2024

• [train_engine] support fsdp

  Mddct

  committed
  9406947

• [train_engine] support fsdp

  Mddct

  committed
  67c1451

• unify scaler and amp

  Mddct

  committed
  eb5c720

• fp32&&fp16 works in fsdp env

  Mddct

  committed
  62e018d

• Merge branch 'main' into Mddct-fsdp

  Mddct

  committed
  3475bff

• fix fsdp in cv auto cast

  Mddct

  committed
  d13de47

• try to fix wenet.join fsdp

  Mddct

  committed
  05ccbf3

Commits on Mar 17, 2024

• implementing zero1 under fsdp is almost equivalent to deepspeed's zero1

  Mddct

  committed
  caa97c6

Commits on Mar 18, 2024

• fix clip_and_grad_

  Mddct

  committed
  24639e4

• fix train summary

  Mddct

  committed
  eda080d

• merge main

  Mddct

  committed
  c2a832a

Commits on Mar 22, 2024

• all wenet xxxformer works (-paraformer -transducer)

  Mddct

  committed
  0644294

• Merge branch 'main' into Mddct-fsdp

  Mddct

  committed
  2b966f0

Commits on Mar 23, 2024

• try to fix nan

  Mddct

  committed
  603c600

Commits on Mar 24, 2024

• add barrier for cv

  Mddct

  committed
  979c3e1

• add destroy group for end of all train

  Mddct

  committed
  5f5010a

Commits on Mar 26, 2024

• refactor wrap methods and ckpt works

  Mddct

  committed
  79448ff

• fix ckpt

  Mddct

  committed
  f7aea43

Commits on Mar 27, 2024

• fix cv in dtype != float32

  Mddct

  committed
  1740e39

Commits on Mar 30, 2024

• fix ckpt in model mode

  Mddct

  committed
  6008bab

Commits on Apr 5, 2024

• fix bf16 amp

  Mddct

  committed
  2f1f4df

Commits on Apr 6, 2024

• refactor scaler and autocast, fix fp32 fp16 bf16 for fsdp

  Mddct

  committed
  2d70039

Commits on Apr 7, 2024

• fix fp32 nullcontext to nullcontext()

  Mddct

  committed
  767b0ed

• modify after review

  Mddct

  committed
  9d615e2

• fix lint

  Mddct

  committed
  8716038

• fix lint

  Mddct

  committed
  a92ee10