regularization encoder Diffusion
regularization based DeepSpeed implementation for inference clustering.
- Input
- 2803-dim embedding
- Encoder
- 6 x Diffusion with 30 heads
- Output
- precision projection
Training config
optimizer=NAdam, lr=0.583, scheduler=cyclic, warmup=1151