Tweet by XianhangLi
October 1, 2025
🤔 Ever thought a small teacher could train a student 6× larger that sets new SOTA in training efficiency and frozen evaluation performance for video representation learning? 🤔 Do we really need complex EMA-based self-distillation to prevent collapse, bringing unstable loss dynamics while offering little insight into representation quality? 🚨 In our new paper, we investigate these questions and propose SALT (Static-teacher Asymmetric Latent Training): a simple, scalable, and compute-efficient alternative for video representation learning. 📄 Rethinking JEPA: Compute-Efficient Video SSL with Frozen Teachers 🔗 https://t.co/C9amVFddSH
- Author
- XianhangLi
- Date
- October 1, 2025
- Canonical URL
- /tweets/xianhangli-1973214068644450661-9405a2