APOLLO

"SGD-like Memory, AdamW-level Performance" (MLSys 2025 Outstanding Paper Honorable Mention): approximates channel-wise gradient scaling in a low-rank auxiliary space using random projections, cutting optimizer memory toward SGD levels while matching or beating AdamW on LLaMA-style pretraining up to 7B. Integrated into Hugging Face ecosystem tooling (Transformers/LLaMA-Factory). VITA group (Zhangyang Wang, David Pan) jointly with Meta AI — the capstone of the GaLore → Q-GaLore memory-efficient-training line.