Q-GaLore

Quantized successor to GaLore (ICML 2024 oral, merged into Hugging Face Transformers): INT4 projection matrices plus INT8 weights with adaptive lazy subspace updates, enabling LLaMA-7B pretraining from scratch on a single 16GB GPU. In-window continuation of the VITA group's memory-efficient training line (GaLore → Q-GaLore → APOLLO).