MegaScale

library
ByteDance
System for scaling LLM training to over 10,000 GPUs.
Paper (arXiv)

Paper

arXiv: 2402.15627

infrastructuretrainingscaling