FastMoE

library
BAAI
First open-source system for training trillion-parameter MoE models on commodity hardware.
GitHubPaper (arXiv)

Library

GitHub Repository

moetraininginfrastructure