Symphony-MoE

A novel framework for merging disparate, identically-architected pre-trained models (e.g., Qwen2.5-Coder and Qwen2) into a coherent Mixture-of-Experts (MoE) model. Symphony-MoE uses a two-stage approach involving layer-aware fusion and functional alignment to harmonize mismatched parameter spaces.