Lightweight distributed data processing framework built on DuckDB and 3FS. Sorted 110.5 TiB in 30 minutes at 3.66 TiB/min. Released as part of DeepSeek Open Source Week.

Library

GitHub Repository

infrastructuredata-processingopen-source

Related