Pool spare GPU capacity to run LLMs at larger scale

mesh-llm is a tool that pools spare GPU capacity across machines to run LLMs at larger scale, automatically distributing models via pipeline parallelism for dense models or expert sharding for MoE models with zero cross-node inference traffic. It provides an OpenAI-compatible API, supports multi-model serving, latency-aware peer selection, and demand-aware rebalancing, requiring only a single command to join or create a mesh network.