Inference by Xorbits AI

Overview

Xinference allows users to easily swap GPT for any LLM with minimal code changes, enabling the deployment of various open-source and multimodal models. It is designed for developers looking to implement a unified inference API across cloud, on-premises, or local environments.