Xinference

Unified, production-ready inference API for open-source LLMs, speech, and multimodal models.

Bintang

9,648

Fork

897

Bahasa

Python

Kategori

Inference & Serving

Dapatkan kode

# Clone repositori
git clone https://github.com/xorbitsai/inference

Apa itu Xinference?

Unified, production-ready inference API for open-source LLMs, speech, and multimodal models.

Apakah Xinference aman digunakan?

Xinference adalah proyek open source dari xorbitsai di GitHub. Tinjau kode sumber dan lisensi sebelum menggunakannya di produksi.

Bagaimana memulai dengan Xinference?

git clone https://github.com/xorbitsai/inference

Xinference ditulis dalam bahasa apa?

openSource.faqWhatLanguageAnswer

Run large models locally with a single command — LLMs, vision, and embeddings on any machine.

LLM inference in C/C++ with quantization — runs models on CPU, GPU, and edge devices.

High-throughput, memory-efficient inference and serving engine for LLMs with PagedAttention.