Jon Saad-FalconandClaude Opus 4.6 961b451047 feat(rust): add vLLM, SGLang, and llama.cpp engine backends ...

Adds three new inference engine implementations to the Rust engine crate:
- VLLMEngine: OpenAI-compatible /v1/chat/completions API
- SGLangEngine: OpenAI-compatible API with SGLang-specific features
- LlamaCppEngine: Native /completion endpoint with ChatML prompt formatting

All backends implement InferenceEngine trait with generate, stream,
list_models, and health methods. Includes comprehensive unit tests.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

2026-03-07 00:28:29 +00:00

..

crates

feat(rust): add vLLM, SGLang, and llama.cpp engine backends

2026-03-07 00:28:29 +00:00

Cargo.lock

Add Generics, reduce dynamic dispatch, python integration

2026-03-04 18:13:45 -08:00

Cargo.toml

Add Generics, reduce dynamic dispatch, python integration

2026-03-04 18:13:45 -08:00