MacLustr AI — Intelligence, running locally.
High-performance AI inference powered by Apple Silicon and MLX.
If you speak OpenAI, you already speak MacLustr.
Keep your SDK, your message format and your streaming loop. Change the base URL, use a MacLustr key, and call the model served on our hardware.
- Base URL
https://llm.spboucher.ai/v1- Model
maclustr-qwen3- Auth
Authorization: Bearer mlx_sk_…
Inference infrastructure, not a chat widget.
Everything below is implemented in the gateway today and described precisely in the documentation.
-
OpenAI-compatible
Point the official OpenAI SDKs at a new base URL. Chat completions, text completions and model listing follow the same request and response shapes.
Learn more -
Token streaming
Server-sent events with keepalive comments during prefill, an optional usage chunk, and explicit in-stream error events.
Learn more -
Private by construction
Inference runs on our own Apple Silicon hardware. No third-party model API is involved, and prompts and completions are never written to logs: only counters and timings are recorded.
Learn more -
Tool calling
Declare functions, receive structured tool calls, return results, continue the conversation. Each call arrives whole in a single chunk.
Learn more -
Validated JSON
JSON object and JSON Schema modes are checked server-side, with one automatic repair attempt on non-streaming requests. Honest semantics: validated, not grammar-constrained.
Learn more -
Per-app keys and limits
Each application gets its own key, stored hashed, with scoped models and endpoints, request and token rates, daily and monthly quotas, and a concurrency cap.
Learn more -
Fair scheduling
Four priority classes with anti-starvation aging, a bounded queue, and backpressure that answers 503 with Retry-After instead of timing out silently.
Learn more -
Batching and prefix cache
The MLX engine batches concurrent generations and reuses KV state for shared prompt prefixes. Cached prompt tokens are reported in every usage object.
Learn more
A short, observable path from request to token.
Requests are authenticated, metered and queued by priority before they reach the MLX engine. The engine and the gateway run on the same Apple Silicon machine.
Start with one request.
Read the five-minute guide, or try the model against the real API in the Playground.