Using Malibu with the OpenAI SDK

The OpenAI SDK is the Malibu SDK. Every buyer-facing pattern in one page.

Malibu speaks OpenAI-compatible /v1/chat/completions. If you have openai-python or openai-node installed, you already have a Malibu client: set the base URL and use your Malibu API key. This page collects copy-paste patterns for streaming, tool calling, structured output, prompt-cache reuse, provider attribution, and signed-receipt capture. There is no malibu SDK, by design. Nearly every agent framework already targets openai, and a wrapper would only add friction. For the few Malibu-specific pieces (one header, a receipt), you set a header on the client or run the receipt verifier. Clients that speak the Responses API or Anthropic Messages instead can use the Responses and Messages facades.

Base setup

from openai import OpenAI

client = OpenAI(
    base_url="https://api.malibu.tech/v1",
    api_key="<your-malibu-api-key>",
)

TypeScript / JavaScript:

import OpenAI from "openai";

const client = new OpenAI({
  baseURL: "https://api.malibu.tech/v1",
  apiKey: process.env.MALIBU_API_KEY,
});

Get an API key from the console: Get an API key. To have a coding agent do the wiring, see Agent-readable onboarding (https://get.malibu.tech/skill.md).

Basic completion

resp = client.chat.completions.create(
    model="mlx-community/Llama-3.2-3B-Instruct-4bit",
    messages=[{"role": "user", "content": "Hello, world"}],
)
print(resp.choices[0].message.content)

The set of models being served changes as Macs connect and disconnect. List what is routable now with client.models.list(), which needs your key. Examples on this page use mlx-community/Llama-3.2-3B-Instruct-4bit, which was served by several providers on 23 Sep 2026.

Streaming

stream = client.chat.completions.create(
    model="mlx-community/Llama-3.2-3B-Instruct-4bit",
    messages=[{"role": "user", "content": "Explain WebSocket ping/pong in one sentence."}],
    stream=True,
    stream_options={"include_usage": True},
)
for chunk in stream:
    if chunk.choices:
        print(chunk.choices[0].delta.content or "", end="", flush=True)
    if chunk.usage:
        print("\n", chunk.usage)

These are Server-Sent Events with OpenAI-compatible chunks. The final chunk has an empty choices list and carries usage; Malibu sends it whether or not you set include_usage. Streaming responses carry no receipt header.

Tool calling

Malibu accepts OpenAI-shape tools and tool_choice. That covers multi-turn tool loops, streamed tool-call deltas, and tool_call_id back-references.

resp = client.chat.completions.create(
    model="mlx-community/Qwen3.6-27B-4bit",
    messages=[{"role": "user", "content": "What's the weather in Vilnius?"}],
    tools=[
        {
            "type": "function",
            "function": {
                "name": "get_weather",
                "description": "Get the current weather for a city",
                "parameters": {
                    "type": "object",
                    "properties": {"city": {"type": "string"}},
                    "required": ["city"],
                },
            },
        }
    ],
)

for call in resp.choices[0].message.tool_calls or []:
    print(call.function.name, call.function.arguments)  # arguments is a JSON string

Note. function.arguments is a JSON-encoded string, as in OpenAI’s wire shape. Call json.loads(call.function.arguments) to get a dict. Caps: 1 MiB per function.arguments, 2 MiB of tool-call arguments per response, 256 KiB per role: "tool" message. Only some model families emit tool_calls[]: Qwen-family models do, while Llama 3.1 / 3.2 reply with plain text. See Tool calling for the model list. For your obligations as a buyer before executing emitted tool calls, see Agentic tooling → Buyer-side validation.

Structured output (JSON schema)

The provider validates the output against your schema after inference. This is validation, not constrained decoding: output that fails validation returns an error instead of invalid JSON. It works in both streaming and non-streaming mode. Strict schemas need additionalProperties: false on every object.

resp = client.chat.completions.create(
    model="mlx-community/Llama-3.2-3B-Instruct-4bit",
    messages=[
        {"role": "system", "content": "Extract contact info as JSON."},
        {"role": "user", "content": "Call John at 555-0100, john@example.com"},
    ],
    response_format={
        "type": "json_schema",
        "json_schema": {
            "name": "contact",
            "schema": {
                "type": "object",
                "properties": {
                    "name": {"type": "string"},
                    "phone": {"type": "string"},
                    "email": {"type": "string"},
                },
                "required": ["name", "phone", "email"],
                "additionalProperties": False,
            },
            "strict": True,
        },
    },
)

import json
contact = json.loads(resp.choices[0].message.content)

Supported schema features and limits: Structured output.

Prompt-cache reuse and sticky conversations

Repeated scaffolds (the same system prompt and tools) are cached automatically, with no client change. usage.prompt_tokens_details.cached_tokens reports the reuse, and those tokens bill at the normal prompt rate. To ask the router to prefer the provider that served earlier turns, and get the cache-hit rate on reused tokens when it does, send a stable tag in X-MacProvider-Conversation:

client_sticky = OpenAI(
    base_url="https://api.malibu.tech/v1",
    api_key="<your-malibu-api-key>",
    default_headers={"X-MacProvider-Conversation": "thread-42"},
)

history = [{"role": "user", "content": "Summarize our plan so far."}]

resp = client_sticky.chat.completions.create(
    model="mlx-community/Llama-3.2-3B-Instruct-4bit",
    messages=history,  # your full message list, resent each turn
)
print(resp.usage.prompt_tokens_details.cached_tokens, getattr(resp.usage, "cached_prompt_tokens", 0))

The tag is opaque and buyer-chosen: 1–128 characters from [A-Za-z0-9._:-]. An empty header is ignored. A whitespace-only or too-long tag, or one with other characters, returns 400.
The gateway HMACs it with your account_id before forwarding, so tags never collide across accounts.
Stickiness is a preference, not a guarantee. The request often routes to another provider, and then cached_prompt_tokens is 0. The cache discount applies only when the sticky provider serves the request.

Details: Sticky conversations.

Which provider served this request?

Every response that reached a provider carries X-Provider-Id:

raw = client.chat.completions.with_raw_response.create(
    model="mlx-community/Llama-3.2-3B-Instruct-4bit",
    messages=[{"role": "user", "content": "Hello"}],
)
served_by = raw.headers.get("X-Provider-Id")
request_id = raw.headers.get("X-Request-ID")

It is informational. Buyers cannot pin requests to a provider. See Provider attribution.

Signed inference receipts

Some non-streaming responses carry a v0.3 receipt in the X-MacProvider-Receipt header (currently roughly a quarter to a third). Streams never do. For the rest, use GET /v1/receipts. The provider signs a tuple that binds the canonical prompt hash, output hash, model ID, provider public key, and timestamp. Capture it like this:

raw = client.chat.completions.with_raw_response.create(
    model="mlx-community/Llama-3.2-3B-Instruct-4bit",
    messages=[{"role": "user", "content": "Hello"}],
)
resp = raw.parse()
receipt = raw.headers.get("X-MacProvider-Receipt")
provider_id = raw.headers.get("X-Provider-Id")

Offline verification of current receipts is pending a verifier update. See Receipt verifier. Keep the receipt with the raw request and response bodies so you can verify it later. Details: Signed receipts.

Full working example

import json
from openai import OpenAI

client = OpenAI(
    base_url="https://api.malibu.tech/v1",
    api_key="<your-malibu-api-key>",
    default_headers={"X-MacProvider-Conversation": "session-2026-09-23-a"},
)

raw = client.chat.completions.with_raw_response.create(
    model="mlx-community/Llama-3.2-3B-Instruct-4bit",
    messages=[
        {"role": "system", "content": "You are a helpful assistant."},
        {"role": "user", "content": "Give me one interesting fact about Apple Silicon."},
    ],
    response_format={
        "type": "json_schema",
        "json_schema": {
            "name": "fact",
            "schema": {
                "type": "object",
                "properties": {"fact": {"type": "string"}, "source": {"type": "string"}},
                "required": ["fact", "source"],
                "additionalProperties": False,
            },
            "strict": True,
        },
    },
)
resp = raw.parse()
receipt = raw.headers.get("X-MacProvider-Receipt")

print("request id: ", raw.headers.get("X-Request-ID"))
print("served by:  ", raw.headers.get("X-Provider-Id"))
print("cached toks:", getattr(resp.usage, "cached_prompt_tokens", 0))
print("receipt:    ", (receipt or "none")[:60])
print("payload:    ", json.loads(resp.choices[0].message.content))

Header reference

Header	Direction	Purpose
`X-MacProvider-Conversation`	Request	Sticky-conversation tag for cache-hit billing
`Idempotency-Key`	Request	Reject duplicate submissions with `409` (no replay)
`X-Request-ID`	Response	Request identifier (also accepted on request)
`X-Provider-Id`	Response	Which provider served this request
`X-MacProvider-Receipt`	Response	Ed25519-signed receipt (some non-streaming responses)
`X-RateLimit-*`	Response	Daily token quota

Full reference: API → Headers.

Framework compatibility

Frameworks built on openai-python or openai-node take a base URL and an API key and should work unchanged. Malibu does not certify each one.

LangChain. ChatOpenAI(base_url="https://api.malibu.tech/v1", api_key=...)
LlamaIndex. Use OpenAILike(api_base="https://api.malibu.tech/v1", api_key=..., model=...), since Malibu model IDs are not OpenAI model names.
Instructor. Structured output via response_format goes through the patched OpenAI client. Pydantic models that generate $ref or anyOf (for example Optional fields) are rejected with 400, so flatten them first.
Aider / Cline / Continue. Set the OpenAI-compatible base URL to https://api.malibu.tech/v1.

If the OpenAI SDK works against OpenAI but a request fails against Malibu, that is a wire-shape bug. Report it with the X-Request-ID and a minimal repro.

What’s not in the OpenAI SDK

Receipt verification is an offline check, so it lives in a separate tool (receipt verifier). Verification of current receipts is pending a verifier update. Routing, retries, quota, the model list, tool calling, structured output, streaming, and prompt-cache reuse all work through the OpenAI SDK.

Litepaper

Getting started

Buyers

Providers

Network & trust

Guides index

Ledger

Using Malibu with the OpenAI SDK

Using Malibu with the OpenAI SDK

Base setup

Basic completion

Streaming

Tool calling

Structured output (JSON schema)

Prompt-cache reuse and sticky conversations

Which provider served this request?

Signed inference receipts

Full working example

Header reference

Framework compatibility

What’s not in the OpenAI SDK

​Using Malibu with the OpenAI SDK

​Base setup

​Basic completion

​Streaming

​Tool calling

​Structured output (JSON schema)

​Prompt-cache reuse and sticky conversations

​Which provider served this request?

​Signed inference receipts

​Full working example

​Header reference

​Framework compatibility

​What’s not in the OpenAI SDK

Using Malibu with the OpenAI SDK

Base setup

Basic completion

Streaming

Tool calling

Structured output (JSON schema)

Prompt-cache reuse and sticky conversations

Which provider served this request?

Signed inference receipts

Full working example

Header reference

Framework compatibility

What’s not in the OpenAI SDK