Chat Completions

The chat completions endpoint is fully compatible with the OpenAI chat completions API. Lilac serves models via a customized fork of vLLM tuned for idle-GPU scheduling and shared warm endpoints, so you get access to both standard OpenAI parameters and vLLM-specific extras.

Endpoint

POST https://api.getlilac.com/v1/chat/completions

Basic Example

Python
JavaScript
cURL

from openai import OpenAI

client = OpenAI(
    base_url="https://api.getlilac.com/v1",
    api_key="your-lilac-api-key",
)

response = client.chat.completions.create(
    model="moonshotai/kimi-k2.6",
    messages=[
        {"role": "system", "content": "You are a helpful assistant."},
        {"role": "user", "content": "What is GPU inference?"},
    ],
)

print(response.choices[0].message.content)

import OpenAI from "openai";

const client = new OpenAI({
  baseURL: "https://api.getlilac.com/v1",
  apiKey: "your-lilac-api-key",
});

const response = await client.chat.completions.create({
  model: "moonshotai/kimi-k2.6",
  messages: [
    { role: "system", content: "You are a helpful assistant." },
    { role: "user", content: "What is GPU inference?" },
  ],
});

console.log(response.choices[0].message.content);

curl https://api.getlilac.com/v1/chat/completions \
  -H "Authorization: Bearer your-lilac-api-key" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "moonshotai/kimi-k2.6",
    "messages": [
      {"role": "system", "content": "You are a helpful assistant."},
      {"role": "user", "content": "What is GPU inference?"}
    ]
  }'

Request Parameters

Required

Parameter	Type	Description
`model`	`string`	Model ID (e.g., `moonshotai/kimi-k2.6`). See Models.
`messages`	`array`	Conversation history. Each message has a `role` (`system`, `user`, `assistant`, `tool`) and `content`.

Sampling

Parameter	Type	Default	Description
`temperature`	`float`	`1.0`	Sampling temperature (0–2). Lower values are more deterministic.
`top_p`	`float`	`1.0`	Nucleus sampling — considers tokens with cumulative probability >= `top_p`.
`top_k`	`integer`	`-1`	Limits sampling to the top K tokens. `-1` disables.
`min_p`	`float`	`0.0`	Minimum relative probability threshold for token consideration.
`seed`	`integer`	`null`	Seed for deterministic sampling (best effort).

Output

Parameter	Type	Default	Description
`max_tokens`	`integer`	model-dependent	Maximum tokens to generate.
`max_completion_tokens`	`integer`	`null`	Upper bound including reasoning tokens. Preferred for reasoning models.
`n`	`integer`	`1`	Number of completions to generate.
`stop`	`string` or `array`	`null`	Up to 4 sequences where generation stops.
`stream`	`boolean`	`false`	Stream partial token deltas via SSE.
`stream_options`	`object`	`null`	Options like `{"include_usage": true}` to get token counts in the stream.

Penalties

Parameter	Type	Default	Description
`frequency_penalty`	`float`	`0.0`	Penalizes tokens by frequency in output so far (-2.0 to 2.0).
`presence_penalty`	`float`	`0.0`	Penalizes tokens that have appeared at all (-2.0 to 2.0).
`repetition_penalty`	`float`	`1.0`	Multiplicative penalty on repeated tokens. `1.0` = no penalty.
`logit_bias`	`object`	`null`	Map of token ID → bias value (-100 to 100).

Log Probabilities

Parameter	Type	Default	Description
`logprobs`	`boolean`	`false`	Return log probabilities of output tokens.
`top_logprobs`	`integer`	`null`	Number of most likely tokens to return at each position (0–20). Requires `logprobs: true`.

Tool Calling

Parameter	Type	Default	Description
`tools`	`array`	`null`	List of tool definitions with `type: "function"` and a function schema.
`tool_choice`	`string` or `object`	`"auto"`	`"none"`, `"auto"`, `"required"`, or `{"type": "function", "function": {"name": "..."}}`.

Structured Output

Parameter	Type	Default	Description
`response_format`	`object`	`null`	`{"type": "text"}`, `{"type": "json_object"}`, or `{"type": "json_schema", "json_schema": {...}}`.

Reasoning

Parameter	Type	Default	Description
`chat_template_kwargs`	`object`	`null`	Toggles model behavior exposed by the chat template. Used to enable or disable chain-of-thought reasoning — see below.

Some models (like Kimi K2.6 and GLM 5.2) include chain-of-thought reasoning by default. When reasoning is active, the model’s chain-of-thought is returned in a separate reasoning field on the response message. Reasoning tokens are included in completion_tokens and count toward your usage — including all reasoning controls below. These are model-specific chat-template controls, not universal OpenAI parameters. The exact key and accepted values depend on each model’s chat template.

Per-model reasoning controls

Model	Control	Values	Default
Kimi K2.6	`chat_template_kwargs.thinking`	`true` \| `false`	on
GLM 5.2	`reasoning_effort` or `chat_template_kwargs.reasoning_effort`	`max` \| `high`	`high`
GLM 5.2	`chat_template_kwargs.enable_thinking`	`true` \| `false`	on
Gemma 4	`chat_template_kwargs.enable_thinking`	`true` \| `false`	off
MiniMax M3	`chat_template_kwargs.thinking_mode`	`adaptive` \| `enabled` \| `disabled`	`adaptive`

Notes:

GLM 5.2 supports reasoning_effort with two levels — high (default) and max. max is the opt-in highest-quality mode for long-horizon agentic and complex problem-solving tasks, at the cost of higher latency and token usage. You can send reasoning_effort either as a top-level field (OpenAI-style) or inside chat_template_kwargs. Disable thinking entirely with chat_template_kwargs.enable_thinking: false; when thinking is disabled, reasoning_effort has no effect.
MiniMax M3 does not use a boolean toggle. It accepts thinking_mode with three values: adaptive (default — the model decides), enabled (always think), and disabled (never think).
Defaults differ per model: Kimi K2.6 and GLM 5.2 have reasoning on by default; Gemma 4 has reasoning off by default; MiniMax M3 is adaptive by default. See the Models page for per-model details.

Thinking toggle keys

The key that toggles reasoning is defined by each model’s chat template, not by the API, so it differs per model family:

Key	Models that honor it
`thinking` (bool)	Moonshot chat templates — e.g. Kimi K2.6
`enable_thinking` (bool)	Z.ai GLM / Google Gemma / SGLang-style templates — e.g. GLM 5.x, Gemma 4
`reasoning_effort` (`max` \| `high`)	GLM 5.2 — also accepted as a top-level field
`thinking_mode` (`adaptive` \| `enabled` \| `disabled`)	MiniMax M3

Unknown keys inside chat_template_kwargs are silently ignored by chat templates, so the safe, forward-compatible approach is to send both keys. This works across all current Lilac models and any future model whose template uses either convention:

{ "chat_template_kwargs": { "thinking": false, "enable_thinking": false } }

Python
JavaScript
cURL

# Disable reasoning (works for both Moonshot- and GLM-style templates)
response = client.chat.completions.create(
    model="moonshotai/kimi-k2.6",
    messages=[{"role": "user", "content": "What is 2+2?"}],
    extra_body={
        "chat_template_kwargs": {
            "thinking": False,
            "enable_thinking": False,
        }
    },
)

const response = await client.chat.completions.create({
  model: "moonshotai/kimi-k2.6",
  messages: [
    { role: "user", content: "What is 2+2?" }
  ],
  chat_template_kwargs: {
    thinking: false,
    enable_thinking: false,
  },
});

curl https://api.getlilac.com/v1/chat/completions \
  -H "Authorization: Bearer your-lilac-api-key" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "moonshotai/kimi-k2.6",
    "messages": [
      {"role": "user", "content": "What is 2+2?"}
    ],
    "chat_template_kwargs": {
      "thinking": false,
      "enable_thinking": false
    }
  }'

If you’re targeting a single model on purpose (e.g. for minimal payloads) and want to use only the key its template actually honors, see the per-model notes on the Models page.

GLM 5.x chain-of-thought leakage. Even with the correct toggle key, GLM 5.x models on the current vLLM build may still leak chain-of-thought into the content field, terminated by a bare </think> marker — see vllm-project/vllm#31319. Clients that require hard-suppressed output should post-process the response: when reasoning is disabled, discard everything in content up to and including the first </think> marker.

Disabling reasoning can significantly reduce token costs for straightforward queries where chain-of-thought isn’t needed. Reasoning tokens always count toward completion_tokens and your billed usage.

Streaming

Enable streaming to receive tokens as they’re generated:

Python
JavaScript

stream = client.chat.completions.create(
    model="moonshotai/kimi-k2.6",
    messages=[
        {"role": "user", "content": "Write a haiku about GPUs."}
    ],
    stream=True,
)

for chunk in stream:
    if chunk.choices[0].delta.content:
        print(chunk.choices[0].delta.content, end="")

const stream = await client.chat.completions.create({
  model: "moonshotai/kimi-k2.6",
  messages: [
    { role: "user", content: "Write a haiku about GPUs." }
  ],
  stream: true,
});

for await (const chunk of stream) {
  process.stdout.write(chunk.choices[0]?.delta?.content || "");
}

Vision

Pass images as URLs or base64 data URIs in the content array:

Python
JavaScript
cURL

response = client.chat.completions.create(
    model="moonshotai/kimi-k2.6",
    messages=[
        {
            "role": "user",
            "content": [
                {"type": "text", "text": "Describe this image."},
                {
                    "type": "image_url",
                    "image_url": {
                        "url": "https://example.com/image.jpg",
                        "detail": "auto"
                    }
                }
            ]
        }
    ],
)

print(response.choices[0].message.content)

const response = await client.chat.completions.create({
  model: "moonshotai/kimi-k2.6",
  messages: [
    {
      role: "user",
      content: [
        { type: "text", text: "Describe this image." },
        {
          type: "image_url",
          image_url: {
            url: "https://example.com/image.jpg",
            detail: "auto",
          },
        },
      ],
    },
  ],
});

console.log(response.choices[0].message.content);

curl https://api.getlilac.com/v1/chat/completions \
  -H "Authorization: Bearer your-lilac-api-key" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "moonshotai/kimi-k2.6",
    "messages": [
      {
        "role": "user",
        "content": [
          {"type": "text", "text": "Describe this image."},
          {"type": "image_url", "image_url": {"url": "https://example.com/image.jpg", "detail": "auto"}}
        ]
      }
    ]
  }'

Tool Calling

response = client.chat.completions.create(
    model="moonshotai/kimi-k2.6",
    messages=[
        {"role": "user", "content": "What's the weather in SF?"}
    ],
    tools=[
        {
            "type": "function",
            "function": {
                "name": "get_weather",
                "description": "Get current weather for a location",
                "parameters": {
                    "type": "object",
                    "properties": {
                        "location": {"type": "string"}
                    },
                    "required": ["location"]
                }
            }
        }
    ],
)

tool_call = response.choices[0].message.tool_calls[0]
print(tool_call.function.name)       # "get_weather"
print(tool_call.function.arguments)  # '{"location": "San Francisco"}'

Structured Output

Force the model to return valid JSON matching a schema:

response = client.chat.completions.create(
    model="moonshotai/kimi-k2.6",
    messages=[
        {"role": "user", "content": "List 3 programming languages and their year of creation."}
    ],
    response_format={
        "type": "json_schema",
        "json_schema": {
            "name": "languages",
            "schema": {
                "type": "object",
                "properties": {
                    "languages": {
                        "type": "array",
                        "items": {
                            "type": "object",
                            "properties": {
                                "name": {"type": "string"},
                                "year": {"type": "integer"}
                            },
                            "required": ["name", "year"]
                        }
                    }
                },
                "required": ["languages"]
            }
        }
    },
)

Response Format

{
  "id": "chatcmpl-abc123",
  "object": "chat.completion",
  "created": 1717000000,
  "model": "moonshotai/kimi-k2.6",
  "choices": [
    {
      "index": 0,
      "message": {
        "role": "assistant",
        "content": "GPU inference is the process of...",
        "reasoning": "The user is asking about...",
        "tool_calls": []
      },
      "finish_reason": "stop"
    }
  ],
  "usage": {
    "prompt_tokens": 25,
    "completion_tokens": 42,
    "total_tokens": 67
  }
}

The reasoning field is present when the model uses chain-of-thought reasoning. It is not counted separately in the response — reasoning tokens are included in completion_tokens.

Getting Started

API Reference

Billing

Chat Completions

Endpoint

Basic Example

Request Parameters

Required

Sampling

Output

Penalties

Log Probabilities

Tool Calling

Structured Output

Reasoning

Per-model reasoning controls

Thinking toggle keys

Streaming

Vision

Tool Calling

Structured Output

Response Format

​Endpoint

​Basic Example

​Request Parameters

​Required

​Sampling

​Output

​Penalties

​Log Probabilities

​Tool Calling

​Structured Output

​Reasoning

​Per-model reasoning controls

​Thinking toggle keys

​Streaming

​Vision

​Tool Calling

​Structured Output

​Response Format

Endpoint

Basic Example

Request Parameters

Required

Sampling

Output

Penalties

Log Probabilities

Tool Calling

Structured Output

Reasoning

Per-model reasoning controls

Thinking toggle keys

Streaming

Vision

Tool Calling

Structured Output

Response Format