> ## Documentation Index
> Fetch the complete documentation index at: https://docs.getlilac.com/llms.txt
> Use this file to discover all available pages before exploring further.

# Chat Completions

> Use the OpenAI-compatible chat completions endpoint to generate model responses from conversation history, with streaming and tool calling.

The chat completions endpoint is fully compatible with the OpenAI chat completions API. Lilac serves models via a customized fork of [vLLM](https://docs.vllm.ai/) tuned for idle-GPU scheduling and shared warm endpoints, so you get access to both standard OpenAI parameters and vLLM-specific extras.

## Endpoint

```
POST https://api.getlilac.com/v1/chat/completions
```

## Basic Example

<Tabs>
  <Tab title="Python">
    ```python theme={null}
    from openai import OpenAI

    client = OpenAI(
        base_url="https://api.getlilac.com/v1",
        api_key="your-lilac-api-key",
    )

    response = client.chat.completions.create(
        model="moonshotai/kimi-k2.6",
        messages=[
            {"role": "system", "content": "You are a helpful assistant."},
            {"role": "user", "content": "What is GPU inference?"},
        ],
    )

    print(response.choices[0].message.content)
    ```
  </Tab>

  <Tab title="JavaScript">
    ```javascript theme={null}
    import OpenAI from "openai";

    const client = new OpenAI({
      baseURL: "https://api.getlilac.com/v1",
      apiKey: "your-lilac-api-key",
    });

    const response = await client.chat.completions.create({
      model: "moonshotai/kimi-k2.6",
      messages: [
        { role: "system", content: "You are a helpful assistant." },
        { role: "user", content: "What is GPU inference?" },
      ],
    });

    console.log(response.choices[0].message.content);
    ```
  </Tab>

  <Tab title="cURL">
    ```bash theme={null}
    curl https://api.getlilac.com/v1/chat/completions \
      -H "Authorization: Bearer your-lilac-api-key" \
      -H "Content-Type: application/json" \
      -d '{
        "model": "moonshotai/kimi-k2.6",
        "messages": [
          {"role": "system", "content": "You are a helpful assistant."},
          {"role": "user", "content": "What is GPU inference?"}
        ]
      }'
    ```
  </Tab>
</Tabs>

## Request Parameters

### Required

| Parameter  | Type     | Description                                                                                            |
| ---------- | -------- | ------------------------------------------------------------------------------------------------------ |
| `model`    | `string` | Model ID (e.g., `moonshotai/kimi-k2.6`). See [Models](/inference/models).                              |
| `messages` | `array`  | Conversation history. Each message has a `role` (`system`, `user`, `assistant`, `tool`) and `content`. |

### Sampling

| Parameter     | Type      | Default | Description                                                                 |
| ------------- | --------- | ------- | --------------------------------------------------------------------------- |
| `temperature` | `float`   | `1.0`   | Sampling temperature (0–2). Lower values are more deterministic.            |
| `top_p`       | `float`   | `1.0`   | Nucleus sampling — considers tokens with cumulative probability >= `top_p`. |
| `top_k`       | `integer` | `-1`    | Limits sampling to the top K tokens. `-1` disables.                         |
| `min_p`       | `float`   | `0.0`   | Minimum relative probability threshold for token consideration.             |
| `seed`        | `integer` | `null`  | Seed for deterministic sampling (best effort).                              |

### Output

| Parameter               | Type                | Default         | Description                                                               |
| ----------------------- | ------------------- | --------------- | ------------------------------------------------------------------------- |
| `max_tokens`            | `integer`           | model-dependent | Maximum tokens to generate.                                               |
| `max_completion_tokens` | `integer`           | `null`          | Upper bound including reasoning tokens. Preferred for reasoning models.   |
| `n`                     | `integer`           | `1`             | Number of completions to generate.                                        |
| `stop`                  | `string` or `array` | `null`          | Up to 4 sequences where generation stops.                                 |
| `stream`                | `boolean`           | `false`         | Stream partial token deltas via SSE.                                      |
| `stream_options`        | `object`            | `null`          | Options like `{"include_usage": true}` to get token counts in the stream. |

### Penalties

| Parameter            | Type     | Default | Description                                                    |
| -------------------- | -------- | ------- | -------------------------------------------------------------- |
| `frequency_penalty`  | `float`  | `0.0`   | Penalizes tokens by frequency in output so far (-2.0 to 2.0).  |
| `presence_penalty`   | `float`  | `0.0`   | Penalizes tokens that have appeared at all (-2.0 to 2.0).      |
| `repetition_penalty` | `float`  | `1.0`   | Multiplicative penalty on repeated tokens. `1.0` = no penalty. |
| `logit_bias`         | `object` | `null`  | Map of token ID → bias value (-100 to 100).                    |

### Log Probabilities

| Parameter      | Type      | Default | Description                                                                                |
| -------------- | --------- | ------- | ------------------------------------------------------------------------------------------ |
| `logprobs`     | `boolean` | `false` | Return log probabilities of output tokens.                                                 |
| `top_logprobs` | `integer` | `null`  | Number of most likely tokens to return at each position (0–20). Requires `logprobs: true`. |

### Tool Calling

| Parameter     | Type                 | Default  | Description                                                                               |
| ------------- | -------------------- | -------- | ----------------------------------------------------------------------------------------- |
| `tools`       | `array`              | `null`   | List of tool definitions with `type: "function"` and a function schema.                   |
| `tool_choice` | `string` or `object` | `"auto"` | `"none"`, `"auto"`, `"required"`, or `{"type": "function", "function": {"name": "..."}}`. |

### Structured Output

| Parameter         | Type     | Default | Description                                                                                        |
| ----------------- | -------- | ------- | -------------------------------------------------------------------------------------------------- |
| `response_format` | `object` | `null`  | `{"type": "text"}`, `{"type": "json_object"}`, or `{"type": "json_schema", "json_schema": {...}}`. |

### Reasoning

| Parameter              | Type     | Default | Description                                                                                                            |
| ---------------------- | -------- | ------- | ---------------------------------------------------------------------------------------------------------------------- |
| `chat_template_kwargs` | `object` | `null`  | Toggles model behavior exposed by the chat template. Used to enable or disable chain-of-thought reasoning — see below. |

Some models (like Kimi K2.6 and GLM 5.2) include chain-of-thought reasoning by default. When reasoning is active, the model's chain-of-thought is returned in a separate `reasoning` field on the response message. Reasoning tokens are included in `completion_tokens` and count toward your usage — including all reasoning controls below.

These are **model-specific chat-template controls**, not universal OpenAI parameters. The exact key and accepted values depend on each model's chat template.

#### Per-model reasoning controls

| Model      | Control                                                       | Values                                | Default    |
| ---------- | ------------------------------------------------------------- | ------------------------------------- | ---------- |
| Kimi K2.6  | `chat_template_kwargs.thinking`                               | `true` \| `false`                     | on         |
| GLM 5.2    | `reasoning_effort` or `chat_template_kwargs.reasoning_effort` | `max` \| `high`                       | `high`     |
| GLM 5.2    | `chat_template_kwargs.enable_thinking`                        | `true` \| `false`                     | on         |
| Gemma 4    | `chat_template_kwargs.enable_thinking`                        | `true` \| `false`                     | off        |
| MiniMax M3 | `chat_template_kwargs.thinking_mode`                          | `adaptive` \| `enabled` \| `disabled` | `adaptive` |

Notes:

* **GLM 5.2** supports `reasoning_effort` with two levels — `high` (default) and `max`. `max` is the opt-in highest-quality mode for long-horizon agentic and complex problem-solving tasks, at the cost of higher latency and token usage. You can send `reasoning_effort` either as a top-level field (OpenAI-style) or inside `chat_template_kwargs`. Disable thinking entirely with `chat_template_kwargs.enable_thinking: false`; when thinking is disabled, `reasoning_effort` has no effect.
* **MiniMax M3** does not use a boolean toggle. It accepts `thinking_mode` with three values: `adaptive` (default — the model decides), `enabled` (always think), and `disabled` (never think).
* Defaults differ per model: Kimi K2.6 and GLM 5.2 have reasoning **on** by default; Gemma 4 has reasoning **off** by default; MiniMax M3 is **adaptive** by default. See the [Models](/inference/models) page for per-model details.

#### Thinking toggle keys

The key that toggles reasoning is defined by each model's chat template, not by the API, so it differs per model family:

| Key                                                     | Models that honor it                                                     |
| ------------------------------------------------------- | ------------------------------------------------------------------------ |
| `thinking` (bool)                                       | Moonshot chat templates — e.g. Kimi K2.6                                 |
| `enable_thinking` (bool)                                | Z.ai GLM / Google Gemma / SGLang-style templates — e.g. GLM 5.x, Gemma 4 |
| `reasoning_effort` (`max` \| `high`)                    | GLM 5.2 — also accepted as a top-level field                             |
| `thinking_mode` (`adaptive` \| `enabled` \| `disabled`) | MiniMax M3                                                               |

Unknown keys inside `chat_template_kwargs` are silently ignored by chat templates, so **the safe, forward-compatible approach is to send both keys**. This works across all current Lilac models and any future model whose template uses either convention:

```json theme={null}
{ "chat_template_kwargs": { "thinking": false, "enable_thinking": false } }
```

<Tabs>
  <Tab title="Python">
    ```python theme={null}
    # Disable reasoning (works for both Moonshot- and GLM-style templates)
    response = client.chat.completions.create(
        model="moonshotai/kimi-k2.6",
        messages=[{"role": "user", "content": "What is 2+2?"}],
        extra_body={
            "chat_template_kwargs": {
                "thinking": False,
                "enable_thinking": False,
            }
        },
    )
    ```
  </Tab>

  <Tab title="JavaScript">
    ```javascript theme={null}
    const response = await client.chat.completions.create({
      model: "moonshotai/kimi-k2.6",
      messages: [
        { role: "user", content: "What is 2+2?" }
      ],
      chat_template_kwargs: {
        thinking: false,
        enable_thinking: false,
      },
    });
    ```
  </Tab>

  <Tab title="cURL">
    ```bash theme={null}
    curl https://api.getlilac.com/v1/chat/completions \
      -H "Authorization: Bearer your-lilac-api-key" \
      -H "Content-Type: application/json" \
      -d '{
        "model": "moonshotai/kimi-k2.6",
        "messages": [
          {"role": "user", "content": "What is 2+2?"}
        ],
        "chat_template_kwargs": {
          "thinking": false,
          "enable_thinking": false
        }
      }'
    ```
  </Tab>
</Tabs>

If you're targeting a single model on purpose (e.g. for minimal payloads) and want to use only the key its template actually honors, see the per-model notes on the [Models](/inference/models) page.

<Warning>
  **GLM 5.x chain-of-thought leakage.** Even with the correct toggle key, GLM 5.x models on the current vLLM build may still leak chain-of-thought into the `content` field, terminated by a bare `</think>` marker — see [vllm-project/vllm#31319](https://github.com/vllm-project/vllm/issues/31319). Clients that require hard-suppressed output should post-process the response: when reasoning is disabled, discard everything in `content` up to and including the first `</think>` marker.
</Warning>

<Tip>
  Disabling reasoning can significantly reduce token costs for straightforward queries where chain-of-thought isn't needed. Reasoning tokens always count toward `completion_tokens` and your billed usage.
</Tip>

## Streaming

Enable streaming to receive tokens as they're generated:

<Tabs>
  <Tab title="Python">
    ```python theme={null}
    stream = client.chat.completions.create(
        model="moonshotai/kimi-k2.6",
        messages=[
            {"role": "user", "content": "Write a haiku about GPUs."}
        ],
        stream=True,
    )

    for chunk in stream:
        if chunk.choices[0].delta.content:
            print(chunk.choices[0].delta.content, end="")
    ```
  </Tab>

  <Tab title="JavaScript">
    ```javascript theme={null}
    const stream = await client.chat.completions.create({
      model: "moonshotai/kimi-k2.6",
      messages: [
        { role: "user", content: "Write a haiku about GPUs." }
      ],
      stream: true,
    });

    for await (const chunk of stream) {
      process.stdout.write(chunk.choices[0]?.delta?.content || "");
    }
    ```
  </Tab>
</Tabs>

## Vision

Pass images as URLs or base64 data URIs in the `content` array:

<Tabs>
  <Tab title="Python">
    ```python theme={null}
    response = client.chat.completions.create(
        model="moonshotai/kimi-k2.6",
        messages=[
            {
                "role": "user",
                "content": [
                    {"type": "text", "text": "Describe this image."},
                    {
                        "type": "image_url",
                        "image_url": {
                            "url": "https://example.com/image.jpg",
                            "detail": "auto"
                        }
                    }
                ]
            }
        ],
    )

    print(response.choices[0].message.content)
    ```
  </Tab>

  <Tab title="JavaScript">
    ```javascript theme={null}
    const response = await client.chat.completions.create({
      model: "moonshotai/kimi-k2.6",
      messages: [
        {
          role: "user",
          content: [
            { type: "text", text: "Describe this image." },
            {
              type: "image_url",
              image_url: {
                url: "https://example.com/image.jpg",
                detail: "auto",
              },
            },
          ],
        },
      ],
    });

    console.log(response.choices[0].message.content);
    ```
  </Tab>

  <Tab title="cURL">
    ```bash theme={null}
    curl https://api.getlilac.com/v1/chat/completions \
      -H "Authorization: Bearer your-lilac-api-key" \
      -H "Content-Type: application/json" \
      -d '{
        "model": "moonshotai/kimi-k2.6",
        "messages": [
          {
            "role": "user",
            "content": [
              {"type": "text", "text": "Describe this image."},
              {"type": "image_url", "image_url": {"url": "https://example.com/image.jpg", "detail": "auto"}}
            ]
          }
        ]
      }'
    ```
  </Tab>
</Tabs>

## Tool Calling

```python theme={null}
response = client.chat.completions.create(
    model="moonshotai/kimi-k2.6",
    messages=[
        {"role": "user", "content": "What's the weather in SF?"}
    ],
    tools=[
        {
            "type": "function",
            "function": {
                "name": "get_weather",
                "description": "Get current weather for a location",
                "parameters": {
                    "type": "object",
                    "properties": {
                        "location": {"type": "string"}
                    },
                    "required": ["location"]
                }
            }
        }
    ],
)

tool_call = response.choices[0].message.tool_calls[0]
print(tool_call.function.name)       # "get_weather"
print(tool_call.function.arguments)  # '{"location": "San Francisco"}'
```

## Structured Output

Force the model to return valid JSON matching a schema:

```python theme={null}
response = client.chat.completions.create(
    model="moonshotai/kimi-k2.6",
    messages=[
        {"role": "user", "content": "List 3 programming languages and their year of creation."}
    ],
    response_format={
        "type": "json_schema",
        "json_schema": {
            "name": "languages",
            "schema": {
                "type": "object",
                "properties": {
                    "languages": {
                        "type": "array",
                        "items": {
                            "type": "object",
                            "properties": {
                                "name": {"type": "string"},
                                "year": {"type": "integer"}
                            },
                            "required": ["name", "year"]
                        }
                    }
                },
                "required": ["languages"]
            }
        }
    },
)
```

## Response Format

```json theme={null}
{
  "id": "chatcmpl-abc123",
  "object": "chat.completion",
  "created": 1717000000,
  "model": "moonshotai/kimi-k2.6",
  "choices": [
    {
      "index": 0,
      "message": {
        "role": "assistant",
        "content": "GPU inference is the process of...",
        "reasoning": "The user is asking about...",
        "tool_calls": []
      },
      "finish_reason": "stop"
    }
  ],
  "usage": {
    "prompt_tokens": 25,
    "completion_tokens": 42,
    "total_tokens": 67
  }
}
```

<Note>
  The `reasoning` field is present when the model uses chain-of-thought reasoning. It is not counted separately in the response — reasoning tokens are included in `completion_tokens`.
</Note>