LiteLLM Proxy Integration

Overview

LiteLLM Proxy is a widely used AI gateway middleware that lets you call models from many providers through a single OpenAI-style interface. This guide shows how to configure AIone as a provider in LiteLLM Proxy.

If you call the AIone API directly rather than through LiteLLM Proxy, see Quick Start.

The examples in this guide are based on LiteLLM 1.101.

Basic configuration

Add AIone as a provider in LiteLLM Proxy's config.yaml:

model_list:
  # Claude models
  - model_name: claude-sonnet-4-6
    litellm_params:
      model: openai/claude-sonnet-4-6
      api_base: "https://api.aiin1.ai/v1"
      api_key: "sk-nex-your-key-here"
 
  # GPT models
  - model_name: gpt-5.4
    litellm_params:
      model: openai/gpt-5.4
      api_base: "https://api.aiin1.ai/v1"
      api_key: "sk-nex-your-key-here"
 
  # Gemini text models
  - model_name: gemini-2.5-pro
    litellm_params:
      model: openai/gemini-2.5-pro
      api_base: "https://api.aiin1.ai/v1"
      api_key: "sk-nex-your-key-here"

Key point: the model prefix and `api_base` go together

LiteLLM uses the prefix of the model field to decide which protocol to speak, and api_base to decide where to send the request. As long as api_base points at AIone, the request never bypasses AIone, whatever the prefix.

Prefix	Protocol LiteLLM sends	Matching `api_base`	Use for
`openai/`	OpenAI Chat (`/chat/completions`)	`https://api.aiin1.ai/v1`	Text models; quick image generation
`gemini/`	Gemini native (`/models/{model}:generateContent`)	`https://api.aiin1.ai/v1beta`	Image models when you need a specific resolution or aspect ratio

Note that the paths differ: openai/ pairs with /v1, gemini/ pairs with /v1beta. LiteLLM appends the rest of the path itself, so gemini/ with /v1 returns 404.

Gemini image model configuration

Resolution and aspect ratio for Gemini image models are passed through the Gemini-native parameter generationConfig.imageConfig, so in LiteLLM use the gemini/ prefix with /v1beta:

model_list:
  - model_name: gemini-image
    litellm_params:
      model: gemini/gemini-3.1-flash-image
      api_base: "https://api.aiin1.ai/v1beta"
      api_key: "sk-nex-your-key-here"

See "Gemini Image Generation" for the available image models; the common choices are gemini-3.1-flash-image (fast) and gemini-3-pro-image (consistent quality).

Setting resolution and aspect ratio

LiteLLM drops top-level fields it does not recognise, so generationConfig must be placed inside extra_body to reach AIone. Three equivalent ways:

Option 1: preset in config.yaml

Best when the resolution is fixed — clients need no changes:

model_list:
  - model_name: gemini-image-2k
    litellm_params:
      model: gemini/gemini-3.1-flash-image
      api_base: "https://api.aiin1.ai/v1beta"
      api_key: "sk-nex-your-key-here"
      extra_body:
        generationConfig:
          responseModalities: ["IMAGE"]
          imageConfig:
            imageSize: "2K"
            aspectRatio: "16:9"

Option 2: pass it per request

Best when the resolution varies per request:

{
  "model": "gemini-image",
  "messages": [
    {"role": "user", "content": "Draw a cat in a spacesuit"}
  ],
  "extra_body": {
    "generationConfig": {
      "responseModalities": ["IMAGE"],
      "imageConfig": {"imageSize": "4K", "aspectRatio": "16:9"}
    }
  }
}

Option 3: `extra_body` in the Python SDK

from openai import OpenAI
 
# Connect to your LiteLLM Proxy
client = OpenAI(
    api_key="sk-your-litellm-key",
    base_url="http://localhost:4000/v1",  # LiteLLM Proxy address
)
 
response = client.chat.completions.create(
    model="gemini-image",
    messages=[{"role": "user", "content": "Draw a cat in a spacesuit"}],
    extra_body={
        "generationConfig": {
            "responseModalities": ["IMAGE"],
            "imageConfig": {"imageSize": "2K", "aspectRatio": "16:9"},
        }
    },
)
 
for img in response.choices[0].message.images:
    data_url = img["image_url"]["url"]  # data:image/png;base64,...

imageSize accepts 512 / 1K / 2K / 4K; aspectRatio supports 14 values. See "Gemini Image Generation" for the full list and the actual pixel dimensions.

Shorthand: when you only need an image and do not care about size, the OpenAI-style "modalities": ["image"] can replace responseModalities; LiteLLM converts it automatically.

Reference image input

Pass the reference image with the standard OpenAI multimodal messages.content; LiteLLM converts it to the Gemini-native form:

{
  "model": "gemini-image",
  "messages": [
    {
      "role": "user",
      "content": [
        {"type": "text", "text": "Replace the background of this picture with a starry sky"},
        {
          "type": "image_url",
          "image_url": {"url": "data:image/png;base64,iVBORw0KGgoAAA..."}
        }
      ]
    }
  ],
  "modalities": ["image"]
}

Send base64 data directly rather than an external URL: some CDNs (for example Alibaba Cloud CDN) apply hotlink protection or format conversion, so a direct link may not be retrievable. base64 is embedded in the request body and is unaffected by network or CDN policy.

Reference images count toward prompt_tokens.

Response format

When called through the gemini/ prefix, LiteLLM puts the image in message.images[] and sets message.content to null:

{
  "choices": [{
    "message": {
      "role": "assistant",
      "content": null,
      "images": [
        {
          "type": "image_url",
          "index": 0,
          "image_url": {"url": "data:image/png;base64,iVBORw0KGgoAAA..."}
        }
      ]
    },
    "finish_reason": "stop"
  }],
  "usage": {
    "prompt_tokens": 2,
    "completion_tokens": 1120,
    "total_tokens": 1122,
    "completion_tokens_details": {"text_tokens": 0, "image_tokens": 1120}
  }
}

For programmatic handling read message.images directly; do not parse content. usage.completion_tokens_details.image_tokens is the image output token count and the basis for billing.

Full configuration example

A complete LiteLLM Proxy configuration covering several model families:

model_list:
  # === Claude ===
  - model_name: claude-opus-4-6
    litellm_params:
      model: openai/claude-opus-4-6
      api_base: "https://api.aiin1.ai/v1"
      api_key: "sk-nex-your-key-here"
 
  - model_name: claude-sonnet-4-6
    litellm_params:
      model: openai/claude-sonnet-4-6
      api_base: "https://api.aiin1.ai/v1"
      api_key: "sk-nex-your-key-here"
 
  # === GPT ===
  - model_name: gpt-5.4
    litellm_params:
      model: openai/gpt-5.4
      api_base: "https://api.aiin1.ai/v1"
      api_key: "sk-nex-your-key-here"
 
  # === Gemini text ===
  - model_name: gemini-2.5-pro
    litellm_params:
      model: openai/gemini-2.5-pro
      api_base: "https://api.aiin1.ai/v1"
      api_key: "sk-nex-your-key-here"
 
  # === Gemini image (note: gemini/ prefix + /v1beta) ===
  - model_name: gemini-image
    litellm_params:
      model: gemini/gemini-3.1-flash-image
      api_base: "https://api.aiin1.ai/v1beta"
      api_key: "sk-nex-your-key-here"
 
  - model_name: gemini-image-pro
    litellm_params:
      model: gemini/gemini-3-pro-image
      api_base: "https://api.aiin1.ai/v1beta"
      api_key: "sk-nex-your-key-here"
 
  # A dedicated entry preset to 4K ultra-wide
  - model_name: gemini-image-4k-wide
    litellm_params:
      model: gemini/gemini-3.1-flash-image
      api_base: "https://api.aiin1.ai/v1beta"
      api_key: "sk-nex-your-key-here"
      extra_body:
        generationConfig:
          responseModalities: ["IMAGE"]
          imageConfig:
            imageSize: "4K"
            aspectRatio: "21:9"

FAQ

Image model returns 404 Not Found

With the gemini/ prefix, api_base must be https://api.aiin1.ai/v1beta. Using /v1 or just the domain returns 404 — LiteLLM appends /models/{model}:generateContent to api_base, so the path does not match.

`generationConfig` has no effect; images always come back at the default size

LiteLLM drops top-level fields it does not recognise. generationConfig must be inside extra_body:

In config.yaml: litellm_params.extra_body.generationConfig
In the request body: top-level extra_body.generationConfig
Python SDK: extra_body={"generationConfig": {...}}

Where is the image data

With the gemini/ prefix the image is in message.images[] and message.content is null. Iterate images and read image_url.url (a data URI).

LiteLLM Proxy timeouts

High-resolution generation takes longer. Raise the timeout in the LiteLLM Proxy configuration:

litellm_settings:
  request_timeout: 600  # seconds

Model not found

The part after the prefix in litellm_params.model must be a model ID that AIone supports; see GET https://api.aiin1.ai/v1/models for the full list
For the full naming rules, see Model Naming and Compatibility