LiteLLM Proxy Integration

Overview

LiteLLM Proxy is a popular AI gateway middleware that lets you call multiple API providers using a unified OpenAI-compatible format. This guide covers how to configure AIone as an upstream provider in LiteLLM Proxy.

If you're calling the AIone API directly without LiteLLM Proxy, see Quick Start.

Basic Configuration

Add AIone as a provider in your LiteLLM Proxy config.yaml:

model_list:
  # Claude models
  - model_name: claude-sonnet-4-6
    litellm_params:
      model: openai/claude-sonnet-4-6
      api_base: "https://api.portal.aiin1.ai/v1"
      api_key: "sk-nex-your-key-here"
 
  # GPT models
  - model_name: gpt-5.4
    litellm_params:
      model: openai/gpt-5.4
      api_base: "https://api.portal.aiin1.ai/v1"
      api_key: "sk-nex-your-key-here"
 
  # Gemini text models
  - model_name: gemini-2.5-pro
    litellm_params:
      model: openai/gemini-2.5-pro
      api_base: "https://api.portal.aiin1.ai/v1"
      api_key: "sk-nex-your-key-here"

Key Point: Model Name Prefix

The model field must use the openai/ prefix (e.g., openai/claude-sonnet-4-6) because AIone exposes an OpenAI-compatible endpoint at /v1/chat/completions.

Using anthropic/ or gemini/ prefixes will cause LiteLLM to connect directly to the official Anthropic / Google APIs, bypassing AIone.

Gemini Image Model Configuration

Gemini image models require custom parameters like imageConfig to be passed through. LiteLLM Proxy does not forward non-standard fields by default — you need to use extra_body.

Option 1: Preset Parameters in config.yaml

Best for fixed default parameters:

model_list:
  - model_name: gemini-image
    litellm_params:
      model: openai/gemini-3-pro-image-preview
      api_base: "https://api.portal.aiin1.ai/v1"
      api_key: "sk-nex-your-key-here"
      extra_body:
        aspect_ratio: "1:1"
        image_size: "4K"

Option 2: Dynamic Parameters in Request Body

Best when parameters vary per request. Place custom parameters inside extra_body:

{
  "model": "gemini-image",
  "messages": [
    {"role": "user", "content": "Draw a cat in a spacesuit"}
  ],
  "max_tokens": 4096,
  "extra_body": {
    "image_size": "4K",
    "aspect_ratio": "16:9"
  }
}

You can also use the nested imageConfig format — both are equivalent:

{
  "model": "gemini-image",
  "messages": [
    {"role": "user", "content": "Draw a cat in a spacesuit"}
  ],
  "max_tokens": 4096,
  "extra_body": {
    "imageConfig": {
      "aspect_ratio": "16:9",
      "image_size": "4K"
    }
  }
}

Option 3: Using extra_body in Python SDK

from openai import OpenAI
 
# Connect to your LiteLLM Proxy
client = OpenAI(
    api_key="sk-your-litellm-key",
    base_url="http://localhost:4000/v1",  # LiteLLM Proxy address
)
 
response = client.chat.completions.create(
    model="gemini-image",
    messages=[{"role": "user", "content": "Draw a cat in a spacesuit"}],
    max_tokens=4096,
    extra_body={
        "image_size": "4K",
        "aspect_ratio": "16:9",
    },
)
 
# Text content
print(response.choices[0].message.content)
 
# Image data (if present)
images = getattr(response.choices[0].message, "images", None)
if images:
    for img in images:
        base64_data = img["image_url"]["url"]  # data:image/jpeg;base64,...

Simplified Model Naming

You don't need a separate LiteLLM model entry for each resolution. Configure a single model name and control resolution via request parameters:

model_list:
  # One model name, control resolution with image_size parameter
  - model_name: gemini-image
    litellm_params:
      model: openai/gemini-3-pro-image-preview
      api_base: "https://api.portal.aiin1.ai/v1"
      api_key: "sk-nex-your-key-here"

Specify resolution in the request via extra_body:

{"extra_body": {"image_size": "4K"}}

Priority rule: If you use both a model name with a resolution suffix (e.g., -4k) and the image_size parameter, the parameter takes precedence. The suffix only applies when image_size is not provided.

Reference Image Input

When passing reference images (e.g., for image editing or style transfer), we recommend using base64 data instead of external URLs:

{
  "model": "gemini-image",
  "messages": [
    {
      "role": "user",
      "content": [
        {"type": "text", "text": "Replace the background with a starry sky"},
        {
          "type": "image_url",
          "image_url": {
            "url": "data:image/jpeg;base64,/9j/4AAQ..."
          }
        }
      ]
    }
  ],
  "max_tokens": 4096,
  "extra_body": {"image_size": "2K"}
}

Why base64?

AIone servers (Hong Kong node) may encounter network issues or anti-hotlinking restrictions when downloading images from certain CDNs (e.g., Alibaba Cloud CDN). Base64 embeds the image directly in the request body, avoiding network and CDN policy issues — it's the most reliable method.

Response Format

AIone provides OpenAI-compatible response formatting for Gemini image models:

Text-only Response

{
  "choices": [{
    "message": {
      "role": "assistant",
      "content": "Here is the model's text response"
    }
  }]
}

Response with Images

{
  "choices": [{
    "message": {
      "role": "assistant",
      "content": "Here is your generated image:\n\n![image](data:image/jpeg;base64,/9j/4AAQ...)",
      "images": [
        {
          "type": "image_url",
          "index": 0,
          "image_url": {
            "url": "data:image/jpeg;base64,/9j/4AAQ...",
            "detail": "auto"
          }
        }
      ]
    }
  }]
}

The two fields serve different purposes:

Field	Type	Description
`content`	string	Markdown-formatted text + images, OpenAI spec compliant
`images`	array	Structured image data for programmatic extraction

Important: LiteLLM's openai/ handler operates in pure passthrough mode — it will not automatically extract images from content. If your application needs to programmatically process images, use the images field.

Full Configuration Example

Here's a complete LiteLLM Proxy configuration with multiple model types:

model_list:
  # === Claude ===
  - model_name: claude-opus-4-6
    litellm_params:
      model: openai/claude-opus-4-6
      api_base: "https://api.portal.aiin1.ai/v1"
      api_key: "sk-nex-your-key-here"
 
  - model_name: claude-sonnet-4-6
    litellm_params:
      model: openai/claude-sonnet-4-6
      api_base: "https://api.portal.aiin1.ai/v1"
      api_key: "sk-nex-your-key-here"
 
  # === GPT ===
  - model_name: gpt-5.4
    litellm_params:
      model: openai/gpt-5.4
      api_base: "https://api.portal.aiin1.ai/v1"
      api_key: "sk-nex-your-key-here"
 
  # === Gemini Text ===
  - model_name: gemini-2.5-pro
    litellm_params:
      model: openai/gemini-2.5-pro
      api_base: "https://api.portal.aiin1.ai/v1"
      api_key: "sk-nex-your-key-here"
 
  # === Gemini Image ===
  - model_name: gemini-image
    litellm_params:
      model: openai/gemini-3-pro-image-preview
      api_base: "https://api.portal.aiin1.ai/v1"
      api_key: "sk-nex-your-key-here"
 
  - model_name: gemini-image-flash
    litellm_params:
      model: openai/gemini-3.1-flash-image-preview
      api_base: "https://api.portal.aiin1.ai/v1"
      api_key: "sk-nex-your-key-here"

Troubleshooting

extra_body Parameters Not Taking Effect

Verify you're using the correct passthrough method:

In config.yaml: Add extra_body under litellm_params
In request body: Place parameters inside the top-level extra_body field
In Python SDK: Use the extra_body={} parameter

LiteLLM drops unrecognized top-level fields by default. All non-standard parameters (image_size, aspect_ratio, imageConfig, etc.) must be passed via extra_body.

Image Generation Returns 500

Verify the model field uses the openai/ prefix
Ensure max_tokens is set (recommended: 4096)
4K image generation takes longer (2-3 minutes) — check that your LiteLLM Proxy timeout is sufficient

Where Is the Image Data?

content field: Contains Markdown-formatted images (![image](data:...))
images field: Contains structured base64 image data
LiteLLM's openai/ handler does not auto-extract images — read the images field directly

LiteLLM Proxy Timeout

4K image generation can take 2-3 minutes. Increase the timeout in your LiteLLM Proxy config:

litellm_settings:
  request_timeout: 600  # seconds

We also recommend using "stream": true in your requests — the AIone gateway sends keepalive heartbeats every 10 seconds to prevent intermediate network devices from dropping the connection.

Model Name Not Found

The model name in litellm_params.model must match an AIone-supported model ID
Check the full list via GET https://api.portal.aiin1.ai/v1/models
For complete naming rules, see Model Naming and Compatibility