Integrate multiple AI models through one documented API

Welcome to the All AI Ask Public API. This API allows you to programmatically compare multiple LLMs and evaluate their outputs using AI judges.

Endpoint

POST https://api.allaiask.com/api/v1/prompt

Authentication

Include your API key in the request headers: Authorization: Bearer <YOUR_API_KEY>

---

Request Body

Field	Type	Required	Description
prompt	string	Yes	The prompt text to send to the models.
models	string[]	Yes	Array of model IDs to generate responses from.
judges	string[]	No	Array of model IDs to use as judges for comparing the outputs.
judge_prompt	string	No	Custom grading instructions for judges (e.g. "Focus on technical accuracy").
judge_weights	object	No	Weights for the judge (accuracy, speed, cost). Default: 1, 0, 0.
attachments	object[]	No	Array of base64 encoded files (images/PDFs depending on model support).
system_prompt	string	No	Custom instructions to set model behavior/persona.
generation_settings	object	No	Optional parameters like temperature, maxTokens, topP.
response_schema	object	No	Optional JSON schema to force a shared structured output across all models.
model_tuning	object	No	Per-model fine-tuning options (thinking budget, reasoning effort). See below.
zero_data_retention	boolean	No	Prevent prompt text, system prompt, model outputs, and judge reasoning/metadata from being logged persistently in database records. Default: false (or user profile preference).

Attachments Object

{
  "name": "image.png",
  "type": "image/png",
  "base64": "..."
}

Judging Weights Object

Balance the priorities for the judge evaluation. All values are 0.0 to 1.0.

{
  "accuracy": 1.0,
  "speed": 0.5,
  "cost": 0.2
}

Generation Settings Object

Fine-tune how models generate responses.

{
  "temperature": 0.7,    // 0.0 to 2.0 (Lower = deterministic, Higher = creative)
  "maxTokens": 1024,     // Limit the length of the generated response
  "topP": 1.0            // Nucleus sampling threshold
}

Structured Outputs (Response Schema)

Force all models to output valid JSON matching a specific schema. Native schema enforcement is used for OpenAI, Gemini, Groq (select models), and Amazon Bedrock/Nova (via Converse forced tool use); other providers receive a strengthened prompt instruction. Using a shared schema allows for consistent comparison and automated evaluation.

{
  "type": "object",
  "properties": {
    "summary": { "type": "string" },
    "confidence": { "type": "number" }
  },
  "required": ["summary", "confidence"],
  "additionalProperties": false
}

[!IMPORTANT] To ensure compatibility with OpenAI's strict mode, always include "additionalProperties": false in your schema objects.

Model Tuning Object

Pass per-model fine-tuning parameters keyed by model ID. This object must be assigned to the "model_tuning" field in your request payload. Only set values you want to override — omitted models use defaults.

{
  "model_tuning": {
    "claude-sonnet-4-6": { "thinkingBudget": 8000 },
    "gemini-3.5-flash":   { "thinkingBudget": 8192, "webSearch": true },
    "o3-mini":            { "reasoningEffort": "high" },
    "grok-4.3":           { "reasoningEffort": "medium" }
  }
}

Model	Parameter	Type	Description
claude-sonnet-4-6, claude-opus-4-7, claude-opus-4-6	thinkingBudget	integer	Extended thinking token budget (1024–32000). Set 0 to disable.
gemini-3.5-flash, gemini-3.1-pro, gemini-3.1-flash, gemini-3.1-flash-lite	thinkingBudget	integer	Gemini thinking token budget (0–24576). Set 0 to disable.
gemini-3.5-flash, gemini-3.1-pro, gemini-3.1-flash, gemini-3.1-flash-lite	webSearch	boolean	Enable Google Search grounding (web search).
o3-mini, o3, o4-mini, gpt-5.4-pro, gpt-5.4, gpt-5.4-mini, gpt-5.4-nano, gpt-5, gpt-5-mini, gpt-5-nano	reasoningEffort	string	"low", "medium", or "high". Controls reasoning compute spend.
grok-4.20-0309-reasoning, grok-4.3	reasoningEffort	string	"none", "low", "medium", or "high". Controls reasoning compute spend.
cerebras-gpt-oss-120b, cerebras-glm-4.7	reasoningEffort	string	"low", "medium", or "high". Controls reasoning compute spend on Cerebras.

[!NOTE] Amazon Nova models (nova-pro, nova-lite, nova-micro) and Mistral models do not expose reasoning/thinking tuning parameters — omit them from model_tuning.

---

Example Request

curl -X POST https://api.allaiask.com/api/v1/prompt \
  -H "Authorization: Bearer <YOUR_API_KEY>" \
  -H "Content-Type: application/json" \
  -d '{
    "prompt": "What are the benefits of side-by-side LLM comparison?",
    "models": ["gpt-4.1", "claude-sonnet-4-6", "gemini-3.1-pro"],
    "judges": ["o3-mini"],
    "judge_weights": { "accuracy": 1, "speed": 0.5 },
    "judge_prompt": "Evaluate the clarity and technical depth of the explanations.",
    "zero_data_retention": true,
    "model_tuning": {
      "claude-sonnet-4-6": { "thinkingBudget": 4000 },
      "gemini-3.1-pro": { "webSearch": true }
    },
    "generation_settings": {
      "temperature": 0.7,
      "maxTokens": 1024
    },
    "response_schema": {
      "type": "object",
      "properties": {
        "answer": { "type": "string" }
      }
    }
  }'

---

Supported Models

You can use any of these IDs in the models array.

[!IMPORTANT] Models are routed by modelId, not by a separate provider field. Each modelId is globally unique and the gateway maps it to the correct provider internally. You never specify a provider — just the ID.

The same underlying model served by more than one provider has a distinct modelId. The provider is encoded in the ID (usually as a prefix), so there is never a collision. For example, OpenAI's gpt-oss-120b open-weight model is available on two providers:

modelId	Provider	Notes
gpt-oss-120b	Groq	Low time-to-first-token (~60–120ms)
cerebras-gpt-oss-120b	Cerebras	~2–3× higher throughput on long outputs

To compare the same model across providers, simply list both IDs: "models": ["gpt-oss-120b", "cerebras-gpt-oss-120b"]. They run as independent entries and appear as separate results in the response.

Reasoning & Flagship Models

The most capable models for complex problem solving, reasoning, and advanced logic.

• gpt-5.4 (OpenAI)
• gpt-5.4-pro (OpenAI)
• gpt-5 (OpenAI)
• gpt-4.1 (OpenAI)
• gpt-4o (OpenAI)
• o3-mini (OpenAI)
• grok-4.20-0309-reasoning (xAI)
• claude-opus-4-6 (Anthropic)
• claude-sonnet-4-6 (Anthropic)
• gemini-3.1-pro (Google)
• gpt-oss-120b (Groq/OpenAI)
• cerebras-gpt-oss-120b (Cerebras/OpenAI)
• cerebras-glm-4.7 (Cerebras/Z.ai)
• deepseek-v4-pro (DeepSeek)
• mistral-large (Mistral)
• nova-pro (Amazon Bedrock)

Fast & Efficient Models

Optimized for high-speed, cost-effective multimodal chat and generation.

• gpt-5.4-mini (OpenAI)
• gpt-5.4-nano (OpenAI)
• gpt-5-mini (OpenAI)
• gpt-5-nano (OpenAI)
• grok-4.20-0309-non-reasoning (xAI)
• grok-4.3 (xAI)
• claude-haiku-4-5 (Anthropic)
• gemini-3.5-flash (Google)
• gemini-3.1-flash-lite (Google)
• llama-4-scout (Groq)
• gpt-oss-20b (Groq/OpenAI)
• qwen3-32b (Groq)
• deepseek-v4-flash (DeepSeek)
• mistral-medium (Mistral)
• mistral-small (Mistral)
• ministral-8b (Mistral) — cheapest flat-rate option
• codestral (Mistral) — code-specialised
• nova-lite (Amazon Bedrock)
• nova-micro (Amazon Bedrock) — cheapest, fastest budget model

Legacy / Dated Models

These models are still fully supported by the API but have been superseded by newer generations.

• gpt-4o-mini (OpenAI)
• gpt-4-turbo (OpenAI)
• claude-sonnet-4.5 (Anthropic)
• claude-opus-4.5 (Anthropic)
• claude-opus-4.1 (Anthropic)
• claude-sonnet-4 (Anthropic)
• claude-opus-4 (Anthropic)
• gemini-3.1-flash (Google)
• gemini-2.5-flash (Google)
• gemini-2.5-flash-lite (Google)
• llama-3.3-70b (Groq)
• llama-3.1-8b (Groq)
• grok-3 (xAI)
• grok-3-mini (xAI)

---

Supported Judges

While most models can technically judge, we recommend using these IDs in the judges array for the most reliable evaluations:

• Primary Choice: gpt-5.4-pro or gpt-4.1
• Alternative: claude-opus-4-6 or gemini-3.1-pro
• Specialized Reasoning: grok-4.20-0309-reasoning or deepseek-v4-pro

Note: Judging involve a "position swap" test (running the evaluation twice) to ensure consistency and eliminate bias. When judges are requested in the API, the response will wait for these evaluations to complete before returning.

---

Response Body

Field	Type	Description
promptLogId	string	Unique identifier for the request.
results	object[]	Array of outputs from each requested model.
evaluations	object[]	Array of judging results (pairwise comparisons).
creditsUsed	number	Total credits deducted from your balance for this request.

Result Object

{
  "modelId": "gpt-4o-mini",
  "status": "completed",
  "output": "The response text...",
  "tokensIn": 150,
  "tokensOut": 320,
  "latencyMs": 850,
  "providerCost": 0.00015 // Raw cost from the provider
}

---

Context & Prompt Caching

To optimize performance and reduce API usage costs, several providers supported by the All AI Ask platform offer context/prompt caching. This allows you to cache large, repetitive inputs (such as system instructions, file contents, or extensive chat history).

1. Automatic Caching

For the following providers, caching is fully automatic at the infrastructure level. You do not need to make any API request changes; the platform will automatically reuse cache segments when matching prefixes are sent:

• OpenAI: Automatically caches identical prompt prefixes $\ge$ 1,024 tokens. Match segments receive a 50% discount on input tokens.
• DeepSeek: Automatically caches identical prompt prefixes in multiples of 64 tokens. Matches receive a 90% discount on input tokens.
• Mistral: Automatically caches identical prompt prefixes at their gateway.

2. Explicit Caching: Anthropic (Claude)

Anthropic allows explicit prompt caching for large system instructions, history, or attachments.

• Requirement: Minimum prompt length of 1,024 tokens (Sonnet/Opus) or 2,048 tokens (Haiku). Up to 4 breakpoints can be declared.
• Implementation: Specify "cache_control": { "type": "ephemeral" } within the system_prompt array or on specific history elements.

Example Request (Anthropic Caching)

{
  "prompt": "Evaluate the new code edits.",
  "models": ["claude-sonnet-4-6"],
  "system_prompt": [
    {
      "type": "text",
      "text": "You are a senior code reviewer with 20+ years of experience...",
      "cache_control": { "type": "ephemeral" }
    }
  ],
  "history": [
    {
      "role": "user",
      "content": "Here is the codebase: [insert massive codebase files...]",
      "cache_control": { "type": "ephemeral" }
    },
    {
      "role": "assistant",
      "content": "Understood. I have reviewed the codebase structure."
    }
  ]
}

3. Explicit Caching: Google (Gemini)

Gemini supports explicit context caching for extremely large files, videos, or prompt chains.

• Requirement: Minimum of 32,768 tokens (often equivalent to ~1 hour of audio/video or a massive codebase).
• Implementation:
  1. Create a cache resource directly via Google's Gemini API /cachedContents endpoint using your API credentials.
  2. Pass the returned cache name string (e.g., cachedContents/1234abcd) in the "model_tuning" field under "cachedContent" for your Gemini model.

Example Request (Gemini Caching)

{
  "prompt": "Summarize key findings from this video.",
  "models": ["gemini-3.1-pro"],
  "model_tuning": {
    "gemini-3.1-pro": {
      "cachedContent": "cachedContents/1234567890abcdef"
    }
  }
}

---

Error Codes

• 400: Missing prompt or invalid model IDs.
• 401: Invalid or missing API Key.
• 402: Insufficient Balance.
  • Response includes estimatedCreditsRequired.
• 429: Rate limit exceeded.