LLM providers

Two options decide which models your deployment runs on. The providers option holds the accounts and endpoints that CKEditor AI On-Premises calls. The models option holds the models the service offers to end users and says which feature each model serves.

Set both. The service does not start until every conversation, review, and action feature has a model.

This is the configuration behind Model providers. Read that page first if you are still deciding whether to run CKEditor AI on your own models. For secrets, databases, and storage, see Required configuration.

Providers

The providers option is required. The service does not start without at least one provider, and a model that names a provider you did not define stops startup too.

The service supports the following LLM providers:

OpenAI
Anthropic
Google (Gemini models)
Azure OpenAI
Amazon Bedrock
Google Vertex AI (Gemini and Anthropic models)
Custom providers compatible with the OpenAI Chat Completions API

google and vertex are separate provider types. google calls the Gemini API with an API key, and vertex calls Vertex AI in your Google Cloud project.

Declare each provider in the providers option. You choose the key, and a model names that key in its own provider option.

{
	"providers": {
		"anthropic": {
			"type": "anthropic",
			"name": "Anthropic",
			"apiKeys": ["api_key1", "api_key2", "api_key3"]
		},
		"google": {
			"type": "google",
			"name": "Google",
			"apiKeys": ["api_key1", "api_key2", "api_key3"]
		},
		"openai": {
			"type": "openai",
			"name": "OpenAI",
			"apiKeys": ["api_key1", "api_key2", "api_key3"]
		},
		"azure": {
			"type": "azure",
			"name": "Azure",
			"resourceName": "azure-resource-name",
			"apiKeys": ["api_key1", "api_key2", "api_key3"]
		},
		"bedrock": {
			"type": "bedrock",
			"name": "Bedrock",
			"region": "us-east-1",
			"apiKeys": ["api_key1", "api_key2", "api_key3"]
		},
		"vertex": {
			"type": "vertex",
			"name": "Vertex",
			"project": "google-project-id",
			"location": "global",
			"apiKeys": ["api_key1", "api_key2", "api_key3"]
		},
		"your-custom-provider": {
			"type": "openai-compatible",
			"name": "Custom provider",
			"baseUrl": "https://your-custom-provider.com",
			"headers": {
				"Authorization": "Bearer token",
				"X-Custom-Header": "custom_value"
			}
		}
	}
}

The value should be provided as a stringified JSON object:

PROVIDERS='{
	"anthropic": {
		"type": "anthropic",
		"name": "Anthropic",
		"apiKeys": ["api_key1", "api_key2", "api_key3"]
	},
	"google": {
		"type": "google",
		"name": "Google",
		"apiKeys": ["api_key1", "api_key2", "api_key3"]
	},
	"openai": {
		"type": "openai",
		"name": "OpenAI",
		"apiKeys": ["api_key1", "api_key2", "api_key3"]
	},
	"azure": {
		"type": "azure",
		"name": "Azure",
		"resourceName": "azure-resource-name",
		"apiKeys": ["api_key1", "api_key2", "api_key3"]
	},
	"bedrock": {
		"type": "bedrock",
		"name": "Bedrock",
		"region": "us-east-1",
		"apiKeys": ["api_key1", "api_key2", "api_key3"]
	},
	"vertex": {
		"type": "vertex",
		"name": "Vertex",
		"project": "google-project-id",
		"location": "global",
		"apiKeys": ["api_key1", "api_key2", "api_key3"]
	},
	"your-custom-provider": {
		"type": "openai-compatible",
		"name": "Custom provider",
		"baseUrl": "https://your-custom-provider.com",
		"headers": {
			"Authorization": "Bearer token",
			"X-Custom-Header": "custom_value"
		}
	}
}'

Every provider accepts the following options:

type (required) – the provider type: openai, anthropic, google, azure, bedrock, vertex, or openai-compatible.
name (optional) – the name displayed in the models list. Defaults to the provider key.
apiKeys (required for openai, anthropic, and google; optional for the remaining types) – the API keys for the provider. The service uses the first key. If the provider rejects that key as invalid, the service uses the next one.
baseUrl (required for openai-compatible, optional for the other types) – the base URL of the provider. For an openai-compatible provider, every request goes to {baseUrl}/chat/completions. For the other types, baseUrl replaces the default API URL.
headers (optional) – additional headers sent with every request to the provider.

Azure OpenAI needs either apiKeys or the authentication option, see Azure OpenAI. Amazon Bedrock and Google Vertex AI take their native credentials instead: an access key or a service account key, a role the service assumes, or the credentials of the host it runs on. An openai-compatible provider needs no keys when its endpoint is open or authenticates through headers.

OpenAI, Anthropic, and Google

These three need only an API key. Set type and apiKeys, and add baseUrl or headers if your traffic goes through a proxy. The google type calls the Gemini API. For Gemini or Claude models served from your own Google Cloud project, use Google Vertex AI instead.

{
	"providers": {
		"openai": {
			"type": "openai",
			"apiKeys": ["your-openai-api-key"]
		},
		"anthropic": {
			"type": "anthropic",
			"apiKeys": ["your-anthropic-api-key"]
		},
		"google": {
			"type": "google",
			"apiKeys": ["your-gemini-api-key"]
		}
	}
}

Azure OpenAI

The Azure OpenAI provider also accepts:

resourceName (optional) – the Azure resource name. You can point to the resource with baseUrl instead.
apiVersion (optional, default: v1) – the API version, sent as the api-version query parameter.
useDeploymentBasedUrls (optional, default: false) – use the legacy deployment-based URLs, {baseUrl}/deployments/{model id}/chat/completions, instead of the default Responses API URLs under {baseUrl}/v1. Set it to true when your deployment exposes only the deployment-based Chat Completions endpoint.
authentication (optional) – token-based authentication. Set either apiKeys or authentication, not both.
- type (required) – either bearer-token or entra-id.
- tokenSecret (required for bearer-token) – the token sent with every request to the Azure endpoint.
- scope (required for entra-id) – the scope requested from Microsoft Entra ID, for example https://cognitiveservices.azure.com/.default.

With bearer-token, the service sends the token you provide. With entra-id, the service obtains tokens from the Azure default credential chain. The identity comes from the host: environment variables, a managed identity, or a workload identity. The service refreshes the tokens itself.

The id of every model this provider serves must be your Azure deployment name, and the name must start with the id of the OpenAI model behind it. See id under Custom models.

{
	"providers": {
		"azureBearer": {
			"type": "azure",
			"name": "Azure OpenAI with bearer token",
			"baseUrl": "https://your-resource.openai.azure.com",
			"authentication": {
				"type": "bearer-token",
				"tokenSecret": "your_token"
			}
		},
		"azureEntra": {
			"type": "azure",
			"name": "Azure OpenAI with Microsoft Entra ID",
			"baseUrl": "https://your-resource.openai.azure.com",
			"authentication": {
				"type": "entra-id",
				"scope": "https://cognitiveservices.azure.com/.default"
			}
		}
	}
}

Amazon Bedrock

The Amazon Bedrock provider also accepts:

region (required) – the AWS region that serves the models, for example us-east-1.
credentials (optional) – IAM credentials, an alternative to apiKeys.
- accessKeyId (required) – the AWS access key ID.
- secretAccessKey (required) – the AWS secret access key.
- sessionToken (optional) – the AWS session token.
assumedRole (optional) – an IAM role the service assumes through AWS STS before it calls Bedrock.
- roleArn (required) – the ARN of the role to assume.
- externalId (optional) – the external ID that the trust policy of the role requires.
- sessionDurationSeconds (optional) – the lifetime of the temporary credentials, from 900 to 43200 seconds, capped by the maximum session duration of the role. Left unset, the AWS default applies.
- roleSessionName (optional, default: tiugo-ai-service) – the session name recorded in AWS CloudTrail. It has to match ^[\w+=,.@-]{2,64}$.

The service authenticates with the first of these you configure:

assumedRole: the service calls sts:AssumeRole and signs Bedrock requests with the temporary credentials that come back, renewing them before they expire. The AssumeRole call itself uses credentials when you set both options, and the credentials of the host when assumedRole stands alone. The role carries the Bedrock permissions, so this is the cross-account setup.
credentials: the service signs requests with the access key you provide. Temporary keys also need sessionToken, and the service does not renew them.
apiKeys: the service sends the Bedrock API key as a bearer token, with no request signing.
The credentials of the host: with none of the three options set, or with apiKeys resolving to an empty string, the service takes the identity of the machine it runs on. The AWS SDK looks for it in its own order: environment variables, the shared AWS configuration files, a web identity token file, the container credentials endpoint, and the instance metadata service. A service on EC2, ECS, or EKS reaches Bedrock with no secret in providers at all.

{
	"providers": {
		"bedrockAssumedRole": {
			"type": "bedrock",
			"name": "Bedrock with an assumed role",
			"region": "us-east-1",
			"assumedRole": {
				"roleArn": "arn:aws:iam::123456789012:role/ckeditor-ai-bedrock",
				"externalId": "your-external-id",
				"sessionDurationSeconds": 3600
			}
		},
		"bedrockHostCredentials": {
			"type": "bedrock",
			"name": "Bedrock with the credentials of the host",
			"region": "us-east-1"
		}
	}
}

Warning

The AWS SDK reads the AWS_BEARER_TOKEN_BEDROCK environment variable as a Bedrock API key, and it wins over every signed path above, assumedRole and credentials included. Leave it unset unless it is the identity you want.

Google Vertex AI

The Google Vertex AI provider also accepts:

project (required) – the Google Cloud project ID. You can set it here or in the GOOGLE_VERTEX_PROJECT environment variable.
location (required) – the Google Cloud location, for example us-central1. You can set it here or in the GOOGLE_VERTEX_LOCATION environment variable.
credentials (optional) – service account credentials, an alternative to apiKeys.
- clientEmail (required) – the service account email.
- privateKey (required) – the service account private key.
impersonation (optional) – a service account the service impersonates before it calls Vertex AI.
- targetPrincipal (required) – the email of the service account to impersonate.
- lifetimeSeconds (optional, default: 3600, maximum: 43200) – the lifetime of the access tokens issued for that service account.

As with Bedrock, the service authenticates with the first of these you configure:

impersonation: the service asks the IAM Credentials API for a short-lived access token for targetPrincipal, scoped to https://www.googleapis.com/auth/cloud-platform, and calls Vertex AI with that token, renewing it as it expires. The token request uses credentials when you set both options, and the credentials of the host when impersonation stands alone. Grant the source identity the roles/iam.serviceAccountTokenCreator role on the target service account.
credentials: the service signs a JWT with the service account key you provide and exchanges it for an access token.
apiKeys: the service calls Vertex AI in express mode with the API key.
The credentials of the host: with none of the three options set, the service takes the application default credentials of the machine it runs on. The Google auth library looks for them in its own order: the file that GOOGLE_APPLICATION_CREDENTIALS points to, the file that gcloud auth application-default login writes, and the service account attached to the host. A service on Google Cloud reaches Vertex AI with no secret in providers at all.

{
	"providers": {
		"vertexImpersonation": {
			"type": "vertex",
			"name": "Vertex AI with an impersonated service account",
			"project": "your-google-project-id",
			"location": "us-central1",
			"impersonation": {
				"targetPrincipal": "ckeditor-ai@your-google-project-id.iam.gserviceaccount.com",
				"lifetimeSeconds": 3600
			}
		},
		"vertexHostCredentials": {
			"type": "vertex",
			"name": "Vertex AI with the credentials of the host",
			"project": "your-google-project-id",
			"location": "us-central1"
		}
	}
}

Warning

An API key authenticates Gemini models only, because express mode is a Gemini feature. Requests to Claude models on Vertex AI always carry a Google OAuth token, so a provider that has only apiKeys falls back to the credentials of the host for them. Give a provider that serves Claude models credentials or impersonation, or run it on a host that carries application default credentials.

Custom providers

Any service that implements the OpenAI Chat Completions API can be a provider. Examples: a LiteLLM proxy in front of your own models, a hosted gateway such as Groq, or an Ollama instance on your own hardware. Declare it with type: "openai-compatible". Every request goes to {baseUrl}/chat/completions, so include any prefix the routes sit under, such as the /v1 in the examples below.

{
	"providers": {
		"litellm": {
			"type": "openai-compatible",
			"name": "LiteLLM",
			"baseUrl": "http://litellm.internal:4000/v1",
			"apiKeys": ["sk-litellm-master-key"]
		},
		"groq": {
			"type": "openai-compatible",
			"name": "Groq",
			"baseUrl": "https://api.groq.com/openai/v1",
			"apiKeys": ["gsk_your_groq_key"]
		},
		"ollama": {
			"type": "openai-compatible",
			"name": "Ollama",
			"baseUrl": "http://ollama.internal:11434/v1"
		}
	}
}

The value should be provided as a stringified JSON object:

PROVIDERS='{
	"litellm": {
		"type": "openai-compatible",
		"name": "LiteLLM",
		"baseUrl": "http://litellm.internal:4000/v1",
		"apiKeys": ["sk-litellm-master-key"]
	},
	"groq": {
		"type": "openai-compatible",
		"name": "Groq",
		"baseUrl": "https://api.groq.com/openai/v1",
		"apiKeys": ["gsk_your_groq_key"]
	},
	"ollama": {
		"type": "openai-compatible",
		"name": "Ollama",
		"baseUrl": "http://ollama.internal:11434/v1"
	}
}'

The Ollama entry above has no apiKeys, because a local Ollama endpoint requires no authentication. If the endpoint expects something other than a bearer token, send it in headers, as in the your-custom-provider example at the top of the page.

An openai-compatible provider has no default model list. Declare every model it serves in the models option, with provider set to the provider key. See Custom models.

Custom models

The OpenAI, Anthropic, and Google providers come with a default list of models. The models option replaces the default list with your own. It is also where you declare the models served by Azure OpenAI, Amazon Bedrock, Google Vertex AI, and openai-compatible providers, which have no defaults.

{
	"models": [
		{
			"id": "model1",
			"name": "Model 1",
			"description": "Model 1 description",
			"provider": "your-custom-provider",
			"recommended": true,
			"capabilities": {
				"webSearch": true,
				"reasoning": false
			},
			"features": ["conversations", "reviews", "actions"]
		},
		{
			"id": "model2",
			"name": "Model 2",
			"description": "Model 2 description",
			"provider": "your-custom-provider",
			"recommended": true,
			"capabilities": {
				"webSearch": true,
				"reasoning": false
			},
			"features": ["conversations", "reviews", "actions"]
		}
	]
}

The value should be provided as a stringified JSON array:

MODELS='[
	{
		"id": "model1",
		"name": "Model 1",
		"description": "Model 1 description",
		"provider": "your-custom-provider",
		"recommended": true,
		"capabilities": {
			"webSearch": true,
			"reasoning": false
		},
		"features": ["conversations", "reviews", "actions"]
	},
	{
		"id": "model2",
		"name": "Model 2",
		"description": "Model 2 description",
		"provider": "your-custom-provider",
		"recommended": true,
		"capabilities": {
			"webSearch": true,
			"reasoning": false
		},
		"features": ["conversations", "reviews", "actions"]
	}
]'

Note

For the best results, use models of the same class as the latest major releases from Anthropic, Google, or OpenAI. Older models and smaller models produce weaker results.

For each model you can set the following options:

id (required) – the model identifier used when calling the provider. It must be unique across all models.

Note
For the Azure OpenAI provider, the model id must be your Azure deployment name. Deployment names are free-form, so start them with the underlying OpenAI model id. The service reads that prefix to recognize the model family and apply the right request options.
provider (required) – the key of the provider that serves the model, as defined in providers. The match is case-insensitive.
description (required) – the description displayed in the models list.
type (optional) – either standard (the default) or agent, described in the Agent models section below.
name (optional) – the name displayed in the models list. Defaults to the model id.
recommended (optional) – whether the model belongs to the recommended list. CKEditor 5 selects a recommended model by default.
capabilities (optional) – what the model can do, as an object with the following keys:
- webSearch (optional, default: false) – whether the model can use the web search feature.
- reasoning (optional, default: false) – whether the model can use the reasoning feature.
contextLimits (optional) – the limits applied to a single conversation, as an object with the following keys:
- maxContextLength (optional, default: 256000) – the maximum context length in characters.
- maxFiles (optional, default: 100) – the maximum number of files in a context.
- maxFileSize (optional, default: 25 MB) – the maximum size of a single file, in bytes.
- maxTotalFileSize (optional, default: 30 MB) – the maximum total size of all files in a context, in bytes.
- maxTotalPdfFilePages (optional, default: 100) – the maximum total number of pages across all PDF files in a context.
Note
contextLimits only caps what the model accepts. If a model does not accept file input at all, turn off the file upload permission for the users who can select it.
features (optional) – the features the model serves. System reviews, system actions, and conversation title generation run on a model that the service picks. This option is how you decide which one.

Note
The service fails to start unless every conversation, review, and action feature has a model. List conversations, reviews, and actions on at least one model each, or cover their sub-features one by one. Some features also need a model that supports structured output.

Available features

conversations – all conversation features.
conversations.titleGeneration – conversation title generation.
reviews – all reviews.
reviews.correctness – the correctness review.
reviews.clarity – the clarity review.
reviews.readability – the readability review.
reviews.make-longer – the review that makes text longer.
reviews.make-shorter – the review that makes text shorter.
reviews.make-tone-casual – the review that makes the tone casual.
reviews.make-tone-direct – the review that makes the tone direct.
reviews.make-tone-friendly – the review that makes the tone friendly.
reviews.make-tone-confident – the review that makes the tone confident.
reviews.make-tone-professional – the review that makes the tone professional.
reviews.translate – the translation review.
actions – all actions.
actions.make-longer – the action that makes text longer.
actions.make-shorter – the action that makes text shorter.
actions.make-tone-casual – the action that makes the tone casual.
actions.make-tone-direct – the action that makes the tone direct.
actions.make-tone-friendly – the action that makes the tone friendly.
actions.make-tone-confident – the action that makes the tone confident.
actions.make-tone-professional – the action that makes the tone professional.
actions.translate – the translation action.
actions.continue – the continue action.
actions.fix-grammar – the action that fixes grammar.
actions.improve-writing – the action that improves writing.

Model resolution order

Feature names form a dot-separated hierarchy. In reviews.correctness, the parent feature is reviews. When several models declare features at different levels, a request goes to the model with the most specific match:

A request for reviews.correctness goes to a model that declares reviews.correctness, even if a model that declares reviews comes first in the configuration.
A feature with no exact match falls back to its parent. With only reviews and reviews.correctness configured, a reviews.clarity request goes to the model that declares reviews.
When several models declare the same feature, the order in the configuration ranks them. The first model serves the request. If that model is unavailable, the next model serves it.

For example, take the following configuration, in this order:

Claude Haiku 4.5 with ["reviews.correctness"]
GPT 5 Mini with ["reviews.correctness"]
Gemini 3 Flash with ["reviews"]

Request	Routed to	Fallback	Reason
`reviews.correctness`	Claude Haiku 4.5	GPT 5 Mini	exact match, and Claude Haiku 4.5 comes first in the configuration
`reviews.clarity`	Gemini 3 Flash	–	no exact match, so the request falls back to the parent feature, `reviews`

Agent models

An agent model is a virtual model. It calls no provider of its own. It lists standard models in fallbackOrder, and each request goes to the first model on that list that is available. If none is available, the request fails.

To the end user, an agent model looks like any other model. The service exposes only the id, name, and description of the agent model. It never sends the fallback list or the provider names to the client.

The type property of a models entry sets the kind: standard (the default, described in Custom models) or agent.

{
	"models": [
		{
			"type": "standard",
			"id": "claude-major-model",
			"name": "Claude major model",
			"description": "Most powerful model in Claude family",
			"provider": "anthropic",
			"capabilities": {
				"webSearch": true,
				"reasoning": true
			}
		},
		{
			"id": "gpt-newest-model",
			"name": "GPT newest model",
			"description": "Newest model in GPT family",
			"provider": "openai",
			"capabilities": {
				"webSearch": true,
				"reasoning": true
			}
		},
		{
			"type": "agent",
			"id": "default-agent",
			"name": "CKEditor AI Agent",
			"description": "Automatically selects the best available model.",
			"recommended": true,
			"capabilities": {
				"webSearch": true,
				"reasoning": true
			},
			"features": ["conversations", "reviews", "actions"],
			"fallbackOrder": ["claude-major-model", "gpt-newest-model"]
		}
	]
}

The value should be provided as a stringified JSON array:

MODELS='[
	{
		"type": "standard",
		"id": "claude-major-model",
		"name": "Claude major model",
		"description": "Most powerful model in Claude family",
		"provider": "anthropic",
		"capabilities": {
			"webSearch": true,
			"reasoning": true
		}
	},
	{
		"id": "gpt-newest-model",
		"name": "GPT newest model",
		"description": "Newest model in GPT family",
		"provider": "openai",
		"capabilities": {
			"webSearch": true,
			"reasoning": true
		}
	},
	{
		"type": "agent",
		"id": "default-agent",
		"name": "CKEditor AI Agent",
		"description": "Automatically selects the best available model.",
		"recommended": true,
		"capabilities": {
			"webSearch": true,
			"reasoning": true
		},
		"features": ["conversations", "reviews", "actions"],
		"fallbackOrder": ["claude-major-model", "gpt-newest-model"]
	}
]'

An agent model takes the same options as a standard model (id, name, description, recommended, capabilities, contextLimits, and features), with the following differences:

type (required) – must be set to agent.
fallbackOrder (required) – a non-empty, ordered array of ids of standard models defined in the same models array. The service uses the first available one.
provider – ignored. Each request runs on the standard model that serves it and uses the provider of that model.
id – must be unique across all models, as for standard models. No model of either type may use the reserved agent- prefix.

Note

If an agent model declares webSearch or reasoning as true, every model in its fallbackOrder must declare it too. Otherwise the service fails to start with a validation error.

When to use agent models

Agent models separate the model the end user picks from the model that runs. Use them to:

Change models without touching clients – reorder or replace the entries in fallbackOrder to roll out a newer model, cut costs, or replace a retired one. End users keep the same agent model, and editor configuration and permissions stay as they are.
Keep your model choices private – end users and your integration see the agent model only, not the models and providers behind it.
Keep answering during provider outages – the service moves to the next available model, so the agent model still answers while one provider is down or throttled.

Model availability

The service tracks the health of every model. It takes an unhealthy model out of rotation for 60 seconds. All instances share this state through Redis, so a model that is out of rotation is out everywhere.

A model goes out of rotation after three failures, or three responses that took more than 10 seconds to produce their first token, within 5 minutes. The service counts the failures and the slow responses separately. One healthy response resets both counts. After 60 seconds, the service sends one request to the model as a probe. If the request succeeds, the model returns to rotation. If it fails, the model waits another 60 seconds.

The service picks the model before it sends the request, and the choice does not change mid-request. A request to a standard model that is out of rotation fails, as there is nothing to fall back to. An agent model takes the first model in its fallbackOrder that is in rotation and fails only when none is.

A standard model offered directly to end users has no fallback, so an outage at its provider reaches them as failed requests. To avoid that, offer an agent model instead and list models from two providers in its fallbackOrder.