Adding Support for a New LLM Model

This skill guides you through adding support for a newly released LLM.

Quick Reference

Files to Modify

File

Purpose

front/types/assistant/models/{provider}.ts

Model ID + configuration

front/lib/api/assistant/token_pricing.ts

Pricing per million tokens

front/types/assistant/models/models.ts

Central registry

front/lib/api/llm/clients/{provider}/types.ts

Router whitelist

sdks/js/src/types.ts

SDK types

front/components/providers/types.ts

UI availability (optional)

front/lib/api/llm/tests/llm.test.ts

Integration tests

Prerequisites

Before adding, gather:

Model ID

Exact provider identifier (e.g.,

gpt-4-turbo-2024-04-09

)

Context size

Total context window in tokens

Pricing

Input/output cost per million tokens

Capabilities

Vision, structured output, reasoning effort levels

Tokenizer

Compatible tokenizer for token counting

Step-by-Step: Adding an OpenAI Model

Step 1: Add Model Configuration

Edit

front/types/assistant/models/openai.ts

:

export

const

GPT_4_TURBO_2024_04_09_MODEL_ID

=

"gpt-4-turbo-2024-04-09"

as

const

;

export

const

GPT_4_TURBO_2024_04_09_MODEL_CONFIG

:

ModelConfigurationType

=

{

providerId

:

"openai"

,

modelId

:

GPT_4_TURBO_2024_04_09_MODEL_ID

,

displayName

:

"GPT 4 turbo"

,

contextSize

:

128_000

,

recommendedTopK

:

32

,

recommendedExhaustiveTopK

:

64

,

largeModel

:

true

,

description

:

"OpenAI's GPT 4 Turbo model for complex tasks (128k context)."

,

shortDescription

:

"OpenAI's second best model."

,

isLegacy

:

false

,

isLatest

:

false

,

generationTokensCount

:

2048

,

supportsVision

:

true

,

minimumReasoningEffort

:

"none"

,

maximumReasoningEffort

:

"none"

,

defaultReasoningEffort

:

"none"

,

supportsResponseFormat

:

false

,

tokenizer

:

{

type

:

"tiktoken"

,

base

:

"cl100k_base"

}

,

}

;

Step 2: Add Pricing

Edit

front/lib/api/assistant/token_pricing.ts

:

const

CURRENT_MODEL_PRICING

:

Record

<

BaseModelIdType

,

PricingEntry

>

=

{

// ... existing

"gpt-4-turbo-2024-04-09"

:

{

input

:

10.0

,

// USD per million input tokens

output

:

30.0

,

// USD per million output tokens

cache_read_input_tokens

:

1.0

,

// Optional: cached reads

cache_creation_input_tokens

:

12.5

,

// Optional: cache creation

}

,

}

;

Step 3: Register in Central Registry

Edit

front/types/assistant/models/models.ts

:

export

const

MODEL_IDS

=

[

// ... existing

GPT_4_TURBO_2024_04_09_MODEL_ID

,

]

as

const

;

export

const

SUPPORTED_MODEL_CONFIGS

:

ModelConfigurationType

[

]

=

[

// ... existing

GPT_4_TURBO_2024_04_09_MODEL_CONFIG

,

]

;

Step 4: Update Router Whitelist

Edit

front/lib/api/llm/clients/openai/types.ts

:

export

const

OPENAI_WHITELISTED_MODEL_IDS

=

[

// ... existing

GPT_4_TURBO_2024_04_09_MODEL_ID

,

]

as

const

;

Step 5: Update SDK Types

Edit

sdks/js/src/types.ts

:

const

ModelLLMIdSchema

=

FlexibleEnumSchema

<

// ... existing

|

"gpt-4-turbo-2024-04-09"

>

(

)

;

Step 6: Add to UI (Optional)

Edit

front/components/providers/types.ts

:

export

const

USED_MODEL_CONFIGS

:

readonly

ModelConfig

[

]

=

[

// ... existing

GPT_4_TURBO_2024_04_09_MODEL_CONFIG

,

]

as

const

;

Step 7: Test (Mandatory)

Edit

front/lib/api/llm/tests/llm.test.ts

:

const

MODELS

=

{

// ... existing

[

GPT_4_TURBO_2024_04_09_MODEL_ID

]

:

{

runTest

:

true

,

// Enable for testing

providerId

:

"openai"

,

}

,

}

;

Run test:

RUN_LLM_TEST

=

true npx vitest

--config

lib/api/llm/tests/vite.config.js lib/api/llm/tests/llm.test.ts

--run

After test passes

, set

runTest: false

to avoid expensive CI runs.

Adding Anthropic Models

Same pattern with Anthropic-specific files:

front/types/assistant/models/anthropic.ts

- Add

CLAUDE_X_MODEL_ID

and config

front/lib/api/llm/clients/anthropic/types.ts

- Add to

ANTHROPIC_WHITELISTED_MODEL_IDS

front/types/assistant/models/models.ts

- Register in central registry

front/lib/api/assistant/token_pricing.ts

- Add pricing

sdks/js/src/types.ts

- Update SDK types

Test and validate

Model Configuration Properties

Property

Description

supportsVision

Can process images

supportsResponseFormat

Supports structured output (JSON)

minimumReasoningEffort

Min reasoning level ("none", "low", "medium", "high")

maximumReasoningEffort

Max reasoning level

defaultReasoningEffort

Default reasoning level

tokenizer

Tokenizer config for token counting

Validation Checklist

Model config added to provider file

Pricing updated (input, output, cache if applicable)

Registered in central registry (

MODEL_IDS

+

SUPPORTED_MODEL_CONFIGS

)

Router whitelist updated

SDK types updated

UI config added (if needed)

Integration test passes

Test disabled after validation

Troubleshooting

Model not in UI

Check

USED_MODEL_CONFIGS

in

front/components/providers/types.ts

API calls failing

Verify model ID matches provider's exact identifier, check router whitelist

Token counting errors

Validate context size and tokenizer configuration

Pricing issues

Ensure prices are per million tokens in USD Reference See front/types/assistant/models/openai.ts and anthropic.ts for examples Provider docs: OpenAI, Anthropic, Google, Mistral