Dự phòng model

Tự động chuyển đổi dự phòng giữa các phòng thí nghiệm. Mẫu tuần tự và song song.

Pattern 1: sequential fallback (simple)

from openai import OpenAI

client = OpenAI(base_url="https://meshtok.com/v1", api_key="sk-...")

FALLBACK_CHAIN = [
    "openai/gpt-5.2",
    "anthropic/claude-sonnet-5",
    "deepseek/deepseek-v4-pro",
    "bigmodel/glm-5.2",
]

def chat_with_fallback(messages, **kwargs):
    last_err = None
    for model in FALLBACK_CHAIN:
        try:
            resp = client.chat.completions.create(model=model, messages=messages, **kwargs)
            return resp.choices[0].message.content
        except Exception as e:
            last_err = e
    raise last_err

Pattern 2: parallel race (lowest latency)

import asyncio
from openai import AsyncOpenAI

client = AsyncOpenAI(base_url="https://meshtok.com/v1", api_key="sk-...")

async def try_model(model, messages):
    try:
        resp = await client.chat.completions.create(model=model, messages=messages)
        return resp.choices[0].message.content
    except Exception:
        return None

async def race(models, messages):
    tasks = [asyncio.create_task(try_model(m, messages)) for m in models]
    for coro in asyncio.as_completed(tasks):
        result = await coro
        if result:
            for t in tasks: t.cancel()
            return result
    raise RuntimeError("all models failed")

Choosing fallback models

• Frontier: GPT-5.x, Claude Sonnet/Opus, Gemini 2.5 Pro
• Strong open-weight: DeepSeek V4, GLM 5.2, Qwen 3, Kimi K2
• Cheap/fast: DeepSeek V4 Flash, GLM Flash, Haiku