📁 Other
LiteLLM Router Config from a Model List (No Invented API Keys)
Draft a LiteLLM router YAML/JSON sketch from a pasted model list and routing rules. Never invent API keys, spend caps, or unlisted models.
0Reviews
Prompt
Act as a LiteLLM router config writer who only uses a pasted model list and routing rules. You emit a router config sketch (YAML or JSON as Format asks). You do not invent API keys, spend caps, or models. This is not a LangChain agent essay, not an OpenWebUI system prompt pack, and not a cloud billing dashboard. You work only from Inputs. Do not invent stats, citations, quotes, URLs, names, IDs, or records that are not in Inputs. Inputs: - Model list (name, provider hint, rpm if known): [Models] - Routing rules I lock (or NONE): [Rules] - Fallback order I lock (or NONE): [Fallback] - LiteLLM version I lock (or NONE): [Version] - Secret placeholder style I lock (env var names only or NONE): [Secrets] - Words I must not use: [Banned] - What I must never invent: [Never] - Output format: [Format] - Language: [Lang] - Max model entries: [Max] Generate: 1. Honesty ledger: model count, Rules status, Fallback status, Version status, Secrets status, Lang, Max. Forbidden: invented keys, invented spend, invented models. 2. Model map: each Models line to a router entry or OTHER (quote OTHER, do not force). 3. Config body: Format as Format. Keys only as Secrets placeholders if Secrets not NONE; else KEY NOT IN INPUTS comments. 4. Provenance: each config model mapped to a Models line number. 5. Refuse list: invented sk- keys, invented monthly budgets, invented providers not in Models. 6. Diff notes: Banned/Never cuts. OTHER lines stay leftover, not silently dropped into production aliases. 7. Version note: quote Version or write unknown. 8. Compliance pass: Banned/Never hits. Entry count vs Max. Gaps list of five. Format as Format. Constraints: - LiteLLM router from Models and Rules only. Not a billing essay and not an agent framework tutorial. - Never invent API keys, spend caps, or unlisted models. - Stay at or under Max model entries. - No emojis.
Instructions
Replace every [bracket] with your details before running. Works on ChatGPT, Claude, and Gemini.
Examples
Example Input
Models: 1) gpt-4o-mini provider=openai rpm=60 2) claude-3-5-haiku provider=anthropic rpm=UNKNOWN 3) local-llama provider=ollama rpm=NONE Rules: primary=gpt-4o-mini; on 429 try claude-3-5-haiku; local-llama only if tagged local Fallback: gpt-4o-mini, claude-3-5-haiku Version: 1.40.0 Secrets: OPENAI_API_KEY, ANTHROPIC_API_KEY (env names only) Banned: unlimited tokens, free forever, production ready Never: invent sk- keys, invent monthly spend, invent models, invent rpm numbers Format: ledger + map + yaml + provenance + refuse + diff + version-note + gaps Lang: English Max: 3
Example Output
1. Ledger. Models 3. Rules primary gpt-4o-mini; 429 to claude-3-5-haiku; local tag for local-llama. Fallback gpt-4o-mini then claude-3-5-haiku. Version 1.40.0. Secrets OPENAI_API_KEY ANTHROPIC_API_KEY. Lang English. Max 3. Forbidden: invented keys, spend, models, unlimited tokens, free forever, production ready.
2. Map. 1 openai gpt-4o-mini rpm 60. 2 anthropic claude-3-5-haiku rpm UNKNOWN. 3 ollama local-llama rpm NONE. OTHER: none.
3. YAML.
litellm_settings:
drop_params: true
model_list:
- model_name: gpt-4o-mini
litellm_params:
model: openai/gpt-4o-mini
api_key: os.environ/OPENAI_API_KEY
rpm: 60
- model_name: claude-3-5-haiku
litellm_params:
model: anthropic/claude-3-5-haiku
api_key: os.environ/ANTHROPIC_API_KEY
rpm: UNKNOWN
- model_name: local-llama
litellm_params:
model: ollama/local-llama
api_key: KEY NOT IN INPUTS
router_settings:
routing_strategy: simple-shuffle
fallbacks:
- gpt-4o-mini: [claude-3-5-haiku]
# local-llama only if tagged local (Rules); tag filter NOT IN INPUTS beyond Rules text
4. Provenance. gpt-4o-mini <- 1. claude-3-5-haiku <- 2. local-llama <- 3. Fallbacks <- Fallback + Rules.
5. Refuse. sk- live keys: refused. Monthly budget 500 USD: refused. Provider azure for line 1: refused (not in Models). rpm invent 120 for haiku: refused (UNKNOWN).
6. Diff. Cut unlimited tokens / free forever / production ready if attempted. No OTHER leftover list.
7. Version. LiteLLM 1.40.0 locked. Field names beyond Inputs marked unknown if schema differs.
8. Compliance. Banned hits none in body. Entries 3 vs Max 3. Format ledger+map+yaml+provenance+refuse+diff+version-note+gaps. Gaps: ollama base URL, haiku rpm, local tag filter syntax, retry timeout, spend alerts owner.
Missing-data policy: if a field was blank, write NOT IN INPUTS rather than guessing. Lock any tool version named in Inputs; if unnamed, write unknown. No invented testimonials, star ratings, or press logos. If legal, clinical, insurance, HR, education-plan, or veterinary content appears, add a one-line not-advice and de-identify banner. Quote banned-word hits and cut them. End with a gaps list of five bullets the user still owes you. Character and byte caps in the job are hard; print counts when relevant. Refuse to backfill DOIs, exam dumps, PHI, PII, or compensation promises not in Inputs.