🤖 AI Tools
Promptfoo Eval Case Map from Suite Inventory (No Invented Pass Rates)
Turn a Promptfoo suite inventory into an eval case map only. No invented pass rates, API keys, or token costs beyond the inventory.
0Reviews
Prompt
Act as a Promptfoo eval cartographer who only uses a pasted suite inventory. You write an eval case map the inventory already supports. You do not invent pass rates, API keys, token costs, or latency milliseconds not in Inputs. This is not a Promptfoo billing estimate and not a live CI gate certificate. You work only from Inputs. Do not invent stats, citations, quotes, URLs, names, IDs, or records that are not in Inputs. Inputs: - Suite eval inventory I lock (case stubs, assert cues, provider notes): [Inventory] - Promptfoo / suite version notes I lock: [Version] - Project or repo label I may quote (or UNKNOWN): [Workspace] - Required suite or case names I may quote (or UNKNOWN): [SpecNames] - Case titles already present (or UNKNOWN): [CaseCues] - Assertion labels already present (or UNKNOWN): [AssertCues] - Provider and model notes already present (or UNKNOWN): [ProviderNotes] - Words I must not use: [Banned] - What I must never invent (pass rates, API keys, token costs, latency ms): [Never] - Output format: [Format] - Language: [Lang] Generate: 1. Honesty ledger: Inventory nouns, Version, Workspace, SpecNames, CaseCues, AssertCues, ProviderNotes, Lang. Forbidden: invented pass rates, API keys, token costs, latency ms. Banner: not a Promptfoo billing estimate; not a live CI gate certificate. 2. Case map table: one row per Inventory case stub or assert cue. Missing ProviderNotes write NOT IN INPUTS. Use Promptfoo tests, asserts, providers, prompts, and metrics language when Inventory supports it. Never print pass-rate or token-cost VALUES not in Inventory. 3. Spec name set: only names in SpecNames. Unnamed cases stay NOT IN INPUTS. Never invent API-key VALUES not in Inventory. 4. Version lock: print Version. Refuse Promptfoo features newer than Version if Version is named. 5. Refuse list: inventing pass rates, inventing API keys, inventing token costs, inventing latency ms. 6. Compliance pass: quote Banned and Never hits. Cut them. Format as Format. Constraints: - Map from Inventory only. No invented pass-rate VALUES. Teach Promptfoo case mapping, not a LangSmith or DeepEval swap. - Honor Version. No emojis.
Instructions
Replace every [bracket] with your details before running. Works on ChatGPT, Claude, and Gemini.
Generated Output
This image was generated using the prompt above.
Examples
Example Input
Inventory: case Harbor Quay Refusal Guard as pasted assert cue contains as pasted provider notes openai gpt-4o-mini as pasted; case River Ops Citation Check as pasted assert cue similar as pasted. Pass invent NONE. Token invent NONE. Version: Promptfoo 0.104 as pasted (do not invent unreleased AI suite writer) Workspace: Harbor Quay evals repo as pasted SpecNames: Refusal Guard case map as pasted; Citation Check case map as pasted. API key UNKNOWN. CaseCues: Refusal Guard as pasted; Citation Check as pasted. Latency Burn UNKNOWN. AssertCues: contains as pasted; similar as pasted. javascript UNKNOWN. ProviderNotes: openai gpt-4o-mini as pasted. Pass rate VALUES NOT IN INPUTS. Banned: guaranteed pass, pass invent, token invent, latency invent Never: invent pass rates, invent API keys, invent token costs, invent latency ms Format: ledger + table + specs + version + refuse + compliance Lang: English
Example Output
1. Ledger. Inventory: case Harbor Quay Refusal Guard assert cue contains provider notes openai gpt-4o-mini; case River Ops Citation Check assert cue similar. Pass invent NONE. Token invent NONE. Version Promptfoo 0.104. Workspace Harbor Quay evals repo. SpecNames Refusal Guard case map; Citation Check case map; API key UNKNOWN. CaseCues Refusal Guard; Citation Check; Latency Burn UNKNOWN. AssertCues contains; similar; javascript UNKNOWN. ProviderNotes openai gpt-4o-mini; Pass rate VALUES NOT IN INPUTS. Lang English. Banner: not a Promptfoo billing estimate; not a live CI gate certificate. Forbidden: invented pass rates, API keys, token costs, latency ms, guaranteed pass, pass invent, token invent, latency invent. 2. Case map. 1. Harbor Quay Refusal Guard | CaseCues Refusal Guard as pasted | AssertCues contains as pasted | ProviderNotes openai gpt-4o-mini as pasted | pass VALUES NONE | token NONE | Promptfoo case map only. 2. River Ops Citation Check | CaseCues Citation Check as pasted | AssertCues similar as pasted | ProviderNotes beyond openai gpt-4o-mini NOT IN INPUTS | pass VALUES NONE | token NONE. API key UNKNOWN so write API key NOT IN INPUTS. Latency Burn not invented beyond CaseCues. 3. Spec name set. Refusal Guard case map; Citation Check case map as SpecNames. API key UNKNOWN so write API key NOT IN INPUTS. No pass-rate VALUES beyond Inventory. No third case invented. 4. Version lock. Promptfoo 0.104 as pasted. Unreleased AI suite writer not used. Latency packs NOT IN INPUTS. 5. Refuse. Pass invent: refused. Token invent: refused. API key invent: refused. Latency invent: refused. Guaranteed pass: refused. 6. Compliance. Banned hits none. Format ledger+table+specs+version+refuse+compliance. Gaps: API key policy, Latency Burn CaseCues, javascript AssertCues, pass rate VALUES ProviderNotes, metrics list if any. Missing-data policy: if a field was blank, write NOT IN INPUTS rather than guessing. Lock any tool version named in Inputs; if unnamed, write unknown. No invented testimonials, star ratings, or press logos. If legal, clinical, insurance, HR, education-plan, or veterinary content appears, add a one-line not-advice and de-identify banner. Quote banned-word hits and cut them. End with a gaps list of five bullets the user still owes you. Character and byte caps in the job are hard; print counts when relevant. Refuse to backfill DOIs, exam dumps, PHI, PII, or compensation promises not in Inputs.