#promptfoo
3 prompts found
Promptfoo Eval Case Map from Suite Inventory (No Invented Pass Rates)
Turn a Promptfoo suite inventory into an eval case map only. No invented pass rates, API keys, or token costs beyond the inventory.
Act as a Promptfoo eval cartographer who only uses a pasted suite inventory. You write an eval case map the inventory already supports. You do not invent pass rates, API keys, token costs, or latency milliseconds not in Inputs. This is not a Promptfoo billing estimate and not a live CI gate certificate. You work only from Inputs. Do not invent stats, citations, quotes, URLs, names, IDs, or records that are not in Inputs. Inputs: - Suite eval inventory I lock (case stubs, assert cues, provider notes): [Inventory] - Promptfoo / suite version notes I lock: [Version] - Project or repo label I may quote (or UNKNOWN): [Workspace] - Required suite or case names I may quote (or UNKNOWN): [SpecNames] - Case titles already present (or UNKNOWN): [CaseCues] - Assertion labels already present (or UNKNOWN): [AssertCues] - Provider and model notes already present (or UNKNOWN): [ProviderNotes] - Words I must not use: [Banned] - What I must never invent (pass rates, API keys, token costs, latency ms): [Never] - Output format: [Format] - Language: [Lang] Generate: 1. Honesty ledger: Inventory nouns, Version, Workspace, SpecNames, CaseCues, AssertCues, ProviderNotes, Lang. Forbidden: invented pass rates, API keys, token costs, latency ms. Banner: not a Promptfoo billing estimate; not a live CI gate certificate. 2. Case map table: one row per Inventory case stub or assert cue. Missing ProviderNotes write NOT IN INPUTS. Use Promptfoo tests, asserts, providers, prompts, and metrics language when Inventory supports it. Never print pass-rate or token-cost VALUES not in Inventory. 3. Spec name set: only names in SpecNames. Unnamed cases stay NOT IN INPUTS. Never invent API-key VALUES not in Inventory. 4. Version lock: print Version. Refuse Promptfoo features newer than Version if Version is named. 5. Refuse list: inventing pass rates, inventing API keys, inventing token costs, inventing latency ms. 6. Compliance pass: quote Banned and Never hits. Cut them. Format as Format. Constraints: - Map from Inventory only. No invented pass-rate VALUES. Teach Promptfoo case mapping, not a LangSmith or DeepEval swap. - Honor Version. No emojis.
Promptfoo Eval Suite Map from Test Inventory (No Invented Scores)
Turn a Promptfoo test inventory into an eval suite map only. No invented scores, pass rates, or latency claims beyond the inventory.
Act as a Promptfoo eval engineer who only uses a pasted test inventory. You write an eval suite map the inventory already supports. You do not invent scores, pass rates, latency claims, or token totals. This is not a live promptfoo eval run and not a model leaderboard. You work only from Inputs. Do not invent stats, citations, quotes, URLs, names, IDs, or records that are not in Inputs. Inputs: - Test inventory I lock (case stubs, assert cues, status notes): [Inventory] - Promptfoo / provider version notes I lock: [Version] - Project or workspace label I may quote (or UNKNOWN): [Workspace] - Required case or suite names I may quote (or UNKNOWN): [SpecNames] - Words I must not use: [Banned] - What I must never invent (scores, pass rates, latency claims, token totals): [Never] - Output format: [Format] - Language: [Lang] Generate: 1. Honesty ledger: Inventory nouns, Version, Workspace, SpecNames, Lang. Forbidden: invented scores, pass rates, latency claims, token totals. Banner: not a live promptfoo eval run; not a model leaderboard. 2. Eval suite map table: one row per Inventory case stub or assert cue. Missing status notes write NOT IN INPUTS. Use Promptfoo prompts, providers, tests, assert, and promptfooconfig.yaml language when Inventory supports it. Never print score VALUES not in Inventory. 3. Spec name set: only names in SpecNames. Unnamed cases stay NOT IN INPUTS. Never print pass-rate or latency VALUES not in Inventory. 4. Version lock: print Version. Refuse Promptfoo features newer than Version if Version is named (no unreleased redteam claims). 5. Refuse list: inventing scores, inventing pass rates, inventing latency claims, inventing token totals. 6. Compliance pass: quote Banned and Never hits. Cut them. Format as Format. Constraints: - Map from Inventory only. No invented score VALUES. Teach Promptfoo eval suite mapping, not a generic test swap. - Honor Version. No emojis.
Promptfoo Eval Config Map from Dataset Inventory (No Invented Scores)
Turn a dataset inventory into a Promptfoo eval config map only. No invented scores, pass rates, or API keys beyond the inventory.
Act as a Promptfoo eval engineer who only uses a pasted dataset inventory. You write an eval config map the inventory already supports. You do not invent scores, pass rates, API keys, or user PII. This is not a live Promptfoo run and not an eval certification. You work only from Inputs. Do not invent stats, citations, quotes, URLs, names, IDs, or records that are not in Inputs. Inputs: - Dataset inventory I lock (case stubs, provider cues, assert notes): [Inventory] - Promptfoo / config version notes I lock: [Version] - Project or workspace label I may quote (or UNKNOWN): [Workspace] - Required case or provider names I may quote (or UNKNOWN): [SpecNames] - Words I must not use: [Banned] - What I must never invent (scores, pass rates, API keys, user PII): [Never] - Output format: [Format] - Language: [Lang] Generate: 1. Honesty ledger: Inventory nouns, Version, Workspace, SpecNames, Lang. Forbidden: invented scores, pass rates, API keys, user PII. Banner: not a live Promptfoo run; not an eval certification. 2. Eval config map table: one row per Inventory case stub or provider cue. Missing assert notes write NOT IN INPUTS. 3. Spec name set: only names in SpecNames. Unnamed cases stay NOT IN INPUTS. Never print numeric score or pass-rate VALUES not in Inventory. 4. Version lock: print Version. Refuse Promptfoo config keys newer than Version if Version is named. 5. Refuse list: inventing scores, inventing pass rates, inventing API keys, inventing user PII. 6. Compliance pass: quote Banned and Never hits. Cut them. Format as Format. Constraints: - Map from Inventory only. No invented score VALUES. - Honor Version. No emojis.