Braintrust Eval Dataset Diff Ledger from Experiment Notes (No Invented Score Averages)

Prompt

Act as a Braintrust LLM eval engineer who only uses pasted experiment notes. You compile an eval dataset diff ledger the notes already support. You do not invent score averages, latency percentiles, leaderboard ranks, or token spend totals. This is not a live Braintrust experiment run and not a model vendor pitch.
You work only from Inputs. Do not invent stats, citations, quotes, URLs, names, IDs, or records that are not in Inputs.

Inputs:
- Experiment notes I lock (dataset labels, score cues, diff cues): [ExperimentNotes]
- Braintrust project or version I lock: [Version]
- Experiment name I may quote (or UNKNOWN): [ExperimentName]
- Dataset labels already present (or UNKNOWN): [DatasetLabels]
- Score config cues already present (or UNKNOWN): [ScoreCues]
- Diff or baseline cues already present (or UNKNOWN): [DiffCues]
- Prompt or model labels already present (or UNKNOWN): [PromptModel]
- Words I must not use: [Banned]
- What I must never invent (score averages, latency percentiles, leaderboard ranks, token spend): [Never]
- Output format: [Format]
- Language: [Lang]

Generate:
1. Honesty ledger: ExperimentNotes nouns, Version, ExperimentName, DatasetLabels, ScoreCues, DiffCues, PromptModel, Lang. Forbidden: invented score averages, latency percentiles, leaderboard ranks, token spend.
2. Diff checklist: one checkbox row per DiffCues or DatasetLabels entry. Mark baseline vs challenger only when ExperimentNotes says so. Missing score write NOT IN INPUTS.
3. Score-config table: for each ScoreCues entry, attach only DatasetLabels named beside it in ExperimentNotes. Do not invent an 0.92 average score if absent.
4. Model caution block: quote PromptModel only. Vendor benchmark packs not in ExperimentNotes stay NOT IN INPUTS.
5. Refuse list: inventing 0.92 score averages, inventing p95 640ms latency, inventing leaderboard rank #1, inventing token spend totals.
6. Compliance pass: quote Banned and Never hits. Cut them. Print dataset-label and score-cue counts from ExperimentNotes only. Format as Format.

Constraints:
- Eval dataset diff ledger from ExperimentNotes only. No invented score averages.
- Honor Version. No emojis. Not a live Braintrust Experiments dashboard.

Instructions

Replace every [bracket] with your details before running. Works on ChatGPT, Claude, and Gemini.

Generated Output

This image was generated using the prompt above.

Braintrust Eval Dataset Diff Ledger from Experiment Notes (No Invented Score Averages) - Result

Examples

Example Input

ExperimentNotes: dataset Harbor Quay gold as pasted; diff cue vs Midpoint baseline as pasted. Score average invent NONE. Latency invent NONE.
Version: Braintrust cloud as pasted (do not invent unreleased AI eval coach)
ExperimentName: quay-prompt-v3 as pasted
DatasetLabels: Harbor Quay gold as pasted. Cedar silver UNKNOWN.
ScoreCues: factuality scorer as pasted. Midpoint toxicity UNKNOWN.
DiffCues: vs Midpoint baseline as pasted. Second challenger UNKNOWN.
PromptModel: prompt quay-system-v3 as pasted; model label UNKNOWN.
Banned: 0.92 average, p95 640ms, guaranteed leaderboard #1
Never: invent score averages, invent latency percentiles, invent leaderboard ranks, invent token spend
Format: ledger + diff checklist + score-config table + model caution + refuse + compliance
Lang: English

Example Output

1. Ledger. ExperimentNotes: dataset Harbor Quay gold; diff cue vs Midpoint baseline. Score average invent NONE. Latency invent NONE. Version Braintrust cloud. ExperimentName quay-prompt-v3. DatasetLabels Harbor Quay gold. ScoreCues factuality scorer. DiffCues vs Midpoint baseline. PromptModel quay-system-v3. Model label UNKNOWN. Lang English. Forbidden: invented score averages, latency percentiles, leaderboard ranks, token spend, 0.92 average, p95 640ms, guaranteed leaderboard #1.

2. Diff checklist.
[ ] Harbor Quay gold dataset as pasted.
[ ] vs Midpoint baseline as pasted.
Score for either line NOT IN INPUTS. Cedar silver not listed. Second challenger not listed.

3. Score-config table.
factuality scorer | DatasetLabels Harbor Quay gold as pasted | average NOT IN INPUTS so do not invent 0.92.
Midpoint toxicity | NOT IN INPUTS.
Second factuality scorer not invented.

4. Model caution. prompt quay-system-v3 as pasted. Model label UNKNOWN. Vendor benchmark pack NOT IN INPUTS. Do not invent leaderboard packs.

5. Refuse. 0.92 score averages: refused. p95 640ms latency: refused. leaderboard rank #1: refused. token spend totals: refused. Unreleased AI eval coach: refused.

6. Compliance. Banned hits none. Dataset-label rows 1. Score-cue rows 1. Format ledger+diff checklist+score-config table+model caution+refuse+compliance. Gaps: Cedar silver, Midpoint toxicity, second challenger, model label, score average.

Missing-data policy: if a field was blank, write NOT IN INPUTS rather than guessing. Lock any tool version named in Inputs; if unnamed, write unknown. No invented testimonials, star ratings, or press logos. If legal, clinical, insurance, HR, education-plan, or veterinary content appears, add a one-line not-advice and de-identify banner. Quote banned-word hits and cut them. End with a gaps list of five bullets the user still owes you. Character and byte caps in the job are hard; print counts when relevant. Refuse to backfill DOIs, exam dumps, PHI, PII, or compensation promises not in Inputs.