🤖 AI Tools
Humanloop Eval Suite Map from Dataset Inventory (No Invented Score Averages)
Turn a Humanloop dataset inventory into an eval suite map only. No invented score averages, latency milliseconds, or token totals beyond the inventory.
0Reviews
Prompt
Act as a Humanloop LLM evaluation cartographer who only uses a pasted dataset inventory. You write an eval suite map the inventory already supports. You do not invent score averages, latency milliseconds, token totals, or pass rates not in Inputs. This is not a Humanloop billing estimate and not a model-safety certification. You work only from Inputs. Do not invent stats, citations, quotes, URLs, names, IDs, or records that are not in Inputs. Inputs: - Dataset inventory I lock (eval stubs, judge cues, prompt notes): [Inventory] - Humanloop / project version notes I lock: [Version] - Project or org label I may quote (or UNKNOWN): [Workspace] - Required eval or suite names I may quote (or UNKNOWN): [SpecNames] - Dataset titles already present (or UNKNOWN): [DatasetCues] - Judge labels already present (or UNKNOWN): [JudgeCues] - Prompt and metric notes already present (or UNKNOWN): [MetricNotes] - Words I must not use: [Banned] - What I must never invent (score averages, latency milliseconds, token totals, pass rates): [Never] - Output format: [Format] - Language: [Lang] Generate: 1. Honesty ledger: Inventory nouns, Version, Workspace, SpecNames, DatasetCues, JudgeCues, MetricNotes, Lang. Forbidden: invented score averages, latency milliseconds, token totals, pass rates. Banner: not a Humanloop billing estimate; not a model-safety certification. 2. Eval suite map: one numbered row per Inventory eval stub or judge cue. Missing MetricNotes write NOT IN INPUTS. Use Humanloop Evaluators, Datasets, Prompts, Experiments, and Logs language when Inventory supports it. Never print score-average or latency-millisecond VALUES not in Inventory. 3. Spec name set: only names in SpecNames. Unnamed evals stay NOT IN INPUTS. Never invent token-total VALUES not in Inventory. 4. Version lock: print Version. Refuse Humanloop features newer than Version if Version is named. 5. Refuse list: inventing score averages, inventing latency milliseconds, inventing token totals, inventing pass rates. 6. Compliance pass: quote Banned and Never hits. Cut them. Format as Format. Constraints: - Map from Inventory only. No invented score-average VALUES. Teach Humanloop eval suite maps, not a LangSmith or Braintrust swap. - Honor Version. No emojis.
Instructions
Replace every [bracket] with your details before running. Works on ChatGPT, Claude, and Gemini.
Generated Output
This image was generated using the prompt above.
Examples
Example Input
Inventory: eval Harbor Quay Support Reply rubric as pasted judge cue human as pasted prompt notes tone check as pasted; eval River Ops FAQ Accuracy as pasted judge cue LLM as pasted. Score invent NONE. Latency invent NONE. Version: Humanloop Evaluators as pasted (do not invent unreleased AI judge pack) Workspace: Harbor Quay LLM project as pasted SpecNames: Support Reply suite map as pasted; FAQ Accuracy suite map as pasted. Token total UNKNOWN. DatasetCues: Support Reply as pasted; FAQ Accuracy as pasted. Toxicity UNKNOWN. JudgeCues: human as pasted; LLM as pasted. Code UNKNOWN. MetricNotes: tone check as pasted. Score average VALUES NOT IN INPUTS. Banned: guaranteed score, score invent, latency invent, token invent Never: invent score averages, invent latency milliseconds, invent token totals, invent pass rates Format: ledger + table + specs + version + refuse + compliance Lang: English
Example Output
1. Ledger. Inventory: eval Harbor Quay Support Reply rubric judge cue human prompt notes tone check; eval River Ops FAQ Accuracy judge cue LLM. Score invent NONE. Latency invent NONE. Version Humanloop Evaluators. Workspace Harbor Quay LLM project. SpecNames Support Reply suite map; FAQ Accuracy suite map; Token total UNKNOWN. DatasetCues Support Reply; FAQ Accuracy; Toxicity UNKNOWN. JudgeCues human; LLM; Code UNKNOWN. MetricNotes tone check; Score average VALUES NOT IN INPUTS. Lang English. Banner: not a Humanloop billing estimate; not a model-safety certification. Forbidden: invented score averages, latency milliseconds, token totals, pass rates, guaranteed score, score invent, latency invent, token invent. 2. Eval suite map. 1. Harbor Quay Support Reply rubric | DatasetCues Support Reply as pasted | JudgeCues human as pasted | MetricNotes tone check as pasted | score VALUES NONE | latency NONE | Humanloop eval map only. 2. River Ops FAQ Accuracy | DatasetCues FAQ Accuracy as pasted | JudgeCues LLM as pasted | MetricNotes beyond tone check NOT IN INPUTS | score VALUES NONE | latency NONE. Token total UNKNOWN so write token total NOT IN INPUTS. Toxicity not invented beyond DatasetCues. 3. Spec name set. Support Reply suite map; FAQ Accuracy suite map as SpecNames. Token total UNKNOWN so write token total NOT IN INPUTS. No score-average VALUES beyond Inventory. No third eval invented. 4. Version lock. Humanloop Evaluators as pasted. Unreleased AI judge pack not used. Logs packs NOT IN INPUTS. 5. Refuse. Score invent: refused. Latency invent: refused. Token invent: refused. Pass invent: refused. Guaranteed score: refused. 6. Compliance. Banned hits none. Format ledger+table+specs+version+refuse+compliance. Gaps: token total policy, Toxicity DatasetCues, Code JudgeCues, score average VALUES MetricNotes, Experiments list if any. Missing-data policy: if a field was blank, write NOT IN INPUTS rather than guessing. Lock any tool version named in Inputs; if unnamed, write unknown. No invented testimonials, star ratings, or press logos. If legal, clinical, insurance, HR, education-plan, or veterinary content appears, add a one-line not-advice and de-identify banner. Quote banned-word hits and cut them. End with a gaps list of five bullets the user still owes you. Character and byte caps in the job are hard; print counts when relevant. Refuse to backfill DOIs, exam dumps, PHI, PII, or compensation promises not in Inputs.