Langfuse Trace Dataset Eval Checklist from Ops Notes (No Invented Latency Scores)

Prompt

Act as a Langfuse trace-dataset-eval checklist engineer who only uses pasted ops notes. You compile a trace-dataset-eval checklist the notes already support. You do not invent latency scores, accuracy ranks, eval scoreboards, or guaranteed quality claims. This is not a live Langfuse sync, not a LangSmith invent, and not MLOps consulting advice.
You work only from Inputs. Do not invent stats, citations, quotes, URLs, names, IDs, or records that are not in Inputs.

Inputs:
- Ops notes I lock (trace stubs, dataset cues, eval fragments): [OpsNotes]
- Langfuse project or version notes I lock: [Version]
- Project or workspace label I may quote (or UNKNOWN): [ProjectLabel]
- Trace titles already present (or UNKNOWN): [TraceTitles]
- Dataset cues already present (or UNKNOWN): [DatasetCues]
- Eval cues already present (or UNKNOWN): [EvalCues]
- Session cues already present (or UNKNOWN): [SessionCues]
- Words I must not use: [Banned]
- What I must never invent (latency scores, accuracy ranks, eval scoreboards, guaranteed quality claims): [Never]
- Output format: [Format]
- Language: [Lang]

Generate:
1. Honesty ledger: OpsNotes nouns, Version, ProjectLabel, TraceTitles, DatasetCues, EvalCues, SessionCues, Lang. Banner: not MLOps consulting advice; not a live Langfuse sync. Forbidden: invented latency scores, accuracy ranks, eval scoreboards, guaranteed quality claims.
2. Trace-dataset-eval checklist: one checkbox row per TraceTitles entry. Attach only DatasetCues and EvalCues named beside that trace in OpsNotes. Missing dataset or eval write NOT IN INPUTS.
3. Session sketch: for each SessionCues entry, list traces that name it. Do not invent a 412ms latency claim if absent.
4. Project caution block: quote ProjectLabel and Version only. Prompt packs not in OpsNotes stay NOT IN INPUTS.
5. Refuse list: inventing 412ms latency scores, inventing accuracy ranks, inventing eval scoreboards, inventing guaranteed quality claims.
6. Compliance pass: quote Banned and Never hits. Cut them. Print trace and dataset counts from OpsNotes only. Format as Format.

Constraints:
- Trace-dataset-eval checklist from OpsNotes only. No invented latency scores.
- Honor Version. No emojis. Not a live Langfuse dashboard. Not MLOps consulting advice.

Instructions

Replace every [bracket] with your details before running. Works on ChatGPT, Claude, and Gemini.

Generated Output

This image was generated using the prompt above.

Langfuse Trace Dataset Eval Checklist from Ops Notes (No Invented Latency Scores) - Result

Examples

Example Input

OpsNotes: trace title Harbor Quay Chat Trace as pasted dataset cue dataset:pier-gold as pasted eval cue eval:fog-rubric as pasted; trace title Quay Storm Tool Trace as pasted dataset cue dataset:storm-cases as pasted. Latency invent NONE. Accuracy invent NONE.
Version: Langfuse as pasted (do not invent unreleased AI quality coach)
ProjectLabel: Cedar Pier LLM observability as pasted
TraceTitles: Harbor Quay Chat Trace as pasted; Quay Storm Tool Trace as pasted. Prompt pack UNKNOWN.
DatasetCues: dataset cue dataset:pier-gold as pasted for Harbor Quay Chat Trace; dataset cue dataset:storm-cases as pasted for Quay Storm Tool Trace. Span pack UNKNOWN.
EvalCues: eval cue eval:fog-rubric as pasted for Harbor Quay Chat Trace; Quay Storm Tool Trace eval NOT IN INPUTS.
SessionCues: session cue session:quay-demo as pasted for Harbor Quay Chat Trace. Score pack UNKNOWN.
Banned: 412ms latency, accuracy rank #1, guaranteed eval scoreboard
Never: invent latency scores, invent accuracy ranks, invent eval scoreboards, invent guaranteed quality claims
Format: ledger + trace-dataset-eval checklist + session sketch + project caution + refuse + compliance
Lang: English

Example Output

1. Ledger. OpsNotes: trace title Harbor Quay Chat Trace + dataset cue dataset:pier-gold + eval cue eval:fog-rubric; trace title Quay Storm Tool Trace + dataset cue dataset:storm-cases. Latency invent NONE. Accuracy invent NONE. Version Langfuse. ProjectLabel Cedar Pier LLM observability. TraceTitles Harbor Quay Chat Trace; Quay Storm Tool Trace. DatasetCues dataset:pier-gold for Harbor Quay Chat Trace; dataset:storm-cases for Quay Storm Tool Trace. EvalCues eval:fog-rubric for Harbor Quay Chat Trace; Quay Storm Tool Trace eval NOT IN INPUTS. SessionCues session:quay-demo for Harbor Quay Chat Trace. Score pack UNKNOWN. Lang English. Banner: not MLOps consulting advice; not a live Langfuse sync. Forbidden: invented latency scores, accuracy ranks, eval scoreboards, guaranteed quality claims, 412ms latency, accuracy rank #1, guaranteed eval scoreboard.

2. Trace-dataset-eval checklist.
[ ] Harbor Quay Chat Trace | dataset dataset:pier-gold as pasted | eval eval:fog-rubric as pasted.
[ ] Quay Storm Tool Trace | dataset dataset:storm-cases as pasted | eval NOT IN INPUTS.
Span pack not attached. Prompt pack not added.

3. Session sketch.
session cue session:quay-demo | trace Harbor Quay Chat Trace as pasted.
Quay Storm Tool Trace session | NOT IN INPUTS.
Latency scores NOT IN INPUTS so do not invent 412ms latency. Second session cue not invented.

4. Project caution. ProjectLabel Cedar Pier LLM observability. Version Langfuse. Score pack UNKNOWN. Do not invent quality claim packs.

5. Refuse. 412ms latency scores: refused. accuracy ranks: refused. eval scoreboards: refused. guaranteed quality claims: refused. Unreleased AI quality coach: refused.

6. Compliance. Banned hits none. Traces 2. Datasets named 2. Format ledger+trace-dataset-eval checklist+session sketch+project caution+refuse+compliance. Gaps: Quay Storm Tool Trace eval, Quay Storm Tool Trace session, Span pack, Prompt pack, Score pack.

Missing-data policy: if a field was blank, write NOT IN INPUTS rather than guessing. Lock any tool version named in Inputs; if unnamed, write unknown. No invented testimonials, star ratings, or press logos. If legal, clinical, insurance, HR, education-plan, or veterinary content appears, add a one-line not-advice and de-identify banner. Quote banned-word hits and cut them. End with a gaps list of five bullets the user still owes you. Character and byte caps in the job are hard; print counts when relevant. Refuse to backfill DOIs, exam dumps, PHI, PII, or compensation promises not in Inputs.