🤖 AI Tools
Langfuse Trace Quality Checklist from Eval Inventory (No Invented Latency Ms)
Turn a Langfuse eval inventory into a trace quality checklist only. No invented latency ms, token costs, or API keys beyond the inventory.
0Reviews
Prompt
Act as a Langfuse eval QA lead who only uses a pasted eval inventory. You write a trace quality checklist the inventory already supports. You do not invent latency ms, token costs, API keys, or score thresholds not in Inputs. This is not a Langfuse billing estimate and not a latency SLA certificate. You work only from Inputs. Do not invent stats, citations, quotes, URLs, names, IDs, or records that are not in Inputs. Inputs: - Eval trace inventory I lock (trace stubs, span cues, score notes): [Inventory] - Langfuse / project version notes I lock: [Version] - Project or dataset label I may quote (or UNKNOWN): [Workspace] - Required checklist or dataset names I may quote (or UNKNOWN): [SpecNames] - Trace titles already present (or UNKNOWN): [TraceCues] - Span and observation cues already present (or UNKNOWN): [SpanCues] - Score and rubric notes already present (or UNKNOWN): [ScoreNotes] - Words I must not use: [Banned] - What I must never invent (latency ms, token costs, API keys, score thresholds): [Never] - Output format: [Format] - Language: [Lang] Generate: 1. Honesty ledger: Inventory nouns, Version, Workspace, SpecNames, TraceCues, SpanCues, ScoreNotes, Lang. Forbidden: invented latency ms, token costs, API keys, score thresholds. Banner: not a Langfuse billing estimate; not a latency SLA certificate. 2. Quality checklist: one numbered check item per Inventory trace stub or span cue. Missing ScoreNotes write NOT IN INPUTS. Use Langfuse Traces, Spans, Scores, Datasets, and Observations language when Inventory supports it. Never print latency-ms or token-cost VALUES not in Inventory. 3. Spec name set: only names in SpecNames. Unnamed traces stay NOT IN INPUTS. Never invent API-key VALUES not in Inventory. 4. Version lock: print Version. Refuse Langfuse features newer than Version if Version is named. 5. Refuse list: inventing latency ms, inventing token costs, inventing API keys, inventing score thresholds. 6. Compliance pass: quote Banned and Never hits. Cut them. Format as Format. Constraints: - Checklist from Inventory only. No invented latency-ms VALUES. Teach Langfuse trace quality checking, not a LangSmith or Promptfoo swap. - Honor Version. No emojis.
Instructions
Replace every [bracket] with your details before running. Works on ChatGPT, Claude, and Gemini.
Generated Output
This image was generated using the prompt above.
Examples
Example Input
Inventory: trace Harbor Quay Chat Turn as pasted span cue retrieval.embed as pasted score notes faithfulness as pasted; trace River Ops Tool Call as pasted span cue tool.weather as pasted. Latency invent NONE. Token invent NONE. Version: Langfuse 2.x as pasted (do not invent unreleased AI judge writer) Workspace: Harbor Quay eval project as pasted SpecNames: Chat Turn quality checklist as pasted; Tool Call quality checklist as pasted. API key UNKNOWN. TraceCues: Chat Turn as pasted; Tool Call as pasted. Batch Job UNKNOWN. SpanCues: retrieval.embed as pasted; tool.weather as pasted. rerank UNKNOWN. ScoreNotes: faithfulness as pasted. Latency ms VALUES NOT IN INPUTS. Banned: guaranteed latency, latency invent, token invent, key invent Never: invent latency ms, invent token costs, invent API keys, invent score thresholds Format: ledger + checklist + specs + version + refuse + compliance Lang: English
Example Output
1. Ledger. Inventory: trace Harbor Quay Chat Turn span cue retrieval.embed score notes faithfulness; trace River Ops Tool Call span cue tool.weather. Latency invent NONE. Token invent NONE. Version Langfuse 2.x. Workspace Harbor Quay eval project. SpecNames Chat Turn quality checklist; Tool Call quality checklist; API key UNKNOWN. TraceCues Chat Turn; Tool Call; Batch Job UNKNOWN. SpanCues retrieval.embed; tool.weather; rerank UNKNOWN. ScoreNotes faithfulness; Latency ms VALUES NOT IN INPUTS. Lang English. Banner: not a Langfuse billing estimate; not a latency SLA certificate. Forbidden: invented latency ms, token costs, API keys, score thresholds, guaranteed latency, latency invent, token invent, key invent. 2. Quality checklist. 1. Harbor Quay Chat Turn | TraceCues Chat Turn as pasted | SpanCues retrieval.embed as pasted | ScoreNotes faithfulness as pasted | latency VALUES NONE | tokens NONE | Langfuse trace quality checklist only. 2. River Ops Tool Call | TraceCues Tool Call as pasted | SpanCues tool.weather as pasted | ScoreNotes beyond faithfulness NOT IN INPUTS | latency VALUES NONE | tokens NONE. API key UNKNOWN so write API key NOT IN INPUTS. Batch Job not invented beyond TraceCues. 3. Spec name set. Chat Turn quality checklist; Tool Call quality checklist as SpecNames. API key UNKNOWN so write API key NOT IN INPUTS. No latency-ms VALUES beyond Inventory. No third trace invented. 4. Version lock. Langfuse 2.x as pasted. Unreleased AI judge writer not used. Score threshold packs NOT IN INPUTS. 5. Refuse. Latency invent: refused. Token invent: refused. Key invent: refused. Threshold invent: refused. Guaranteed latency: refused. 6. Compliance. Banned hits none. Format ledger+checklist+specs+version+refuse+compliance. Gaps: API key policy, Batch Job TraceCues, rerank SpanCues, latency ms VALUES ScoreNotes, dataset split labels if any. Missing-data policy: if a field was blank, write NOT IN INPUTS rather than guessing. Lock any tool version named in Inputs; if unnamed, write unknown. No invented testimonials, star ratings, or press logos. If legal, clinical, insurance, HR, education-plan, or veterinary content appears, add a one-line not-advice and de-identify banner. Quote banned-word hits and cut them. End with a gaps list of five bullets the user still owes you. Character and byte caps in the job are hard; print counts when relevant. Refuse to backfill DOIs, exam dumps, PHI, PII, or compensation promises not in Inputs.