LangSmith Run Trace Dataset Map from Project Inventory (No Invented Token Totals)

Prompt

Act as a LangSmith run trace-dataset cartographer who only uses a pasted project inventory. You map runs, traces, and datasets the project already supports. You do not invent token totals, latency averages, evaluator scores, or cost attributions. This is not a live production eval and not a billing audit.
You work only from Inputs. Do not invent stats, citations, quotes, URLs, names, IDs, or records that are not in Inputs.

Inputs:
- Project inventory I lock (runs, traces, datasets, evaluators): [Inventory]
- LangSmith version or project notes I lock: [Version]
- Project or workspace label I may quote (or UNKNOWN): [ProjectLabel]
- Run names I may quote (or UNKNOWN): [RunNames]
- Trace titles already present (or UNKNOWN): [TraceTitles]
- Dataset cues already present (or UNKNOWN): [DatasetCues]
- Evaluator cues already present (or UNKNOWN): [EvaluatorCues]
- Words I must not use: [Banned]
- What I must never invent (token totals, latency averages, evaluator scores, cost attributions): [Never]
- Output format: [Format]
- Language: [Lang]

Generate:
1. Honesty ledger: Inventory nouns, Version, ProjectLabel, RunNames, TraceTitles, DatasetCues, EvaluatorCues, Lang. Forbidden: invented token totals, latency averages, evaluator scores, cost attributions.
2. Run-to-trace map: one row per RunNames entry. Attach only TraceTitles named beside that run in Inventory. Missing trace write NOT IN INPUTS.
3. Dataset sketch table: for each DatasetCues entry, list runs that name it. Do not invent 1.2M tokens if absent.
4. Evaluator caution block: quote EvaluatorCues only. Score rubrics not in Inventory stay NOT IN INPUTS; do not invent LLM-as-judge defaults.
5. Refuse list: inventing 1.2M tokens, inventing 420ms p50, inventing score 0.94, inventing $4.2k attributed cost.
6. Compliance pass: quote Banned and Never hits. Cut them. Print run and trace counts from Inventory only. Format as Format.

Constraints:
- Run trace dataset map from Inventory only. No invented token totals.
- Honor Version. No emojis. Not a live production eval.

Instructions

Replace every [bracket] with your details before running. Works on ChatGPT, Claude, and Gemini.

Generated Output

This image was generated using the prompt above.

LangSmith Run Trace Dataset Map from Project Inventory (No Invented Token Totals) - Result

Examples

Example Input

Inventory: run Tide Lattice Smoke as pasted trace Quay Chain Span as pasted; run Tide Lattice Canary as pasted trace Pier Retriever Span as pasted. Token invent NONE. Latency invent NONE.
Version: LangSmith 0.1 as pasted (do not invent unreleased AI dataset writer)
ProjectLabel: Tide Lattice project as pasted
RunNames: Tide Lattice Smoke as pasted; Tide Lattice Canary as pasted. Tide Epilogue UNKNOWN.
TraceTitles: Quay Chain Span as pasted; Pier Retriever Span as pasted. Closing span UNKNOWN.
DatasetCues: dataset quay-gold as pasted for Tide Lattice Smoke; Canary dataset NOT IN INPUTS.
EvaluatorCues: evaluator exact-match as pasted. LLM-as-judge UNKNOWN.
Banned: 1.2M tokens, score 0.94, guaranteed production ready
Never: invent token totals, invent latency averages, invent evaluator scores, invent cost attributions
Format: ledger + run-trace map + dataset sketch + evaluator caution + refuse + compliance
Lang: English

Example Output

1. Ledger. Inventory: run Tide Lattice Smoke + trace Quay Chain Span; run Tide Lattice Canary + trace Pier Retriever Span. Token invent NONE. Latency invent NONE. Version LangSmith 0.1. ProjectLabel Tide Lattice project. RunNames Tide Lattice Smoke; Tide Lattice Canary. TraceTitles Quay Chain Span; Pier Retriever Span. DatasetCues quay-gold for Tide Lattice Smoke; Canary dataset NOT IN INPUTS. EvaluatorCues exact-match. LLM-as-judge UNKNOWN. Lang English. Forbidden: invented token totals, latency averages, evaluator scores, cost attributions, 1.2M tokens, score 0.94, guaranteed production ready.

2. Run-to-trace map.
Tide Lattice Smoke | trace Quay Chain Span as pasted.
Tide Lattice Canary | trace Pier Retriever Span as pasted.
Closing span not attached. Tide Epilogue not added.

3. Dataset sketch table.
dataset quay-gold | run Tide Lattice Smoke as pasted.
Canary dataset | NOT IN INPUTS.
Token goals NOT IN INPUTS so do not invent 1.2M tokens. Second lattice dataset not invented.

4. Evaluator caution. exact-match as pasted. LLM-as-judge UNKNOWN. Default judge rubric NOT IN INPUTS. Do not invent scoring policy.

5. Refuse. 1.2M tokens: refused. 420ms p50: refused. score 0.94: refused. $4.2k attributed cost: refused. Unreleased AI dataset writer: refused.

6. Compliance. Banned hits none. Runs 2. Traces 2. Format ledger+run-trace map+dataset sketch+evaluator caution+refuse+compliance. Gaps: Canary dataset, Closing span, Tide Epilogue, LLM-as-judge, token goals.

Missing-data policy: if a field was blank, write NOT IN INPUTS rather than guessing. Lock any tool version named in Inputs; if unnamed, write unknown. No invented testimonials, star ratings, or press logos. If legal, clinical, insurance, HR, education-plan, or veterinary content appears, add a one-line not-advice and de-identify banner. Quote banned-word hits and cut them. End with a gaps list of five bullets the user still owes you. Character and byte caps in the job are hard; print counts when relevant. Refuse to backfill DOIs, exam dumps, PHI, PII, or compensation promises not in Inputs.