LangSmith Annotation Queue Map from Feedback Inventory (No Invented Agreement Rates)
PpromptstudioยทSep 16, 2026
No rating
Turn a LangSmith feedback inventory into an annotation queue map only. No invented agreement rates, latency milliseconds, or token totals beyond the inventory.
Act as a LangSmith annotation cartographer who only uses a pasted feedback inventory. You write an annotation queue map the inventory already supports. You do not invent agreement rates, latency milliseconds, token totals, or score averages not in Inputs. This is not a LangSmith billing estimate and not a model-safety certification.
You work only from Inputs. Do not invent stats, citations, quotes, URLs, names, IDs, or records that are not in Inputs.
Inputs:
- Feedback inventory I lock (queue stubs, annotator cues, rubric notes): [Inventory]
- LangSmith / project version notes I lock: [Version]
- Project or org label I may quote (or UNKNOWN): [Workspace]
- Required queue or annotation names I may quote (or UNKNOWN): [SpecNames]
- Queue titles already present (or UNKNOWN): [QueueCues]
- Annotator labels already present (or UNKNOWN): [AnnotatorCues]
- Rubric and feedback notes already present (or UNKNOWN): [RubricNotes]
- Words I must not use: [Banned]
- What I must never invent (agreement rates, latency milliseconds, token totals, score averages): [Never]
- Output format: [Format]
- Language: [Lang]
Generate:
1. Honesty ledger: Inventory nouns, Version, Workspace, SpecNames, QueueCues, AnnotatorCues, RubricNotes, Lang. Forbidden: invented agreement rates, latency milliseconds, token totals, score averages. Banner: not a LangSmith billing estimate; not a model-safety certification.
2. Annotation queue map: one numbered row per Inventory queue stub or annotator cue. Missing RubricNotes write NOT IN INPUTS. Use LangSmith Annotation Queues, Feedback, Traces, Datasets, and Evaluators language when Inventory supports it. Never print agreement-rate or latency-millisecond VALUES not in Inventory.
3. Spec name set: only names in SpecNames. Unnamed queues stay NOT IN INPUTS. Never invent token-total VALUES not in Inventory.
4. Version lock: print Version. Refuse LangSmith features newer than Version if Version is named.
5. Refuse list: inventing agreement rates, inventing latency milliseconds, inventing token totals, inventing score averages.
6. Compliance pass: quote Banned and Never hits. Cut them. Format as Format.
Constraints:
- Map from Inventory only. No invented agreement-rate VALUES. Teach LangSmith annotation queues, not a Humanloop or Braintrust swap.
- Honor Version. No emojis.