#data annotation

Label Studio Labeling Project Builder for LLM Assisted Annotation: Labeling Config XML with Choices, Labels, and TextArea Tags, Annotator Guidelines, Pre-annotation predictions JSON with model_version, Gold Task Review, and Export Checks

Set up a Label Studio project where an LLM drafts labels and people correct them: write the labeling config XML from your label schema, turn edge cases into annotator guidelines, convert model output into the predictions import format with from_name, to_name, type, and value, seed gold tasks to catch drift, and check the export before training on it.

Act as an ML data operations lead who runs annotation projects in Label Studio, imports LLM drafted labels as predictions so annotators correct instead of start from blank, and has thrown away a week of labels because the config names did not match the import file. Inputs: - What is being labeled (text, chat transcripts, images, PDFs as images) and the data field names in each task: [DataSchema] - The label schema: classes or entity types, single or multi choice, required fields, free text fields: [LabelSchema] - Edge cases and disagreements seen so far, with real examples: [EdgeCases] - How the LLM drafts labels today (model, prompt, raw output sample): [LlmOutput] - Team size, Label Studio edition (Community or Enterprise), and review process: [TeamSetup] - Output format: [Format] Generate: 1. A labeling config XML built from DataSchema and LabelSchema: an object tag (Text, HyperText, or Image) whose value is $field, then control tags (Choices with choice="single" or "multiple", Labels for spans, TextArea for notes) with a unique name and the correct toName, required="true" where the schema says so, and hotkeys for the most used classes. 2. A name map table: every control tag name, its toName, its result type (choices, labels, textarea, rectanglelabels), and the value shape it expects. This table is the contract the import file must follow. 3. Annotator guidelines from EdgeCases: one definition per class, a "use this, not that" line for each confusable pair, a rule for when to pick an abstain class, and two short worked examples per hard case. Keep it short enough to paste into the project instructions. 4. A converter spec that turns LlmOutput into a Label Studio tasks file: each task with data matching DataSchema, a predictions list with model_version and an optional score, and result items with from_name, to_name, type, and value exactly as the name map says. For span labels include start, end, text, and labels, with offsets checked against the source text. 5. A small converter script in Python that reads the LLM output, validates every label against LabelSchema, drops or flags invalid ones instead of guessing, and writes the import JSON. 6. A gold task plan: a set of tasks with known answers mixed into the queue, how they are labeled so reviewers can find them, and what to do when an annotator or the LLM disagrees with gold. 7. Review and agreement: based on TeamSetup, either the Enterprise overlap and agreement settings or a Community workaround (duplicate a sample of tasks, export, and compute Cohen's kappa offline). 8. Export checks before training: JSON export, count of tasks with zero annotations, labels outside the schema, prediction accepted without change versus edited, and a note on keeping model_version so drafted labels can be audited later. Constraints: - Use only fields and classes given in DataSchema and LabelSchema. Never invent classes. - Mark any feature you cannot confirm for the stated edition as "check your Label Studio version". No em dashes.