#local llm

llama.cpp GBNF Grammar Writer: Force a Local Model to Return Valid Invoice Extraction JSON

Write a GBNF grammar for llama.cpp that forces a local GGUF model to emit JSON matching your invoice extraction schema: fixed key order, bounded strings and whitespace, enums, dates, nullable fields, and the commands to run it with llama-cli or llama-server, plus the validation that still has to happen in code.

Act as a machine learning engineer who runs local GGUF models with llama.cpp in document extraction pipelines and writes GBNF grammars by hand. You know a grammar controls the shape of the output, not whether the values are true, and you design both halves. Inputs: - Target JSON schema or a sample of the exact JSON wanted: [JsonSchema] - Field rules (required, nullable, enums, date format, max lengths, max array items): [FieldRules] - How llama.cpp is run (llama-cli, llama-server /completion, or the OpenAI compatible endpoint) and the build date if known: [LlamaCppSetup] - Two short sample documents the model will read: [SampleDocs] - Downstream code that consumes the JSON (language, validator): [Consumer] - Output format: [Format] - Language: [Lang] Generate: 1. Grammar design notes: fixed key order from JsonSchema, which fields are nullable, how enums become literal alternatives, and why strings, arrays, and whitespace get upper bounds (an unbounded ws rule lets the model pad forever). 2. The .gbnf file. Start with root ::= and define rules for objects, arrays, string, number, integer, date, enums, and ws. Write literal quotes as "\"" and use a character class that excludes quote, backslash, and control characters for string content. Use {m,n} repetition for bounds and note that older builds may not support it. 3. Run commands for LlamaCppSetup: llama-cli with --grammar-file, or the grammar field in a llama-server /completion request body. Mention --json-schema and the json_schema_to_grammar.py script as the alternative when the schema changes often, and say to confirm paths in the current repo. 4. Prompt text that goes with the grammar: a short instruction plus the document, telling the model to use null when a field is not on the page. 5. Validation in Consumer: JSON parse, schema validation, and business checks the grammar cannot do (line items times price against total, date is a real calendar date). 6. Test plan against SampleDocs: what to look for, including a document with a missing field to confirm null is produced instead of a guess. Constraints: - The grammar must accept every valid output and nothing else; walk through one sample output against the rules. - Do not claim the grammar improves accuracy; it only guarantees parseable structure. - No em dashes.