How to Use the Axolotl QLoRA Config Builder Prompt to Fine Tune an Open Model on One GPU the First Time
Build an Axolotl YAML config for a QLoRA fine tune of an open instruct model on one GPU: a dataset format check for chat_template messages, assistant only training, LoRA and quantization settings, sequence length and sample packing, effective batch and step math, out of memory fallbacks, the preprocess, train, inference, and merge commands, and an evaluation plan against the base model.
Axolotl makes it possible to fine tune an open weight model with a single YAML file, but that file has dozens of options and a small mistake can waste hours. Training on the wrong tokens, a sequence length that wastes memory, or a dataset field name that does not match your file are all common first run problems. The Axolotl QLoRA Fine Tuning Config Builder: Chat Template Dataset Mapping, Assistant Only Loss, Sample Packing, Batch Math for One GPU, Preprocess Checks, and Merge Commands prompt builds a QLoRA config around your actual GPU and data, explains the batch math, and gives you the commands and checks to confirm it is training on the right thing.
What the prompt produces
- A dataset check that confirms your JSONL rows use a messages list and flags rows that would hurt training.
- A complete Axolotl YAML with QLoRA, chat template dataset mapping, assistant only loss, and sample packing.
- Batch math for effective batch size and steps per epoch.
- Out of memory fallbacks in a safe order.
- Commands to preprocess, train, test, and merge.
- An evaluation plan that compares the tuned model with the base model on held out data.
How to fill the inputs
BaseModel is the Hugging Face model ID. Make sure you have accepted its license before training.
GPU lists the card, how many you have, and the VRAM. This decides batch size, sequence length, and whether 4 bit loading is needed.
TaskGoal describes exactly what the tuned model should output. The narrower the output, the easier it is to evaluate.
DatasetSample is two or three real rows pasted as they appear in the file. The prompt checks field names and roles against them.
DatasetSize gives the row count and typical length, which sets sequence length and step math.
EvalPlan says how success will be judged, such as JSON validity and label accuracy on a labeled test set.
Reading the example output
The example fine tunes a 7B instruct model on one 24 GB card to route support tickets into JSON with a queue and a priority:
- The data is checked first. The sample row has system, user, and assistant turns with JSON only output, and the prompt says to remove personal data before training.
- Loss is limited to assistant turns. Setting roles_to_train to assistant and train_on_eos to turn means the model learns to answer, not to repeat tickets.
- Sequence length fits the data. Tickets are short, so 1024 tokens is enough and packing fits several examples per sequence.
- Batch math is shown. Two examples per step times eight accumulation steps gives an effective batch of 16, with the step count before packing worked out from the training rows.
- Memory has a plan. If training runs out of memory, the first fallback keeps the same effective batch by halving the micro batch and doubling accumulation.
- Preprocessing is verified. Running preprocess with the debug flag lets you confirm that only assistant tokens carry labels.
- Evaluation is fair. The base and tuned models get the same system prompt on 300 held out tickets, with a confusion table for queues.
Tips for better results
- Start with a small subset and one epoch to confirm everything runs, then scale up.
- Keep the test file completely separate from training data and check for duplicates.
- Save the exact config and Axolotl version with every run so results are reproducible.
- Compare against the base model with a good system prompt. Sometimes prompting alone is enough.
Mistakes to avoid
- Do not trust config keys from an old tutorial without checking your installed version. Options get renamed.
- Do not train on user turns for an extraction or routing task.
- Do not report accuracy from the validation split you tuned settings on. Use the held out test file.
- Do not include customer names, emails, or order numbers in training rows.
Who it is for
ML engineers at small companies, developers building internal tools on open models, researchers running their first fine tune, and teams deciding whether fine tuning beats prompting for a narrow task.
Related PromptDig links
Open the Axolotl QLoRA Fine Tuning Config Builder: Chat Template Dataset Mapping, Assistant Only Loss, Sample Packing, Batch Math for One GPU, Preprocess Checks, and Merge Commands prompt and paste your dataset sample and GPU details. For more prompts for AI tools and model workflows, Browse more prompts. If you have a fine tuning or model evaluation prompt that works, Share a prompt.