#qlora

Axolotl QLoRA Fine Tuning Config Builder: Chat Template Dataset Mapping, Assistant Only Loss, Sample Packing, Batch Math for One GPU, Preprocess Checks, and Merge Commands

Build an Axolotl YAML config for a QLoRA fine tune of an open instruct model on one GPU: a dataset format check for chat_template messages, assistant only training, LoRA and quantization settings, sequence length and sample packing, effective batch and step math, out of memory fallbacks, the preprocess, train, inference, and merge commands, and an evaluation plan against the base model.

Act as an ML engineer who fine tunes open weight models with Axolotl for small teams, and who writes configs that run the first time on the hardware the team actually has. Inputs: - Base model from the Hugging Face Hub and its license terms you have accepted: [BaseModel] - GPU model, count, and VRAM: [GPU] - The task the tuned model should do and the exact output it should produce: [TaskGoal] - Two or three real dataset rows as they appear in the file: [DatasetSample] - Row count and typical length of a conversation: [DatasetSize] - How success will be judged: [EvalPlan] - Output format: [Format] Generate: 1. A dataset check: confirm DatasetSample is JSONL with a messages list of role and content turns. Flag rows with missing roles, empty assistant turns, or outputs that do not match TaskGoal, and give a one line fix for each problem. 2. A complete Axolotl YAML: base_model, load_in_4bit with adapter: qlora, lora_r, lora_alpha, lora_dropout, lora_target_linear, a datasets entry with type: chat_template, field_messages, roles_to_train set to assistant and train_on_eos: turn, dataset_prepared_path, val_set_size, output_dir, sequence_len chosen from DatasetSize, sample_packing, micro_batch_size, gradient_accumulation_steps, num_epochs, learning_rate, optimizer, lr_scheduler, warmup, bf16, gradient_checkpointing, flash_attention, evals_per_epoch, saves_per_epoch. 3. Batch math: effective batch = micro_batch_size x gradient_accumulation_steps x GPU count, and steps per epoch from the training rows before packing. Note that packing lowers the step count and that Axolotl reports the real number after preprocessing. 4. Out of memory fallbacks for GPU in order: smaller micro_batch_size with more accumulation, shorter sequence_len, then a smaller base model. 5. Commands: axolotl preprocess with --debug to confirm only assistant tokens carry labels, axolotl train, axolotl inference with --lora-model-dir, and axolotl merge-lora. 6. An evaluation plan from EvalPlan: a held out test file that never appears in training, the same system prompt for base and tuned model, and the metric for each output field. Constraints: - Do not invent accuracy gains, loss values, or training times. Say what to measure instead. - Keep personal data out of training rows, or say it must be removed first. - Check key names against the Axolotl version installed, since options are renamed between releases. No em dashes.