Browse Prompts
224 prompts available in ai-tools · Page 2 of 19
Claude Extended Thinking Budget Planner: budget_tokens vs max_tokens, Tool Use with Thinking Blocks Passed Back, Streaming Thresholds, Caching Effects, and a Cost Test Plan
Plan how to turn on extended thinking in the Claude API for a real feature: choose a budget_tokens and max_tokens pair per task, keep thinking blocks intact across tool calls, know which parameters stop working, decide when to stream, and test cost and quality before rollout.
Act as an LLM platform engineer who adds extended thinking to Claude API features in production, sets thinking budgets per task from test data, and keeps tool use, caching, and billing behavior correct. Inputs: - The feature and its task types (for example contract clause analysis, multi step data question answering, code review), with current model and settings: [FeatureAndTasks] - Current request shape: system prompt size, typical input tokens, max_tokens, temperature, tool definitions, and tool_choice: [CurrentRequest] - Whether the feature uses tool calls in a loop and how the conversation history is stored: [ToolLoop] - Latency and cost limits per request and per month: [Limits] - Whether prompt caching is enabled and where the cache breakpoints are: [CachingSetup] - Output format: [Format] Generate: 1. A task by task decision on whether extended thinking is worth enabling for FeatureAndTasks, with the reason. 2. A budget table per task: starting budget_tokens, max_tokens, and why, keeping budget_tokens at the documented minimum or above and below max_tokens unless interleaved thinking applies [confirm in Anthropic docs]. 3. Parameter conflicts in CurrentRequest that change when thinking is on: temperature and top_k changes, forced tool_choice, and assistant prefill, with the fix for each [confirm in Anthropic docs]. 4. A tool loop plan from ToolLoop: pass the thinking blocks, including their signatures, back unchanged in the assistant turn that holds the tool_use, and whether interleaved thinking between tool calls is needed. 5. A streaming rule: when max_tokens is large enough that streaming should be used, and how the UI handles thinking deltas. 6. A caching note from CachingSetup: what changing the thinking budget does to cached prefixes and where to keep breakpoints. 7. A billing note: thinking tokens are billed as output tokens, and on newer models the returned thinking may be a summary while billing covers the full thinking [confirm per model]. 8. A cost and quality test plan against Limits: a fixed eval set, two or three budgets per task, and the metrics to log (usage.output_tokens, latency, pass rate). Constraints: - Do not invent prices, model names, or limits; mark them [confirm in Anthropic docs]. - Never suggest editing or removing thinking blocks mid tool loop. - Plain engineer tone. No em dashes.
OpenAI Chat Completions to Responses API Migration Planner: messages to input Map, instructions Field, Function Tool Format Changes, Structured Output text.format, and State Choices
Plan a move from the OpenAI Chat Completions API to the Responses API: map messages to input and instructions, convert function tool definitions and tool call handling, move structured outputs to text.format, decide on previous_response_id and store, and set up a side by side test before switching traffic.
Act as an LLM platform engineer who migrates production code from OpenAI Chat Completions to the Responses API, rewrites request and response handling, converts tool calling and structured outputs, and runs a shadow comparison before switching traffic. Inputs: - Language and SDK version (Python openai, Node openai, raw HTTP): [SdkAndLanguage] - Current Chat Completions request code or a redacted sample including messages, model, temperature, tools, and response_format: [CurrentRequest] - How the code reads the result today (choices[0].message.content, tool_calls, finish_reason, usage): [ResponseHandling] - Tools in use: your own function tools, and any wish to use built in tools such as web search or file search: [ToolsInUse] - Conversation state today (resend full history, a database of turns) and data retention needs: [StateAndRetention] - Traffic and risk tolerance for the cutover: [CutoverRisk] - Output format: [Format] Generate: 1. A field map from CurrentRequest: messages to input items, system or developer message to instructions, max_tokens to max_output_tokens, response_format to text.format, and any fields with no direct equivalent marked [confirm in API reference]. 2. A rewritten request for SdkAndLanguage using client.responses.create. 3. A response handling map from ResponseHandling: output_text for plain text, output items for function calls, status and incomplete details instead of finish_reason, and usage field names. 4. A function tool conversion for ToolsInUse: flattened definitions with type, name, description, parameters, and strict, plus how to return results as function_call_output items with the matching call_id. 5. A state decision from StateAndRetention: keep sending full history, or use previous_response_id with store, and what that means for retention. 6. A built in tools note if ToolsInUse asks for them, with cost and data handling questions to confirm. 7. A shadow test plan for CutoverRisk: send a sample of real requests to both APIs, compare outputs, tool call accuracy, latency, and token usage, then ramp traffic. 8. A rollback switch: a feature flag that routes back to Chat Completions. Constraints: - Do not invent parameter names, model names, prices, or deprecation dates. Mark uncertain ones [confirm in API reference]. - Keep any secret keys out of the examples. - Plain engineer tone. No em dashes.
OpenAI Structured Outputs Schema Designer: JSON Schema Strict Mode, Refusal Paths, Pydantic Models, and Evaluation Fixtures
Design an OpenAI Structured Outputs setup for a production extractor: a strict JSON Schema, matching Pydantic models, refusal and incompleteness paths, and a small fixture set so you can regression test parsing before you wire the schema into the API.
Act as an applied LLM engineer who ships OpenAI Structured Outputs in strict mode, writes JSON Schema that mirrors Pydantic models, and builds tiny evaluation fixtures before production traffic. Inputs: - Extraction or routing job the model must do: [JobBrief] - Fields that must always appear, optional fields, and enums: [FieldSpec] - Model family you plan to call (name only, no invented version guarantees): [ModelFamily] - Failure modes you have already seen (hallucinated IDs, empty arrays, wrong enum): [FailureModes] - Runtime language (Python Pydantic v2, TypeScript Zod, or JSON Schema only): [Runtime] - Output format: [Format] Generate: 1. A JSON Schema draft with "$schema", "type":"object", "additionalProperties": false, required arrays, and typed properties matching FieldSpec. 2. Strict mode notes: which fields are required vs nullable, and how to represent "unknown" without inventing data. 3. A parallel model in Runtime (Pydantic v2 or Zod) that matches the schema field for field. 4. Tool or response_format wiring sketch using ModelFamily with placeholders for API parameters you are unsure of marked [confirm in OpenAI docs]. 5. Refusal and incompleteness paths: when the model should return a structured error object instead of guessing, with a separate small schema or a status enum inside the main schema. 6. Evaluation fixtures: 3 input snippets and the exact expected JSON objects, plus 1 adversarial input tied to FailureModes. 7. A validation snippet outline: parse, catch schema errors, retry once with a repair instruction, then fail closed. 8. A changelog line for the schema version so product can bump without silent breaks. Constraints: - Do not invent token prices, rate limits, or model release dates. - Prefer enums over free text for categories the product already knows. - Honesty: at most one step that refuses to invent missing identifiers. - No em dashes.
vLLM OpenAI Compatible Server Deployment Planner: vllm serve Flags, max-model-len and gpu-memory-utilization, KV Cache Memory Math, Tensor Parallel Size, Prefix Caching, Quantized Checkpoints, Docker Run, API Key, and a Load Test Plan
Plan a self hosted vLLM endpoint that fits the GPU you actually have: model weight and KV cache memory math, the vllm serve command with each flag justified, tensor parallel choice, when a quantized checkpoint is worth it, a Docker run line, smoke tests against the OpenAI compatible API, a load test, and the metrics to watch once real traffic arrives.
Act as an ML infrastructure engineer who deploys open weight models with vLLM in production, sizes every deployment with memory math before touching a GPU, and treats the startup log as the source of truth. Inputs: - Model repo id, parameter count, and architecture details if known (layers, KV heads, head dim): [ModelChoice] - GPUs: type, count, memory each, and whether they share a node: [GPUInventory] - Workload: typical prompt and output lengths, longest context needed, expected concurrent users, latency target: [WorkloadProfile] - Serving setup: bare metal, Docker, or Kubernetes, and how clients will call it: [ServingSetup] - Constraints such as license gating, data residency, or a required quantization: [ModelConstraints] - Output format: [Format] Generate: 1. Weight memory: parameters times bytes per parameter for the chosen dtype (about 2 bytes for bf16, about 0.5 to 0.6 for 4 bit AWQ or GPTQ checkpoints). 2. KV cache math: bytes per token = 2 x layers x KV heads x head dim x bytes per element; then tokens that fit in the budget left after weights and overhead at the chosen gpu-memory-utilization; then how many full length sequences that is at max-model-len. 3. A decision on tensor-parallel-size from GPUInventory, and whether a quantized checkpoint or fp8 KV cache is needed to meet WorkloadProfile. 4. The vllm serve command with each flag explained: model, served-model-name, max-model-len, gpu-memory-utilization, tensor-parallel-size, max-num-seqs, enable-prefix-caching if prompts share long prefixes, api-key, host, and port. 5. A Docker run line for the vllm/vllm-openai image with GPU access, ipc=host, the Hugging Face cache mounted, and HF_TOKEN for gated models; tell the user to pin an image tag. 6. Smoke tests with curl: /health, /v1/models, one /v1/chat/completions call, and one streaming call. 7. A load test plan: request mix from WorkloadProfile, the benchmark tool shipped with the installed vLLM version, and the numbers to record (time to first token, output tokens per second, failures). 8. What to read in the startup log (KV cache size and maximum concurrency lines) and in /metrics (running and waiting requests, KV cache usage) and what each pattern means. 9. Mark any flag name or metric that changed across vLLM versions as "check vllm serve --help for your version". Constraints: - Show the arithmetic. Commands copy and paste ready. No em dashes.
Claude Code Hooks Config Writer for Team Repos: settings.json PreToolUse and PostToolUse Matchers, Exit Code 2 Blocking, stdin JSON Parsing with jq, Auto Format After Edits, Stop Hook Loop Guard, and a Hook Test Plan
Write Claude Code hooks a team can commit: the right settings file scope, PreToolUse guards that block destructive shell commands and edits to protected paths with exit code 2 and a reason Claude can read, PostToolUse formatting on edited files, a Stop hook that checks work without looping forever, and a plan to test each hook before trusting it.
Act as a developer productivity engineer who maintains Claude Code setups for a team, writes hooks as small shell or Python scripts under .claude/hooks, and treats every hook as code that runs with the developer's own permissions. Inputs: - Repo stack, package manager, formatter, linter, and test or typecheck commands: [RepoStack] - Commands or actions Claude must never run without asking: [BlockedActions] - Files or folders Claude must not edit: [ProtectedPaths] - What should happen automatically after edits or at the end of a task: [Automations] - Scope: personal only, shared with the team, or both: [Scope] - Output format: [Format] Generate: 1. Settings file choice from Scope: user settings in the home .claude folder, project settings in .claude/settings.json committed to git, or .claude/settings.local.json for personal overrides that stay out of git. 2. A hooks block in valid JSON: hook events as keys (PreToolUse, PostToolUse, Stop, and Notification or UserPromptSubmit only if Automations needs them), each with a matcher for tool names such as Bash or Edit|Write|MultiEdit, and command hooks pointing at scripts via the CLAUDE_PROJECT_DIR environment variable. 3. A PreToolUse guard for BlockedActions: read the JSON on stdin with jq, pull tool_input.command, match the dangerous patterns, print a reason to stderr, and exit 2 so the call is blocked and Claude sees why. Exit 0 lets the call through. 4. A path guard for ProtectedPaths on edit tools: read tool_input.file_path, block matches with exit 2 and a clear message. 5. PostToolUse automations from Automations and RepoStack: format only the edited file, keep output quiet, set a timeout, and never fail the session because a formatter is missing. 6. A Stop hook when Automations asks for a final check: run the typecheck or tests, exit 2 with the failures on stderr so Claude keeps working, and read stop_hook_active from stdin so it does not loop forever. 7. Each script in full with a shebang, set options, and a comment on what it reads and returns. Remind the team to make scripts executable. 8. A test plan: pipe sample JSON into each script by hand and check the exit code, then open the hooks menu in Claude Code to confirm they loaded, then try one blocked command in a scratch branch. 9. A security note: hooks run arbitrary commands with your account's access, so review hook changes in pull requests like any other code. Constraints: - Only use hook events, fields, and exit code behavior you are sure of; mark anything else "confirm in the current Claude Code hooks docs". - Quote paths that may contain spaces. No em dashes.
Gemini API Context Caching Planner: Implicit vs Explicit Caching on Gemini 2.5 Models, google-genai caches.create with TTL, Minimum Cacheable Tokens, Stable Prefix Prompt Layout, Storage vs Input Cost Math, and Cache Hit Monitoring
Decide whether and how to cache a large repeated context in the Gemini API: compare implicit caching with explicit caches, lay out prompts so the stable part comes first, create and refresh a cache with the google-genai SDK and a TTL that matches your traffic, check the cost against storage charges using the rates you paste, and confirm hits in usage metadata.
Act as a backend engineer who runs production apps on the Gemini API, has cut monthly token bills by caching large shared contexts, and has debugged the error you get when a request sends a system instruction alongside a cached context. Inputs: - The workload: requests per day, hours when traffic arrives, shared context size in tokens, and typical question size: [Workload] - What repeats across requests (system instruction, manuals, codebase, a long video or PDF) and how often it changes: [SharedContext] - Model and SDK in use: [ModelChoice] - Rates copied from the current Gemini pricing page for input, cached input, and cache storage: [PricingNotes] - Current prompt structure or code snippet: [CurrentCode] - Output format: [Format] Generate: 1. An implicit versus explicit decision: implicit caching is on by default for Gemini 2.5 models and gives a discount only when a request shares a prefix with a recent one, with no guarantee; explicit caching stores the context for a TTL you pay storage for, and gives a predictable discount. Pick one for this Workload and say why. 2. Prompt layout rules for either mode: put the large stable content first and the per request question last, keep the stable part byte identical, and send similar requests close together in time. 3. A minimum size check: explicit caches need a minimum number of input tokens that differs by model; tell the user to measure with client.models.count_tokens and compare against the current minimum for ModelChoice from the docs. 4. Python code with the google-genai SDK: client.caches.create with model, a CreateCachedContentConfig holding display_name, system_instruction, contents, and ttl; then client.models.generate_content with GenerateContentConfig(cached_content=cache.name). Put system_instruction and tools in the cache, not in the request, because the request cannot set them when it uses cached content. 5. TTL and refresh: choose a TTL from the traffic hours in Workload, extend it with client.caches.update when traffic continues, and delete it with client.caches.delete when the day ends. 6. Versioning: caches are not edited in place; create a new cache when SharedContext changes, and put the content version in display_name. 7. Cost math using only PricingNotes: daily cost without caching, with explicit caching (cached tokens at the cached rate plus storage tokens times hours), and the break even request count. 8. Monitoring: log usage_metadata.cached_content_token_count and prompt_token_count per request, alert when the cached share drops, and recreate the cache on a not found error after expiry. Constraints: - Never invent prices or token minimums; use PricingNotes and mark anything else "confirm in the current Gemini docs". - The cache model must match the request model exactly. No em dashes.
Label Studio Labeling Project Builder for LLM Assisted Annotation: Labeling Config XML with Choices, Labels, and TextArea Tags, Annotator Guidelines, Pre-annotation predictions JSON with model_version, Gold Task Review, and Export Checks
Set up a Label Studio project where an LLM drafts labels and people correct them: write the labeling config XML from your label schema, turn edge cases into annotator guidelines, convert model output into the predictions import format with from_name, to_name, type, and value, seed gold tasks to catch drift, and check the export before training on it.
Act as an ML data operations lead who runs annotation projects in Label Studio, imports LLM drafted labels as predictions so annotators correct instead of start from blank, and has thrown away a week of labels because the config names did not match the import file. Inputs: - What is being labeled (text, chat transcripts, images, PDFs as images) and the data field names in each task: [DataSchema] - The label schema: classes or entity types, single or multi choice, required fields, free text fields: [LabelSchema] - Edge cases and disagreements seen so far, with real examples: [EdgeCases] - How the LLM drafts labels today (model, prompt, raw output sample): [LlmOutput] - Team size, Label Studio edition (Community or Enterprise), and review process: [TeamSetup] - Output format: [Format] Generate: 1. A labeling config XML built from DataSchema and LabelSchema: an object tag (Text, HyperText, or Image) whose value is $field, then control tags (Choices with choice="single" or "multiple", Labels for spans, TextArea for notes) with a unique name and the correct toName, required="true" where the schema says so, and hotkeys for the most used classes. 2. A name map table: every control tag name, its toName, its result type (choices, labels, textarea, rectanglelabels), and the value shape it expects. This table is the contract the import file must follow. 3. Annotator guidelines from EdgeCases: one definition per class, a "use this, not that" line for each confusable pair, a rule for when to pick an abstain class, and two short worked examples per hard case. Keep it short enough to paste into the project instructions. 4. A converter spec that turns LlmOutput into a Label Studio tasks file: each task with data matching DataSchema, a predictions list with model_version and an optional score, and result items with from_name, to_name, type, and value exactly as the name map says. For span labels include start, end, text, and labels, with offsets checked against the source text. 5. A small converter script in Python that reads the LLM output, validates every label against LabelSchema, drops or flags invalid ones instead of guessing, and writes the import JSON. 6. A gold task plan: a set of tasks with known answers mixed into the queue, how they are labeled so reviewers can find them, and what to do when an annotator or the LLM disagrees with gold. 7. Review and agreement: based on TeamSetup, either the Enterprise overlap and agreement settings or a Community workaround (duplicate a sample of tasks, export, and compute Cohen's kappa offline). 8. Export checks before training: JSON export, count of tasks with zero annotations, labels outside the schema, prediction accepted without change versus edited, and a note on keeping model_version so drafted labels can be audited later. Constraints: - Use only fields and classes given in DataSchema and LabelSchema. Never invent classes. - Mark any feature you cannot confirm for the stated edition as "check your Label Studio version". No em dashes.
MLX LM LoRA Fine Tuning Planner for Apple Silicon Macs: Quantized Base Model Choice, Chat JSONL Train and Valid Splits, mlx_lm.lora Flags for Memory, Prompt Masking, Test Loss, Adapter Fuse, and Before vs After Checks
Plan a LoRA fine tune of a small open model on your own Mac with mlx-lm: pick a quantized base that fits unified memory, convert your examples into chat JSONL train, valid, and test files, choose mlx_lm.lora flags for memory and loss masking, run evaluation on held out data, fuse the adapter, and compare outputs before and after on the same prompts.
Act as a machine learning engineer who fine tunes small open models on Apple Silicon with Apple's mlx-lm package, runs LoRA on 4 bit models with mlx_lm.lora, and has watched runs crash from memory pressure, learn the system prompt instead of the answer, and look good on training loss while failing held out examples. Inputs: - Mac model, chip, and unified memory: [MacSpecs] - The task in one sentence and what a correct output looks like: [TaskGoal] - Your examples: count, file type, and two real rows: [DatasetSample] - Base model you are considering, or let the plan pick one: [BaseModel] - How you will judge success (exact match on fields, a rubric, a reviewer): [EvalPlan] - Where the model will run afterwards (mlx_lm.generate, a local server, another runtime): [DeployTarget] - Output format: [Format] Generate: 1. A fit check: why the chosen base fits MacSpecs as a 4 bit MLX model from the mlx-community organization, and the smaller fallback if memory pressure appears. Training LoRA on a quantized base is the QLoRA setup. 2. A data plan: convert DatasetSample into chat format JSONL, one object per line with a messages list of system, user, and assistant turns, then split into data/train.jsonl, data/valid.jsonl, and data/test.jsonl with no duplicate rows across files. 3. A conversion script outline in Python that reads the source file, writes the three JSONL files, and prints counts and the longest example in tokens with the model tokenizer. 4. The mlx_lm.lora training command with each flag explained: --model, --train, --data, --batch-size, --num-layers, --iters, --learning-rate, --mask-prompt so loss is computed on the assistant answer only, --grad-checkpoint, --max-seq-length set from the longest example, --steps-per-eval, --val-batches, --save-every, and --adapter-path. 5. Memory moves in order if the run swaps or crashes: lower batch size, keep gradient checkpointing, cut num layers, shorten max sequence length, then switch to the smaller base. 6. Evaluation: the --test run on data/test.jsonl for test loss, plus a scripted pass that generates answers for every test row and scores them by EvalPlan. 7. A before versus after table on the same five test prompts with the base model and with the adapter. 8. Packaging for DeployTarget: mlx_lm.fuse with --save-path, and a note to run --help for current export options such as GGUF, which only some architectures support. Constraints: - Flags change between releases; tell the user to confirm with mlx_lm.lora --help. - Do not promise accuracy numbers or training times. No em dashes.
Speaker Diarized Transcription Pipeline Planner With WhisperX and pyannote speaker-diarization-3.1: Model Size vs VRAM, compute_type, batch_size, Alignment, Speaker Counts, HF Token via Env Var, SRT VTT JSON Outputs, and QA
Plan a WhisperX transcription run with speaker labels: faster-whisper model size and compute type for your GPU, batch size, language and word alignment, pyannote speaker-diarization-3.1 with min and max speakers, gated model access through a Hugging Face token in an environment variable, SRT, VTT, and JSON outputs, a speaker name map, and QA checks for overlaps and very short turns.
Act as a machine learning engineer who runs WhisperX with its faster-whisper backend and pyannote diarization for podcasts, interviews, and meeting archives, and who has fixed CUDA out of memory errors, gated model 401s, and transcripts where every line was labeled the same speaker. Inputs: - Audio: file types, total hours, typical episode length, number of speakers, mics, crosstalk, background music: [AudioSet] - Hardware: GPU model and VRAM, or CPU only, OS, Python and CUDA versions, installed WhisperX version: [Hardware] - Language or languages, accents, and domain terms or names: [LanguageProfile] - Expected speaker count range per file and who the speakers are: [SpeakerInfo] - Outputs needed: SRT, VTT, JSON with word timings and speaker labels, plain text: [OutputSpec] - Accuracy versus speed priority and a deadline: [Priority] - Output format: [Format] Generate: 1. A model plan: faster-whisper model size (for example large-v3, medium, small) and compute_type (float16 on recent NVIDIA GPUs, int8_float16 or int8 to save memory, int8 on CPU) for Hardware. Do not state exact VRAM numbers unless the user supplied them; tell the user to check the project benchmarks and to measure peak memory on one file. 2. batch_size guidance: start value, and the rule to halve it first when CUDA runs out of memory before shrinking the model. 3. Language and alignment: set language explicitly from LanguageProfile instead of auto detect, run the alignment model for that language to get word timings, and note languages without a default alignment model. 4. Diarization: pyannote speaker-diarization-3.1 through WhisperX, min_speakers and max_speakers from SpeakerInfo, then assign word speakers. Access steps: accept the user conditions on the gated model pages for speaker-diarization-3.1 and its segmentation model on Hugging Face, create a read token, and pass it from an environment variable such as HF_TOKEN, never hardcoded. 5. The run: a CLI command and an equivalent Python script with the chosen flags, and a batch loop over AudioSet that skips files already done. 6. Outputs from OutputSpec: SRT and VTT with speaker prefixes, JSON with segments, words, start, end, and speaker, and where each lands. 7. A speaker name map: replace SPEAKER_00 style labels with real names after listening to the first minute, stored per file in a small JSON. 8. QA checks: percent of words with no speaker, turns shorter than about one second, overlapping speech regions, speaker count different from SpeakerInfo, and timestamps that drift; with a fix for each. 9. A troubleshooting table: 401 or gated model error, out of memory, wrong language, one speaker everywhere, music sections hallucinated as text. Constraints: - Function names and defaults change between WhisperX releases; mark such lines CHECK against the installed version. No em dashes.
llama.cpp GGUF Quantization and llama-server Launch Planner: Quant Choice by VRAM, Context Length and KV Cache Budget, GPU Layer Offload, Parallel Slots, Chat Template, and OpenAI Compatible Endpoint Tests
Plan a local model server on your own GPU with llama.cpp: choose a GGUF quantization that fits your VRAM, budget the KV cache for the context length and number of parallel users, set GPU offload, flash attention, and cache type flags, load the right chat template, and test the OpenAI compatible endpoint before pointing apps at it.
Act as a local inference engineer who deploys llama.cpp llama-server on workstations and small servers for teams, and who sizes every launch from the model architecture and the GPU's free memory instead of guessing until it stops crashing. Inputs: - The model and its architecture details from the model card or GGUF metadata (parameters, layers, attention heads, KV heads, head dimension, trained context): [ModelArch] - GPU model, total VRAM, and what else uses it (desktop, other apps); CPU and system RAM: [Hardware] - Context length needed per request and how many concurrent users or requests: [Workload] - Quality priority (coding, extraction, chat) and acceptable speed: [QualityNeeds] - llama.cpp build or release tag and how it was installed: [BuildVersion] - Who connects and from where (localhost only, LAN, an app using the OpenAI SDK): [ClientSetup] - Output format: [Format] Generate: 1. A quantization shortlist (for example Q8_0, Q6_K, Q5_K_M, Q4_K_M, IQ4_XS) with approximate file size for this parameter count, the expected quality tradeoff in plain words, and which one fits QualityNeeds. 2. KV cache math: bytes per token = 2 x layers x KV heads x head dimension x bytes per element, then total for the context in Workload, for f16 and for q8_0 cache types. Use the KV head count, not the attention head count, for grouped query attention models. 3. A VRAM budget table: weights plus KV cache plus a compute buffer allowance against free VRAM, with the recommended quant and context, and what to change first if it does not fit (cache type, context, quant, then partial offload). 4. Parallel slots: how the total context is divided across --parallel slots in most builds, so per user context is clear, and when to raise context instead. 5. The llama-server command with each flag explained: model path, --host and --port, context size, --parallel, GPU layers, flash attention, cache types, --jinja for the model's chat template, an API key, and threads for any CPU part. 6. Endpoint tests with curl: /health, /v1/models, and a /v1/chat/completions request, plus the base_url and key settings for an OpenAI SDK client. 7. Tuning notes: how to read the startup log for offloaded layers and buffer sizes, and the symptom of each common mistake (out of memory at load, gibberish from a wrong template, slow speed from partial offload). Constraints: - Flag names change between releases; tell the user to confirm against llama-server --help for their BuildVersion. - Do not publish benchmark speeds you cannot know; give sizes and math, and label estimates. - Never suggest exposing the server on a public interface without an API key and a firewall. No em dashes.
Axolotl QLoRA Fine Tuning Config Builder: Chat Template Dataset Mapping, Assistant Only Loss, Sample Packing, Batch Math for One GPU, Preprocess Checks, and Merge Commands
Build an Axolotl YAML config for a QLoRA fine tune of an open instruct model on one GPU: a dataset format check for chat_template messages, assistant only training, LoRA and quantization settings, sequence length and sample packing, effective batch and step math, out of memory fallbacks, the preprocess, train, inference, and merge commands, and an evaluation plan against the base model.
Act as an ML engineer who fine tunes open weight models with Axolotl for small teams, and who writes configs that run the first time on the hardware the team actually has. Inputs: - Base model from the Hugging Face Hub and its license terms you have accepted: [BaseModel] - GPU model, count, and VRAM: [GPU] - The task the tuned model should do and the exact output it should produce: [TaskGoal] - Two or three real dataset rows as they appear in the file: [DatasetSample] - Row count and typical length of a conversation: [DatasetSize] - How success will be judged: [EvalPlan] - Output format: [Format] Generate: 1. A dataset check: confirm DatasetSample is JSONL with a messages list of role and content turns. Flag rows with missing roles, empty assistant turns, or outputs that do not match TaskGoal, and give a one line fix for each problem. 2. A complete Axolotl YAML: base_model, load_in_4bit with adapter: qlora, lora_r, lora_alpha, lora_dropout, lora_target_linear, a datasets entry with type: chat_template, field_messages, roles_to_train set to assistant and train_on_eos: turn, dataset_prepared_path, val_set_size, output_dir, sequence_len chosen from DatasetSize, sample_packing, micro_batch_size, gradient_accumulation_steps, num_epochs, learning_rate, optimizer, lr_scheduler, warmup, bf16, gradient_checkpointing, flash_attention, evals_per_epoch, saves_per_epoch. 3. Batch math: effective batch = micro_batch_size x gradient_accumulation_steps x GPU count, and steps per epoch from the training rows before packing. Note that packing lowers the step count and that Axolotl reports the real number after preprocessing. 4. Out of memory fallbacks for GPU in order: smaller micro_batch_size with more accumulation, shorter sequence_len, then a smaller base model. 5. Commands: axolotl preprocess with --debug to confirm only assistant tokens carry labels, axolotl train, axolotl inference with --lora-model-dir, and axolotl merge-lora. 6. An evaluation plan from EvalPlan: a held out test file that never appears in training, the same system prompt for base and tuned model, and the metric for each output field. Constraints: - Do not invent accuracy gains, loss values, or training times. Say what to measure instead. - Keep personal data out of training rows, or say it must be removed first. - Check key names against the Axolotl version installed, since options are renamed between releases. No em dashes.
ComfyUI SDXL Workflow Builder: Node by Node Graph with Checkpoint, LoRA, KSampler Settings, Base and Refiner Step Split, Upscale Pass, and API Format Workflow JSON
Plan a reproducible ComfyUI graph for SDXL before you start dragging wires: a core node list in build order, a wiring map of MODEL, CLIP, VAE, CONDITIONING, and LATENT connections, KSamplerAdvanced settings for a base and refiner step split, an SDXL resolution, an upscale pass, a VRAM plan, and an API format workflow JSON you can queue from a script.
Act as a ComfyUI workflow builder who designs reproducible SDXL node graphs for studios and hobbyists running local GPUs. You wire graphs from the core nodes first and add custom nodes only when the core set cannot do the job. Inputs: - The image goal: subject, style, aspect ratio, and number of variations: [ImageGoal] - Model files exactly as named in the models folders (checkpoints, refiner, VAE, LoRAs with trigger words, upscale models): [ModelFiles] - GPU, VRAM, and whether ComfyUI runs as portable, desktop, or a server: [Hardware] - Positive and negative prompt text the user already likes: [PromptText] - Custom nodes installed through ComfyUI Manager, if any: [CustomNodes] - Whether the workflow runs in the browser or is queued through the HTTP API: [RunMode] - Output format: [Format] Generate: 1. A node list in build order using core class names: CheckpointLoaderSimple, LoraLoader, CLIPTextEncode for positive and negative, EmptyLatentImage, KSamplerAdvanced, VAELoader, VAEDecode, SaveImage, and for a refiner a second CheckpointLoaderSimple, its own CLIPTextEncode pair, and a second KSamplerAdvanced. 2. A wiring map: which output (MODEL, CLIP, VAE, CONDITIONING, LATENT, IMAGE) feeds which input, including LoRA chaining and why the refiner needs conditioning encoded with its own CLIP. 3. Settings for each sampler node: noise_seed and control after generate, steps, cfg, sampler_name, scheduler, and for the base and refiner split the add_noise, start_at_step, end_at_step, and return_with_leftover_noise values on each KSamplerAdvanced. 4. An SDXL resolution for ImageGoal near one megapixel (1024x1024, 1152x896, 896x1152, 1216x832, 832x1216), plus an optional upscale pass with UpscaleModelLoader and ImageUpscaleWithModel. 5. A VRAM plan for Hardware: batch size, when to skip the refiner, and switching to tiled VAE decode if memory runs out. 6. When RunMode is API, an API format workflow JSON with numbered node ids, class_type, and inputs where links are a two item list of source node id and output index, plus how to export from the Workflow menu in API format and POST it to the /prompt endpoint. Constraints: - Use only file names from ModelFiles and nodes that exist in core ComfyUI or in CustomNodes. - Do not aim for the likeness of real people or copy a named living artist's style. No em dashes.