How to Use the OpenTelemetry Collector Config Prompt to Build Agent and Gateway Pipelines That Validate

The OpenTelemetry Collector is the piece that sits between your applications and your observability backend, and its YAML file is where small mistakes turn into dropped spans, memory kills, or user emails sitting in log attributes. Common problems include processors in the wrong order, tail sampling running on agents that only see part of each trace, and secrets pasted into the config. The OpenTelemetry Collector Config Writer for otelcol-contrib: OTLP, Host Metrics, and Filelog Receivers, memory_limiter First, Batch, PII Redaction, Tail Sampling, Agent vs Gateway Pipelines, and otelcol validate prompt produces an otelcol-contrib configuration with a safe processor order, PII redaction, tail sampling on the right tier, and a validation step before anything ships.

What the prompt produces

1. A topology note explaining what runs on the per host agent and what runs on the gateway, including when to route traces by trace ID with the loadbalancing exporter.
2. Receivers for OTLP over gRPC and HTTP, hostmetrics with only the scrapers you need, and filelog with include paths and a parser that matches your log format.
3. Processors in order: memory_limiter first, then attribute or resource processors that delete or hash sensitive keys, then tail_sampling on the gateway only, then batch.
4. Tail sampling policies for errors, slow traces, and a baseline percentage, with decision_wait and num_traces explained.
5. Exporters for each signal, with auth headers read from environment variables.
6. health_check and zpages extensions for probes and live debugging.
7. service.pipelines for traces, metrics, and logs, with every component defined and used.
8. Validation steps using otelcol-contrib validate and a smoke test.
9. A risk list covering what fails first when memory is tight and which Collector metrics show dropped data.

How to fill the inputs

Deployment describes the shape: agent only, gateway only, or both, plus the platform and Collector version. Version matters because some field names change between releases.

Signals lists which services send OTLP, which hosts need host metrics, and which log files to tail, with their format.

Backends names the endpoints for each signal and the header names for auth. Give names only, never token values.

PiiFields lists attributes to delete or hash, such as user.email or authorization headers.

SamplingPolicy states your goals in plain words, such as keep every error trace and ten percent of the rest.

MemoryBudget is the container or host memory limit for each Collector, which drives the memory_limiter settings.

Reading the example output

The example runs agents as a Kubernetes DaemonSet and three gateway replicas:

• The topology note explains the routing: agents use the loadbalancing exporter keyed on traceID so every span of a trace reaches the same gateway, which is what makes tail sampling correct.
• memory_limiter comes first on both tiers with a percentage of the container limit.
• PII is removed twice: on the agent and again on the gateway as a second line of defense.
• Three sampling policies keep errors, traces slower than 800 ms, and ten percent of everything else.
• The exporter reads its Authorization header from an environment variable using the env syntax.
• Separate pipelines handle traces, metrics, and logs, and tail sampling only appears in the traces pipeline.
• Validation runs before deploy, followed by a test span and a check that user.email is missing on the backend.

Tips for better results

• Paste your current config if you have one. The prompt can point out what to reorder instead of starting fresh.
• Give the real log format with one sample line. Parser operators depend on it.
• Run otelcol-contrib validate in CI so broken configs never reach a cluster.
• Watch the Collector's own metrics for refused and dropped data after rollout.

Mistakes to avoid

• Do not put tail_sampling on agents. An agent only sees the spans from its own host.
• Do not place batch before memory_limiter. The limiter should see data first.
• Do not hardcode tokens in YAML or commit them to the repo.
• Do not define components you never reference in a pipeline. Validation may pass, but it confuses the next person.

Who it is for

Platform and SRE teams rolling out OpenTelemetry, developers moving from vendor agents to the Collector, and anyone cleaning up a Collector config that grew by copy and paste.

Related PromptDig links

Open the OpenTelemetry Collector Config Writer for otelcol-contrib: OTLP, Host Metrics, and Filelog Receivers, memory_limiter First, Batch, PII Redaction, Tail Sampling, Agent vs Gateway Pipelines, and otelcol validate prompt and describe your deployment. For more coding prompts, Browse more prompts. If you have a DevOps or observability prompt that works well, Share a prompt.