Validating output

A test run shows what a pipeline produces. The Validation sub-tab defines rules that the output is checked against, so a run passes or fails automatically instead of being read by hand.

Validation pipelines

Rules are grouped into validation pipelines. A validation pipeline holds the rules for one of the configuration's YAML pipelines: when a test run targets that pipeline, the rules in its group are evaluated against the run's output. Each YAML pipeline can have one validation pipeline.

1. Open the Validation sub-tab and select + Pipeline.
2. Select the YAML pipeline the group checks. Pipelines that already have a group cannot be selected again.

Each group renders as a card titled with its pipeline's name. A run that targets a pipeline with no group, or an empty group, is evaluated with no rules: success means the collector ran cleanly.

If the group's pipeline is later removed from the YAML, the card shows a warning and its rules stop running, but the group stays editable until you delete it. Deleting a group deletes its rules.

Validation rules

A validation rule checks a run automatically, so a run passes or fails without you reading the output by hand. Each rule evaluates the captured output, or the collector's stderr, of runs that target its group's pipeline.

To add a rule, expand the group's card, select Add Validation Rule, and choose a Rule type.

Record count and Record attribute are guided builders: they save as OTTL rules and reopen in the builder when edited. A rule that goes beyond the builder's shape, such as a composed condition, opens in OTTL mode instead. The builder's helper points there: for nested fields or composed conditions, switch to OTTL mode.

OTTL expressions are syntax-checked against the collector's OTTL grammar when the rule is saved; a syntax error blocks the save. If the syntax check is unavailable, the rule saves and any error is reported when a run evaluates it. To check a condition from the terminal before saving it, run telflo ottl lint --context <context> --expr '<condition>', which parses it with the same grammar. See OTTL lint.

A run with a failing rule is marked Validation Failed; the CLI and API report the status as Fail.

Attribute checks

The Record attribute rule confirms a pipeline produced the attributes you expect. For example, check that k8s.pod.name is present on every record after a k8sattributes processor runs, or that a sensitive attribute is gone after redaction.

A record attribute rule has the following fields.

With Require set to Matching count…, a Count comparison appears, from Exactly to Between. With Distinct values…, the Operator and Value fields are replaced by a Distinct count comparison: the rule counts the distinct values the attribute takes and compares that count to the threshold.

To check that an attribute is present, set Require to All records match and Operator to Exists. To check that an attribute is absent, set Require to No record matches with the Exists operator.

OTTL conditions

The OTTL condition rule evaluates an OTTL condition against the run's records, using the same expression language as the transform and filter processors.

Distinct values… counts the distinct values a Value expression resolves to, such as attributes["http.route"], and compares the count to the threshold; the condition field becomes an optional row filter. Use it as a cardinality guardrail against a high-cardinality attribute inflating your metrics ingest. At most 0 asserts that the value never appears in the output.

In a CLI test spec or an API request, the same rule is written as a context, a quantifier of all, any, none, count, or distinct, matching the Require options in order, a condition, or for distinct an expression with an optional condition that filters the records first, and, for the count and distinct quantifiers, a count with an operator of eq, ne, gt, gte, lt, lte, or between. See Pipeline tests.

Output and stderr

A Match output text rule reads from one of two streams, set by Read from:

• Output (pipeline records). The OTLP records the pipeline produced, and the default for new rules. Use it to assert on what your processors emit.
• Stderr (collector log). The collector's own diagnostic log. Use it to catch silent processor drops, OTTL parse errors, and exporter rejections that never reach the output.

Compare a run's input and output

A run with an OTLP input offers a Compare input ↔ output view: a side-by-side diff of the input and output records. Detected changes are offered as candidate rules, as chips such as + attribute added, + attribute removed, or + attribute changed. Selecting one opens the rule editor pre-filled with an OTTL rule asserting the output state, so an observed change becomes a rule directly.

Rules created by the AI assistant

The AI assistant's Build mode can run pipeline tests and saves the assertions it checked to the pipeline's validation collection, with the description Created by the AI assistant. A later assistant run replaces only the rules the assistant created; rules you author are never touched. See AI assistant.

Rules saved from the CLI and API

telflo test --save, and PUT /configurations/{id}/versions/{n}/validation-pipelines/{pipeline} on the API, save a set of OTTL rules as the validation collection for one YAML pipeline of a version, creating the collection when the pipeline has none. Rules saved this way carry the description Created via the Telflo API. They are stored as OTTL rules that read the run's output, named as in the spec or Rule 1, Rule 2, and so on. A later save replaces only the rules saved through the API; rules you author here, and rules the AI assistant saved, are kept. A new collection saved without a name is called API tests — <pipeline>.

The API refuses a rule the sandbox would evaluate vacuously: a condition that is always true such as X == X, a single-escaped regex metacharacter such as \d, or a distinct-values expression that is a constant. It also refuses more than 20 rules in one set.

telflo test --saved, or use_saved_rules on the API, runs the rules the collection already holds, whatever their type, including Match output text rules, and reports each result against the rule it belongs to. telflo config rules <id> lists the collections saved on a version. See Pipeline tests.

Regex library

A Match output text rule with Type set to Regex takes the expression in the Regex pattern field. To add common patterns instead of writing them, select Add from regex library on the group's card. The Add validation rules from regex library dialog lists patterns by Category, such as credit card and CVV numbers. Choose Match, either Contains or Does not Contain, select the patterns, and select OK. Each selected pattern becomes its own Match output text rule, named after the pattern.

Use rules with a run

1. Define rules in the validation pipeline for the pipeline you test.
2. Run a test that targets that pipeline. See Running tests.
3. Telflo evaluates the captured output against the group's rules and reports which passed and which failed. A rule that could not be evaluated, for example on a run that errored, shows a Skipped badge with the reason; the CLI prints such a rule as not evaluated.

Combine rules with test suites so a saved test is checked the same way every time.

Static validation

Telflo also validates a configuration structurally and against the collector schema as you build. See Validation. Static validation confirms the configuration is valid. Output validation confirms it behaves as expected on sample data. When the validator itself cannot be reached, the editor reports the configuration as unchecked rather than invalid; see Validation status.