Pipeline tests

telflo test runs the same pipeline tests as the editor's Testing tab: a sample is replayed through one pipeline of a configuration on a real collector, and assertions written in OTTL check what comes out. The CLI describes a run in a small YAML spec file.

The spec file

name: redaction checks        # optional, up to 200 characters; the display name used by --save
signal: logs                  # traces | metrics | logs
pipeline: logs/in             # the pipeline under test, a service.pipelines key
duration: 5                   # seconds, 1 to 60; default 5
collector_version: 0.147.0    # optional; see the precedence below
distribution: otelcol-contrib # optional
sample: ./sample.json         # path relative to the spec file, up to 1 MB
# sample_text: "..."          # or the sample inline; exactly one of the two
rules:                        # 1 to 20 rules
  - name: total count
    context: log              # span | spanevent | metric | datapoint | log | resource | scope
    quantifier: count         # all | any | none | count | distinct
    condition: "true"
    count: { operator: eq, value: 10 }   # eq | ne | gt | gte | lt | lte | between

Which collector runs the test is resolved in this order: the --collector-version and --distribution flags; then, on a local-file run with -f file --config <id>, that configuration's pin; then the spec's collector_version and distribution fields; then, on a stored-version run with --config alone, the version's own pin; then the server default. Prefer the pin: a test that passes on the default collector proves nothing about the one the configuration actually runs on.

Rules with the all, any, none, and count quantifiers take a boolean OTTL condition. A distinct rule instead takes an expression, a value expression whose distinct values are counted, with Concat([...], "|") for composite keys, a required count, and an optional condition that filters the rows first. These are the same rule types as the editor's validation rules.

The sample must match the pipeline's receiver: OTLP JSON for OTLP receivers, raw lines for filelog, syslog lines for syslog.

Authoring rules

The sandbox rejects the mechanical mistakes below, such as tautological conditions, single-escaped regex classes, and constant distinct expressions; the rest are conventions that make a test worth trusting.

1. Assert the behavior, not the sample's literal values. Exact values are legal only in the untouched-control rule and for values the configuration hard-codes.
2. Include three control rules. The total number of surviving records (condition: "true" with the count quantifier), positive proof that the processor acted, and an untouched control: a record the processor must not match, asserted to survive unchanged.
3. No tautologies. Conditions such as X == X or Y == nil or Y != nil are rejected. A bare true is legitimate in the count control and, with any or none, as an explicit non-emptiness or emptiness check; with all it is rejected as vacuous.
4. Escape once for OTTL. OTTL consumes one backslash level, so write \\d and \\b, or prefer character classes such as [0-9], [.], and [+]. A single-escaped regex class such as \d or \b is rejected because it would match nothing and pass vacuously; a single-escaped \. or \+ is not caught, one more reason to prefer the bracket classes.
5. Pick the quantifier for the claim. none for absence and leak checks; count with eq for exact survivor counts; all is vacuously true over zero records, so pair it with a count control; distinct for cardinality.
6. Know the sandbox's limits. Connectors are stripped and pipelines other than the target are removed; $orgvar: tokens are not substituted, so inline real values; Kubernetes enrichment uses a fixed test pod, telflo-test-pod in the demo namespace, so never assert on k8s.pod.uid.

Run a test

telflo test --spec test.yaml --config <id>                 # a stored version, on its collector pin
telflo test --spec test.yaml --config <id> --version 3     # a specific version
telflo test --spec test.yaml -f local.yaml --config <id>   # a local file, still on that configuration's pin
telflo test --spec test.yaml -f local.yaml                 # a local file on the spec's collector, or the server default

The command submits the run, polls until it reaches a final state, every second at first and every five seconds after the first ten polls, and prints each rule's result as PASS, FAIL, or not evaluated. A wrong pipeline: fails immediately with a message that lists the pipelines the configuration does have. --output <file> saves the run's output; --timeout <s> bounds the wait.

A run that targets a stored version with --config, and no -f, also appears in that version's history on the editor's Tests sub-tab, with cli-sample.json as its input file. A run on a local file belongs to no version and is read back through the CLI or the API only.

Save rules to the Validation sub-tab

The editor's Testing panel has a Validation sub-tab holding saved validation rules, one set per configuration version and pipeline. A plain telflo test never touches it. Two flags connect the two; both need --config and neither works with -f.

telflo test --spec test.yaml --config <id> --save    # save the spec's rules, then run them
telflo test --spec test.yaml --config <id> --saved   # run the rules already saved in the app
telflo config rules <id>                             # show what is saved

• --save first saves the spec's rules as the version's validation rules for pipeline, then runs the version's saved set: the spec's rules plus any hand-authored rules that were kept. Use it once your assertions pass, so the rules appear in the Validation tab and can be re-run from the app. The API replaces only rules it saved earlier, marked as created through the Telflo API; rules added by hand in the app are kept. The spec's name sets the rule set's display name; without one, an existing name is kept and a new set is named API tests — <pipeline>.
• --saved runs the rules the version already has, whether authored in the app or by an earlier --save. The spec then needs only signal, pipeline, and sample; inline rules are ignored with a notice.

A token can save rules onto configurations its service account created or that are published. Another member's unpublished draft is not visible to it and returns a not-found error.

Read the results

Each entry in validation_results reports passed and, on failure, an error_message or failure_detail such as no_match, unexpected_match, record_mismatch, or distinct_mismatch, with sample matches and counts. A run status of Fail means assertions failed; Error means the collector or sandbox failed, and the run's stderr, which carries the collector's log and the sandbox's reason, is the place to look first. Exit codes follow the status: 0 for success, 4 for failed assertions, 1 for an error or timeout.