How it works¶
A run is a one-shot Job¶
You run the dokimi CLI on your machine or in CI. It validates your test definition, renders
the Kubernetes objects a run needs, and submits them. From then on, the run happens inside the
cluster, as a one-shot Kubernetes Job, and the CLI streams its log back to you.
flowchart LR
DEF["test.yaml or<br/>suite.robot"] --> CLI["dokimi CLI<br/>(laptop or CI)"]
CLI -->|submit| JOB["One-shot Job<br/>in the cluster"]
JOB -->|streams its log| CLI
JOB -->|reads| NODES["Cluster nodes"]
JOB -->|deploys| PODS["Test pods:<br/>client and server"]
JOB -->|starts| PROM["The run's<br/>Prometheus"]
Three properties follow from this, and they are the point of the design:
- No controller is installed. A Job is not a controller, and it ends with the run. What stays is the run's own Prometheus, kept on purpose so that the run's history outlives it, until you remove it.
- The run does not depend on your connection. The Job is a server-side resource. If the CLI
disconnects, the run carries on, and
dokimi attach <run-id>resumes streaming, from any machine with access to the cluster. --dry-runnever submits anything. It renders the plan and the manifests, and stops.
The phases of a run¶
A run moves through twelve phases, numbered 0 to 11. dokimi's messages refer to them by number, so this table is also a key to what you see in the log.
| # | Phase | What happens | Status |
|---|---|---|---|
| 0 | Authoring | You write a YAML document or a Robot suite | works today |
| 1 | Submission | The CLI validates, renders the manifests, submits the Job, streams its log, and can reattach. --dry-run stops here |
works today |
| 2 | Ingestion | In the Job, the definition is parsed and checked again | works today |
| 3 | Planning | The two nodes are resolved against the cluster, and the test is planned | works today |
| 4 | Preflight | The run claims the cluster, so only one run at a time measures its network. It starts its own Prometheus and checks that it answers queries. Then the cluster is checked before any test runs: the nodes, scheduling, image pull, pod-to-pod connectivity and DNS, with the test's own two pods. dokimi preflight runs this phase on its own |
works today |
| 5 | Suite rendering | The definition is rendered to the Robot suite that actually runs | works today |
| 6 | Execution | Robot Framework runs the suite, which deploys the test pods, waits until both are ready, starts the traffic and waits for it to finish | works today |
| 7 | Measurement | An exporter next to each test pod serves its result as Prometheus metrics. The run reads it directly, and the receiver's view is the result; then it checks that Prometheus holds the same values. No data is never a pass | works today |
| 8 | Validation | Every pass criterion is evaluated against what each pair measured. A test passes, fails, or is an error when a pair couldn't be measured. No data is never a pass | works today |
| 9 | Reporting | Robot's own report, which comes back to your machine, plus a CSV and JSONL result file | Robot's report works today CSV and JSONL planned |
| 10 | Diagnostics | Where the pods ran, how each ended, and which pairs were down, captured while that evidence still exists, on every run | works today |
| 11 | Cleanup and exit | The run cleans up, keeps what a failed run left for you to look at, frees the cluster, and exits with its verdict: 0, 1 or 2 |
works today |
Today a run goes all the way to its verdict. It claims the cluster, starts its Prometheus and checks that it answers, checks the cluster with the test's own pair, runs the test through Robot Framework, reads the result, evaluates the pass criteria against it, records where everything ran, cleans up and exits. Only the CSV and JSONL result files are not built yet, and it says so:
run 20261013-120000-a1b2c3
phase 2 (ingestion): definition parsed
...
phase 3 (planning): 1 test, sequential
pair-001 worker-01 -> worker-02
phase 4 (preflight): cluster claimed for this run until 2026-10-13T12:20:30Z
phase 4 (preflight): monitoring mode managed (requested)
Prometheus http://run-20261013-120000-a1b2c3-prometheus.dokimi-iperf2.svc:9090
...
phase 4 (preflight): Prometheus answers queries
...
query it from your machine with: kubectl port-forward -n dokimi-iperf2 service/run-20261013-120000-a1b2c3-prometheus 9090:9090
phase 4 (preflight): nodes worker-01, worker-02 accept test pods
phase 4 (preflight): pair-001 scheduled, pulled its image and connected, as run-20261013-120000-a1b2c3-pair-001-preflight-client, run-20261013-120000-a1b2c3-pair-001-preflight-server
UDP to the server pod at 10.244.2.7:5001 arrived, and kubernetes.default.svc resolved
phase 4 (preflight): removed 2 test pod(s)
phase 4 (preflight): the cluster is ready
phase 5 (suite rendering): /work/artifacts/20261013-120000-a1b2c3/suite.robot
phase 6 (execution): pair-001 worker-01 -> worker-02, one Robot Framework process
...
phase 7 (measurement): pair-001, read directly from its exporters (inbound): jitter_seconds 1.1e-05, packet_loss_percent 0, test_duration_seconds 30, throughput_bits_per_second 9.49e+08, transferred_bytes 3.56e+09
phase 7 (measurement): Prometheus holds the series of pair-001
phase 8 (validation): pair-001 fail, 2 criterion(s)
fail Throughput Should Be At Least 950 megabits per second: 949 megabits per second
pass Packet Loss Should Be At Most 0 percent: 0 percent
phase 9a (reporting): /work/artifacts/20261013-120000-a1b2c3/log.html, .../report.html
phase 9b (results): the CSV and JSONL result files are not built yet. Each test's outcome is in phase 8 above, in its validation.json and in Robot's report
phase 10 (diagnostics): 5 pod(s), 1 pair(s), none down
...
phase 10 (diagnostics): /work/artifacts/20261013-120000-a1b2c3/diagnostics.json
phase 11 (cleanup): removed node-exporter run-20261013-120000-a1b2c3-node-exporter
phase 11 (cleanup): run kept for inspection (fail): run-20261013-120000-a1b2c3-pair-001-client, run-20261013-120000-a1b2c3-pair-001-server, and its owner and definition. It no longer holds the cluster. Remove it with: dokimi cleanup 20261013-120000-a1b2c3 --namespace dokimi-iperf2
...
artifacts: results/20261013-120000-a1b2c3
removed run 20261013-120000-a1b2c3's Job, now that its artifacts are here. What the run kept stays until: dokimi cleanup 20261013-120000-a1b2c3 --namespace dokimi-iperf2
It exits with 1: a criterion did not hold. A run whose tests all pass exits 0, and one with
a test that couldn't be validly measured exits 2, whatever the others found. A cluster that
can't run the test stops it in phase 4 with 4, before any test, and names the check that
failed.
On the way out it records where every pod ran and removes the node-exporter it deployed. A run
that passed removes its test pods too. This one didn't, so it keeps them, with their logs and
events, and its owner and definition, for you to look at, until dokimi cleanup removes them.
Either way it marks itself finished, so it stops holding the cluster at once.
Where Robot Framework fits¶
Robot Framework has two separate roles in dokimi.
- As a parser, today. A
.robottest definition is read with Robot's own parsing API, so keywords are matched the way Robot matches them. - As the runner and reporter, today. The Job renders your definition to a Robot suite,
the same one
dokimi convertprints, and runs it with one Robot Framework process per pair of nodes. The suite's keywords call dokimi's Python library, which deploys the client and server pods, waits until both are ready, startsiperf2and waits for it to finish. Robot'srebotmerges the reports, and you get Robot's familiarreport.htmlandlog.html.
The report comes back to your machine through the Job's log: submit and attach extract it
into the definition's results directory, under the run id:
results/20261013-120000-a1b2c3/
log.html Robot Framework's log of the run
report.html its report
output.xml every pair's results, merged
suite.robot the suite that ran
pairs/ per pair: its output.xml and validation.json, what its criteria came to
Each pass criterion is a keyword in that report. It passes or fails with the measured value in
its message, and every criterion is checked, not only the first. Each test is tagged
outcome:pass, outcome:fail or outcome:error. A test with a pair that couldn't be measured is
an error, tagged with the reason too, such as error:metrics_missing. It never passes.
Robot Framework is not needed on your machine. The CLI works without it. It runs inside the cluster, in the orchestrator image.
What a run creates¶
A submission creates three objects in the test type's namespace, dokimi-iperf2:
| Object | What it is for |
|---|---|
run-<id>-owner (ConfigMap) |
The run's owner. Everything else the run creates points at it, except its Prometheus |
run-<id>-definition (ConfigMap) |
Your test definition, mounted into the Job, so the run never depends on a file on your disk |
run-<id> (Job) |
The run itself |
In phase 4 the run adds its own Prometheus, run-<id>-prometheus: a ConfigMap, a Deployment
and a Service. Unlike everything else, it is kept after the run, so the run's history outlives
it. If the cluster runs no node-exporter, the run also deploys one, as a DaemonSet, and removes
it again when the run ends.
In phase 6 each test adds a client pod and a server pod. A run that passed removes them. One that didn't keeps them for you to look at, unless the test may still be sending: the client of a test that timed out, say, is always removed, and the run waits until it is gone before it frees the cluster.
Once submit or attach has brought the reports back, the Job goes. So do the owner and the
definition, if the run kept nothing.
Every one of them is labelled app.kubernetes.io/managed-by: dokimi and dokimi.io/run-id: <id>.
Runs and exit codes goes into the details.