Cursus ships Docker end-to-end correctness benchmarks and Go storage microbenchmarks. Published throughput numbers are meaningful only with the workload, hardware, durability, topology, and correctness counters attached.
| Topology | Compose file | Records | Partitions | Producer/consumer |
|---|---|---|---|---|
| Standalone | test/docker-compose.yml |
100000 | 12 | one idempotent batched publisher, one polling group consumer |
| Cluster | test/cluster/docker-compose.yml |
100000 | 12 | one multi-broker publisher, one coordinator/leader-routed polling group consumer |
The source settings are test/publisher/config.yaml, test/consumer/config.yaml, and the matching files under test/cluster. Benchmark storage uses tmpfs in the checked-in compose workloads, so results emphasize broker/client behavior and do not represent durable physical-disk latency.
RUN_E2E_BENCHMARK=1 go test -v -timeout 30m ./test/e2e-benchmark/...
PowerShell:
$env:RUN_E2E_BENCHMARK = "1"
go test -v -timeout 30m ./test/e2e-benchmark/...
make bench runs the standalone compose workload, checks container exit codes
and correctness/fatal markers, prints compose logs, and always tears down the
stack. The Go suite is the authoritative way to validate both topologies.
Docker Engine with Compose, sufficient CPU/memory for the clients and three
brokers, and unused fixture container names are required. The suite builds each
stack, starts it with --no-build --force-recreate, checks client exit and
summaries, and removes containers and volumes. Fixed-name cleanup is limited to
containers whose Compose labels identify this repository’s test stacks.
Without RUN_E2E_BENCHMARK=1, or with go test -short, the Docker tests skip.
Run one topology:
RUN_E2E_BENCHMARK=1 go test -v -timeout 15m -run '^TestStandaloneBenchmark$' ./test/e2e-benchmark/...
RUN_E2E_BENCHMARK=1 go test -v -timeout 15m -run '^TestClusterBenchmark$' ./test/e2e-benchmark/...
For direct log inspection, start one stack and tear it down with the matching compose file after inspection:
docker compose -f test/docker-compose.yml up --build --force-recreate
docker compose -f test/docker-compose.yml down -v --remove-orphans
docker compose -f test/cluster/docker-compose.yml up --build --force-recreate
docker compose -f test/cluster/docker-compose.yml down -v --remove-orphans
The standalone clients are bench-publisher/bench-consumer; cluster clients
are broker-publisher/broker-consumer.
The checked-in workloads use tmpfs and are not evidence of physical-disk durability latency. To run the same standalone correctness workload against a host directory on the intended filesystem, use the separate durable compose file. The directory must already exist, be dedicated to the run, and have enough space for broker logs and temporary compaction files.
$env:RUN_E2E_DURABLE_BENCHMARK = "1"
$null = New-Item -ItemType Directory -Force "D:\cursus-benchmark-results"
$env:CURSUS_DURABLE_BENCHMARK_LOG_DIR = "D:\cursus-benchmark-logs"
$env:CURSUS_DURABLE_BENCHMARK_RESULT = "D:\cursus-benchmark-results\run.json"
$env:CURSUS_BENCHMARK_REVISION = (git rev-parse HEAD)
$env:CURSUS_DURABLE_BENCHMARK_STORAGE = "NVMe model, filesystem, mount options"
go test -v -timeout 30m ./test/e2e-benchmark/... -run '^TestDurableStandaloneBenchmark$'
The result is versioned JSON containing revision, Go/runtime metadata, the operator-declared storage identity, post-run file/byte counts, elapsed time, benchmark summaries, and mandatory publish/consume correctness counters. The harness refuses to run or publish a result when that metadata or any required counter is missing. It is still a standalone workload; transaction, retention, compaction, and replica-recovery soak scenarios are separate operational validation runs and must be reported with their own workload settings.
The harness fails on non-zero container exit, timeout, incomplete/verification markers, anchored panic/fatal logs, failed publishes, missing records, duplicate message IDs, or duplicate logical offsets. Multi-digit counters are parsed numerically. A success banner without zero correctness counters is not sufficient.
Expected consumer summary fields include:
Total Messages
Elapsed Time
Overall TPS
Duplicate (MessageID) : 0
Duplicate (Offset) : 0
Message missing : 0
The test output filters successful logs to benchmark summaries. Full compose service logs are collected when a process times out or verification fails.
Those require purpose-built workloads and failure tests. Do not infer them from this benchmark.
BATCH_COMMIT responses./ready, Raft leader/ISR state, and container-name
collisions.log_level: debug in test configs and
restore INFO after diagnosis.Normal E2E runs on pull requests. The Docker benchmark runs only on pushes to main, after normal E2E succeeds, in the final step of .github/workflows/e2e-tests.yml. It builds its own standalone/cluster compose images; Docker layer caching may help, but the active E2E containers are not reused.
make bench-disk
make bench-serialize
Equivalent commands:
go test ./pkg/disk/ -bench=. -benchmem -count=3 -timeout=120s
go test ./pkg/disk/ -bench=BenchmarkSerializeDiskMessage -benchmem -count=5 -timeout=60s
Microbenchmarks isolate serialization and disk-handler operations from the network and coordinators. Record Go version, OS/filesystem, CPU, disk/tmpfs, benchmark count, and configuration when comparing results.
A useful report includes:
Do not commit generated benchmark result files unless they are intentionally curated evidence for a release or published performance analysis.