cursus

Configuration

This document provides a comprehensive guide to configuring cursus.

It covers all available configuration parameters, their default values, configuration sources, and the precedence order when configuration is specified in multiple locations.

Purpose and Scope

Configuration in cursus controls:

Broker configuration is a flat Config value. YAML uses snake-case keys and JSON uses the dotted keys declared on pkg/config.Config.

Configuration Sources and Precedence

cursus applies configuration in this order:

  1. built-in defaults,
  2. the YAML or JSON file selected by --config or CONFIG_PATH,
  3. explicitly supplied command-line flags,
  4. supported environment-variable overrides,
  5. normalization and security validation.

Values omitted from the command line remain available from the configuration file. Explicit flags override the file, and supported environment variables override both. CONFIG_PATH selects a file only when --config is absent.

Bootstrap admission bytes

New bootstrap transfers use bootstrap_max_transfer_bytes (default 8589934592, 8 GiB) and bootstrap_max_total_bytes (default 34359738368, 32 GiB process-wide). JSON keys are bootstrap.max.transfer.bytes and bootstrap.max.total.bytes; flags are --bootstrap-max-transfer-bytes and --bootstrap-max-total-bytes; environment variables are BOOTSTRAP_MAX_TRANSFER_BYTES and BOOTSTRAP_MAX_TOTAL_BYTES. Both values must be positive, total must be at least transfer, and transfer cannot exceed 288230376151711743 bytes. Invalid or overflowing inputs are errors, including explicit zero.

Both partition receivers and detached consumer metadata recovery share these limits. The transfer reservation includes generated indexes and tail files, so 8 GiB is not a promise of 8 GiB of raw payload. Lower limits affect new admission; they do not revoke existing transfers or block durable pending replay. The independent limits remain eight active disk transfers, a 16 MiB catalog, 256 KiB chunks and a five-minute staging lifetime.

Configuration File Format

Configuration files can be in either YAML or JSON format. The format is detected automatically based on the file extension.

YAML Format

The standard configuration format used in cursus is YAML. Keys are top-level; there is no broker: wrapper:

broker_port: 9000
health_check_port: 9080
enable_exporter: true
exporter_port: 9100
log_dir: "broker-logs"
compression_type: "lz4"
disk_flush_batch_size: 500
disk_write_timeout_ms: 200
linger_ms: 100
log_segment_roll_ms: 604800000
log_cleanup_policy: "delete"

static_consumer_groups:
  - name: "workers"
    consumer_count: 2
    topics: ["orders"]
    topic_partitions:
      orders: 6

JSON Format

The same configuration can be expressed in JSON format:

{
  "broker.port": 9000,
  "health.check.port": 9080,
  "log.dir": "broker-logs",
  "enable.exporter": true,
  "exporter.port": 9100,
  "log.cleanup.interval": 60,
  "tls.enable": false,
  "tls.cert_path": "certs/server.crt",
  "tls.key_path": "certs/server.key",
  "compression.type": "lz4",
  "disk.flush.batch.size": 500,
  "linger.ms": 100,
  "channel.buffer.size": 10000,
  "partition.channel.buffer.size": 10000,
  "consumer.channel.buffer.size": 1000
}

Configuration Structure

The configuration is represented by the Config struct in the codebase, which organizes parameters into logical categories.

Common Parameters

Parameter Type Default Description
broker_port int 9000 Main broker TCP port for client connections
broker_bind_address string “127.0.0.1” Client listener bind address; non-loopback binds require TLS and SASL by default
health_check_port int 9080 HTTP port for /live and /ready
log_dir string “broker-logs” Directory path for persistent log segments
enable_exporter bool true Enable Prometheus metrics exporter
exporter_port int 9100 HTTP port for the Prometheus /metrics endpoint
log_level string “info” Broker log level: debug, info, warn, or error
log_cleanup_interval int 300 Legacy maintenance-loop interval (seconds)

In standalone mode, log_dir also contains __topic_metadata.json, the broker-owned __consumer_offsets topic, and __transaction_state.journal. The versioned topic manifest is atomically replaced before create/update success exposes a new topic definition and is loaded before internal-topic validation, durable group/offset replay, coordinator/static-group initialization, and readiness. Invalid or unsupported manifest/internal metadata fails closed in diagnostics-only mode; the broker does not fall back to guessed ACL, event-sourcing, retention, group, or offset state. A pre-manifest directory with persisted logs requires a complete clean bootstrap; no offline import or migration command is supported.

The broker fsyncs the append-only transaction coordinator journal before acknowledging transaction state transitions and repairs only a torn or checksum-corrupt final record during startup. One encoded snapshot record is limited to 32 MiB, so transaction batches must remain bounded. Include the topic manifest, journal, consumer offset log, and partition directories in one backup and restore procedure.

The health and metrics listeners are unauthenticated operations endpoints. Restrict both ports to a trusted network. /live reports process liveness, while /ready includes topic metadata, consumer metadata, storage, and distributed leader checks. See Broker Observability.

Security and Compression

Parameter Type Default Description
use_tls bool false Enable TLS for TCP connections
tls_cert_path string ”” Path to TLS certificate file
tls_key_path string ”” Path to TLS private key file
internal_broker_port int 0 Optional dedicated broker-to-broker command port
internal_use_tls bool false Require mutual TLS on the internal broker listener
internal_tls_cert_path string ”” Broker certificate for internal mTLS
internal_tls_key_path string ”” Broker private key for internal mTLS
internal_tls_ca_path string ”” CA used to verify peer broker certificates
internal_tls_server_name string ”” Server name used by broker-to-broker mTLS clients
enable_sasl bool false Enable SASL-PLAIN-style token authentication for text commands
sasl_users list [] Principal/token/permissions entries accepted by AUTH and inline authentication
allow_insecure_client_transport bool false Explicit test-only opt-out that permits a non-loopback client listener without both TLS and SASL
compression_type string “none” Preferred codec: none, gzip, snappy, or lz4

When use_tls is enabled and certificate paths are provided, the broker loads the certificate using tls.LoadX509KeyPair() during initialization. The default broker_bind_address is loopback. A wildcard or other non-loopback address fails validation unless TLS and SASL are both configured or allow_insecure_client_transport=true is explicitly set. In distributed mode, internal_broker_port moves broker-to-broker text commands away from the public client listener. If internal_use_tls is enabled, the internal listener requires client certificates signed by internal_tls_ca_path, and peer routers dial the internal port with mTLS using internal_tls_server_name for certificate verification.

When enable_sasl is enabled, protected commands require AUTH principal=<principal> token=<token> or inline principal=<principal> auth_token=<token>. Every user must declare at least one permission from admin, topic.read, topic.write, group, transaction, and *; startup rejects missing, unknown, repeated, or duplicate-principal entries. CONSUME/STREAM require both topic.read and group; TXN_PUBLISH requires transaction and topic.write; SEND_OFFSETS_TO_TXN requires transaction and group. Topic auth_policy=acl is evaluated after the coarse permission check. The environment form is SASL_USERS=principal:token:permission1|permission2, with comma-separated users.

DiskHandler Performance Tuning

These parameters directly affect the write path performance and batching behavior described in DiskHandler and Write Path.

Parameter Type Default Description
disk_flush_batch_size int 50 Number of messages to batch before flushing to disk
linger_ms int 50 Maximum time to wait before flushing (milliseconds)
channel_buffer_size int 1024 Buffer size for DiskHandler’s writeCh channel
disk_write_timeout_ms int 10 Timeout while enqueueing an asynchronous write (ms)
disk_flush_interval_ms int 500 Periodic fsync interval (milliseconds)
log_segment_bytes uint64 1073741824 Maximum segment file size (1GB default)
log_index_size_bytes uint64 10485760 Maximum index file size (10MB default)
log_index_interval_bytes int 4096 Index entry interval in bytes
log_retention_hours int 168 Log retention period in hours (7 days default)
log_retention_bytes int64 -1 Retained byte limit; -1 means unlimited
log_segment_roll_ms int 604800000 Time-based roll interval (7 days)
log_cleanup_policy string “delete” delete, compact, or delete,compact; distributed compaction is safety-gated
log_retention_check_interval_ms int 300000 Delete-retention evaluation interval
log_compaction_check_interval_ms int 300000 Closed-segment compaction evaluation interval
log_min_cleanable_dirty_ratio float64 0.5 Minimum removable-byte ratio before compaction
compression_type string “none” Compression type: “none”, “gzip”, “snappy”, “lz4”

Trade-offs:

Partition and Consumer Channel Tuning

These parameters control the in-memory channel buffer sizes for message distribution within the topic management system.

Parameter Type Default Description
partition_channel_buffer_size int 10000 Buffer size for each Partition’s input channel
consumer_channel_buffer_size int 1000 Buffer size for each Consumer’s message channel
broadcast_channel_buffer_size int 10000 Buffer size for embedded topic broadcast channels

Broker And Cluster Parameters

These values participate in active broker behavior:

Parameter Default Purpose
enabled_distribution false Enable the Raft-backed cluster runtime.
min_insync_replicas 2 Broker fallback minimum for acks=all/-1 when a topic has no min_in_sync_replicas override.
default_replication_factor 3 Default replica count for new distributed topics.
internal_broker_port 0 Dedicated broker-to-broker command listener; configure in production clusters.
internal_auth_token empty Active outbound internal command credential; always required when distribution is enabled.
internal_auth_token_next empty Additional inbound credential accepted during a staged rotation. Brokers never send it until it becomes the active token.
internal_auth_generation empty Non-secret identifier exposed in configuration diagnostics and CLUSTER_STATUS.
internal_use_tls false Enables broker-internal TLS and client-certificate verification.
allow_insecure_cluster_transport false Explicit test-only opt-out from the distributed mTLS requirement.
raft_peers [] Initial Raft peer addresses.
transactional_id_expiration_ms 604800000 Retention for completed transaction payloads. Epoch tombstones remain for fencing; active transactions are not expired.
transaction_timeout_ms 60000 Maximum duration of an open broker transaction before durable timeout abort.
transaction_coordinator_shards 50 Logical transaction-coordinator shard count. Immutable after cluster creation.
transaction_recovery_batch_size 256 Maximum prepared or timed-out transactions handled per recovery batch.
max_transactions 100000 Maximum retained transaction identities, including completed identities kept for fencing.
max_transaction_records 10000 Maximum staged records or durable request assignments retained by one transaction.
max_transaction_bytes 67108864 Maximum dynamic staged payload bytes retained by one transaction.
max_transaction_offsets 10000 Maximum distinct consumer offsets staged by one transaction.
disk_min_free_bytes 268435456 Minimum filesystem bytes reserved after each admitted storage batch.
disk_min_free_percent 5 Minimum filesystem percentage reserved after each admitted storage batch. The stricter byte or percentage threshold wins.
producer_state_ttl_ms 1800000 In-memory producer state cleanup window; durable records/checkpoints remain recovery sources.
raft_port 9001 Raft transport listener.
discovery_port 8000 Broker discovery and internal replication HTTP listener.
raft_snapshot_interval_ms 120000 Interval for evaluating snapshot creation.
raft_snapshot_threshold 8192 Outstanding Raft log entries required before snapshotting.
raft_trailing_logs 10240 Raft log entries retained after a snapshot.
static_cluster_members [] Stable broker-id@host:raft-port membership set.
bootstrap_cluster false Explicitly bootstrap a new Raft cluster.
advertised_host localhost Host advertised for broker discovery.
advertised_broker_port 0 Broker port advertised to peers when different from the listener.
advertised_client_host empty Client-facing host returned by routing metadata.
max_client_connections 1000 Concurrent client connection limit.
max_inflight_requests 256 Request count budget. Distributed mode divides it evenly between client and internal listeners so client traffic cannot stop replication; standalone mode uses the full value for clients. Values below 2 are normalized to 256.
max_inflight_request_bytes 268435456 Encoded-plus-decoded payload budget. Distributed mode divides it evenly between client and internal listeners; standalone mode uses the full value for clients. Values below 128 MiB in standalone mode or 256 MiB with an internal listener are normalized to 256 MiB so each active pool can admit one maximum-size compressed frame.
client_idle_timeout_ms 60000 Idle client connection deadline.
client_request_timeout_ms 30000 Timeout applied to request processing and response writes. A response produced at the processing deadline gets one write attempt bounded by the same value. It is also the rolling per-frame write timeout for streams and, together with the idle timeout, bounds Wire v2 negotiation. An acks=all publish that times out after append returns request_timeout outcome=unknown; its accepted replication continues independently.
max_stream_connections 1000 Concurrent streaming connection limit.
stream_timeout 30m Maximum broker stream lifetime as a Go duration string.
consumer_session_timeout_ms 10000 Group member session timeout.
consumer_heartbeat_check_ms 5000 Broker interval for detecting expired members; normalized below the session timeout.
enable_idempotence false Broker default for producer idempotence; topic/request contracts can enable it explicitly.

Distribution is disabled by default. Production clusters should use a dedicated internal listener, mTLS, least-privilege client users, and explicit advertised addresses.

Topic creation can set min_in_sync_replicas=<N> with 1 <= N <= replication_factor. ALTER_TOPIC_CONFIG topic=<name> min_in_sync_replicas=<N|default> changes or removes that optional durable override. Old topic metadata without the field continues to use the broker fallback. Idempotent publishers require acks=all or acks=-1; the SDK and broker reject weaker combinations rather than silently changing them.

bootstrap_servers and acks are SDK publisher settings, while replication_factor is a topic-definition field. They are not broker configuration keys.

static_consumer_groups is a broker bootstrap facility. Each entry declares name, consumer_count, topics, and an optional topic_partitions map. Missing or non-positive partition counts normalize to one for the corresponding topic. Network clients still use the durable REGISTER_GROUP, JOIN_GROUP, and SYNC_GROUP lifecycle; static in-process groups are not a substitute for that protocol.

Using Configuration in Different Scenarios

Scenario 1: Development with Defaults

For local development, you can run the broker with built-in defaults:

./bin/cursus

This uses:

Scenario 2: Using a Configuration File

Create a configuration file and specify it:

./bin/cursus --config /path/to/config.yaml

// Or using the environment variable:
// export CONFIG_PATH=/path/to/config.yaml
// ./bin/cursus

The environment variable approach is checked in pkg/config/properties.go

Scenario 3: Docker Deployment

In docker-compose deployments, configuration is typically mounted as a volume and referenced via environment variable:

services:
  broker:
    volumes:
      - ./config.yaml:/app/config.yaml:ro
      - cursus-data:/data/logs
    environment:
      - CONFIG_PATH=/app/config.yaml
      - LOG_DIR=/data/logs
      - BROKER_BIND_ADDRESS=0.0.0.0
    ports:
      - "127.0.0.1:9000:9000"
      - "127.0.0.1:9100:9100"
      - "127.0.0.1:9080:9080"
volumes:
  cursus-data:

These loopback bindings are for local inspection and require Docker Engine 28.3.3 or newer. Configure TLS and SASL before publishing the broker listener on a non-loopback host address.

Scenario 4: Deployment-Time Overrides

Use supported environment variables when a deployment must override values loaded from a configuration file:

CONFIG_PATH=config.yaml BROKER_PORT=9001 EXPORTER_PORT=9101 ./bin/cursus

Ordinary value flags are parsed before the file and are overwritten by matching file values. This ordering is part of the current implementation contract.

Scenario 5: High-Throughput Configuration

For maximum throughput at the cost of latency:

disk_flush_batch_size: 1000    # Batch more messages
linger_ms: 200                 # Wait longer before flush
channel_buffer_size: 20000     # Larger write buffer
partition_channel_buffer_size: 20000
consumer_channel_buffer_size: 5000

Scenario 6: Low-Latency Configuration

For minimum latency at the cost of throughput:

disk_flush_batch_size: 50
linger_ms: 10
channel_buffer_size: 1024
partition_channel_buffer_size: 5000
consumer_channel_buffer_size: 500

Configuration Parameter Mapping

The Config struct uses both YAML and JSON tags to support both formats. Here’s how parameter names map between different formats:

Go Field Name YAML Key JSON Key CLI Flag
BrokerPort broker_port broker.port –port
HealthCheckPort health_check_port health.check.port –health-port
LogDir log_dir log.dir –log-dir
EnableExporter enable_exporter enable.exporter –exporter
ExporterPort exporter_port exporter.port –exporter-port
CleanupInterval log_cleanup_interval log.cleanup.interval –cleanup-interval
UseTLS use_tls tls.enable –tls
TLSCertPath tls_cert_path tls.cert_path –tls-cert
TLSKeyPath tls_key_path tls.key_path –tls-key
CompressionType compression_type compression.type –compression-type
InternalBrokerPort internal_broker_port distribution.internal_broker_port –internal-broker-port
InternalUseTLS internal_use_tls internal_tls.enable –internal-tls
InternalTLSCertPath internal_tls_cert_path internal_tls.cert_path –internal-tls-cert
InternalTLSKeyPath internal_tls_key_path internal_tls.key_path –internal-tls-key
InternalTLSCAPath internal_tls_ca_path internal_tls.ca_path –internal-tls-ca
InternalTLSServerName internal_tls_server_name internal_tls.server_name –internal-tls-server-name
EnableSASL enable_sasl sasl.enable –enable-sasl
ProducerStateTTLMS producer_state_ttl_ms producer.state.ttl.ms –producer-state-ttl-ms
TransactionalIDExpirationMS transactional_id_expiration_ms transactional.id.expiration.ms –transactional-id-expiration-ms
TransactionTimeoutMS transaction_timeout_ms transaction.timeout.ms –transaction-timeout-ms
TransactionCoordinatorShards transaction_coordinator_shards transaction.coordinator.shards –transaction-coordinator-shards
TransactionRecoveryBatchSize transaction_recovery_batch_size transaction.recovery.batch.size –transaction-recovery-batch-size
MaxTransactions max_transactions max.transactions –max-transactions
MaxTransactionRecords max_transaction_records max.transaction.records –max-transaction-records
MaxTransactionBytes max_transaction_bytes max.transaction.bytes –max-transaction-bytes
MaxTransactionOffsets max_transaction_offsets max.transaction.offsets –max-transaction-offsets
MaxInflightRequests max_inflight_requests max.inflight.requests –max-inflight-requests
MaxInflightRequestBytes max_inflight_request_bytes max.inflight.request.bytes –max-inflight-request-bytes
DiskFlushBatchSize disk_flush_batch_size disk.flush.batch.size –disk-flush-batch
DiskMinFreeBytes disk_min_free_bytes disk.min.free.bytes –disk-min-free-bytes
DiskMinFreePercent disk_min_free_percent disk.min.free.percent –disk-min-free-percent
LingerMS linger_ms linger.ms –linger-ms
ChannelBufferSize channel_buffer_size channel.buffer.size –channel-buffer
DiskWriteTimeoutMS disk_write_timeout_ms disk.write.timeout.ms –disk-write-timeout
PartitionChannelBufSize partition_channel_buffer_size partition.channel.buffer.size –partition-ch-buffer
ConsumerChannelBufSize consumer_channel_buffer_size consumer.channel.buffer.size –consumer-ch-buffer
SegmentSize log_segment_bytes log.segment.bytes –segment-size
SegmentRollTimeMS log_segment_roll_ms log.segment.roll.ms –segment-roll-time-ms
IndexSize log_index_size_bytes log.index.size.bytes –index-size
CleanupPolicy log_cleanup_policy log.cleanup.policy –cleanup-policy
RetentionHours log_retention_hours log.retention.hours –retention-hours
RetentionBytes log_retention_bytes log.retention.bytes –retention-bytes

Special Configuration Handling

TLS Certificate Loading

When TLS is enabled, certificates are loaded during configuration initialization:

if cfg.UseTLS && cfg.TLSCertPath != "" && cfg.TLSKeyPath != "" {
    cert, err := tls.LoadX509KeyPair(cfg.TLSCertPath, cfg.TLSKeyPath)
    if err != nil {
        return nil, err
    }
    cfg.TLSCert = cert
}

The loaded certificate is stored in the TLSCert field of the Config struct and used by the server when establishing TLS connections.

Bootstrap Servers Parsing

The bootstrap_servers field supports comma-separated values in a single string, which are automatically split:

if len(cfg.BootstrapServers) == 1 && strings.Contains(cfg.BootstrapServers[0], ",") {
    cfg.BootstrapServers = strings.Split(cfg.BootstrapServers[0], ",")
}

This allows configuration like:

bootstrap_servers: "broker1:9000,broker2:9000,broker3:9000"

SDK Client Configuration

Producer delivery

NewDefaultPublisherConfig chooses duplicate-safe acknowledged delivery:

Parameter Default Meaning
acks all Wait for the captured in-sync replica set and committed HWM
enable_idempotence true Fence and deduplicate retries by producer epoch and partition sequence
max_retries 3 Bound retries inside one send attempt

Keep acks=all and size the broker or topic min_in_sync_replicas for the required failure tolerance. To opt into leader-only delivery, set both acks: "1" and enable_idempotence: false; setting only acks: "1" fails validation. This explicit weak mode can lose a leader-only tail during failover. It also stops automatic retry after a partial write or lost acknowledgement and returns ProducerOutcomeUnknownError, because the broker may already have stored the record.

Client TLS

PublisherConfig, ConsumerConfig, and AdminConfig use the same TLS settings and verification rules:

Parameter Type Default Description
use_tls bool false Enable TLS for every broker connection
tls_ca_path string ”” Optional PEM CA bundle added to the platform trust store
tls_server_name string ”” Optional certificate DNS name; when empty, the SDK verifies the dialed broker host
tls_cert_path string ”” Optional PEM client certificate for mutual TLS
tls_key_path string ”” Optional PEM private key; must be set together with tls_cert_path

Server-authenticated TLS only needs a CA bundle when the broker certificate is not already trusted by the host:

consumer:
  broker_addrs: ["broker1:9000"]
  topic: "orders"
  group_id: "my-group"
  use_tls: true
  tls_ca_path: "/var/run/secrets/cursus/ca.crt"
  tls_server_name: "broker.cursus.svc.cluster.local"

For mutual TLS, also set both client identity paths:

  tls_cert_path: "/var/run/secrets/cursus/tls.crt"
  tls_key_path: "/var/run/secrets/cursus/tls.key"

When use_tls is enabled, every SDK client uses the shared transport dialer and performs a context-bounded TLS handshake with TLS 1.2 minimum before Wire v2 negotiation. Certificate and hostname verification are always enabled; the SDK does not expose an insecure verification bypass.

Mount CA and client identity files from a read-only Kubernetes Secret rather than putting PEM data in a ConfigMap or image:

volumes:
  - name: cursus-client-tls
    secret:
      secretName: cursus-client-tls
containers:
  - name: application
    volumeMounts:
      - name: cursus-client-tls
        mountPath: /var/run/secrets/cursus
        readOnly: true

The SDK reads certificates when the client is constructed. After Kubernetes updates a mounted Secret, create replacement SDK clients and drain the old ones. Rotate a private CA in three stages: first distribute a bundle containing both old and new CA certificates, then rotate broker and client identities, and finally remove the old CA after all processes have reloaded the new bundle.

SDK Metrics (Prometheus)

Both PublisherConfig and ConsumerConfig support an enable_metrics field to opt in to Prometheus runtime metrics.

Parameter Type Default Description
enable_metrics bool false Enable Prometheus runtime metric collection
auto_offset_reset string earliest Missing/out-of-range offset policy: earliest, latest, or error
read_isolation string read_committed Consumer visibility: read_committed or read_uncommitted
handler_max_retries int 3 Additional attempts for the same record after a handler error
handler_retry_backoff duration 100ms Initial delay between handler attempts
handler_retry_max_backoff duration 1s Maximum delay between handler attempts

Exhausting the handler retry budget stops the consumer and returns a typed ConsumerHandlerError from Start; it does not commit the failed batch or trigger a group rebalance. auto_offset_reset: error similarly stops the consumer with ConsumerOffsetOutOfRangeError. After Done() closes, Consumer.Err() returns the first fatal runtime cause.

When enabled, the SDK registers the following metrics in a dedicated Prometheus registry:

Producer Metrics:

Metric Type Labels Description
cursus_producer_messages_sent_total Counter topic Messages successfully sent
cursus_producer_send_errors_total Counter topic Send errors
cursus_producer_batch_latency_seconds Histogram topic Batch send latency

Consumer Metrics:

Metric Type Labels Description
cursus_consumer_messages_received_total Counter topic, group Messages received
cursus_consumer_commit_total Counter topic, group Offset commits
cursus_consumer_commit_errors_total Counter topic, group Commit errors
cursus_consumer_poll_latency_seconds Histogram topic, group Poll operation latency
cursus_consumer_rebalance_total Counter topic, group Rebalance events
cursus_consumer_offset_gap_total Counter topic, group Offsets skipped by the configured reset policy
cursus_consumer_compacted_offsets_skipped_total Counter topic, group Logical offset holes skipped because the topic metadata declares compaction
cursus_consumer_stale_workers_total Counter topic, group, worker Assignment workers fenced by a newer generation

To expose metrics via HTTP:

import "github.com/cursus-io/cursus/sdk"

http.Handle("/metrics", sdk.MetricsHandler())
log.Fatal(http.ListenAndServe(":2112", nil))

Configuration Validation

Config.Normalize() applies safe fallbacks for invalid or non-positive values, including write batching, sync intervals, segment/index sizes, retention intervals, channel capacities, replica settings, transaction/producer retention, and topic registry limits. TLS certificate loading still fails startup when configured files are invalid.

Topic materialization is bounded by max_topics (default 10000), max_partitions_per_topic (default 1024), and max_partitions (default 100000). The matching environment variables are MAX_TOPICS, MAX_PARTITIONS_PER_TOPIC, and MAX_PARTITIONS. CREATE validates distributed authoritative metadata before committing a Raft entry, and every broker checks the same limits while materializing or restoring definitions. Lowering a limit below persisted state therefore makes readiness fail instead of loading an unbounded registry; raise the limit or deliberately remove topics before the change.

Transaction memory and journal growth are bounded by max_transactions, max_transaction_records, max_transaction_bytes, and max_transaction_offsets. The corresponding environment variables are the upper-case YAML names. Limits are checked before mutation and again when installing journal or Raft snapshot state. Lowering a limit below durable state causes startup or readiness to fail with the offending bound rather than silently dropping transaction state. Completed identities continue to count until transactional_id_expiration_ms pruning removes them because their producer epochs are part of the fencing contract.

Cleanup policy values normalize to delete, compact, or canonical delete,compact; unknown values fall back to delete with a warning. Distributed application topics accept compact policies only after every active broker advertises lifecycle protocol version 2, and cleaner passes wait for full ISR plus authoritative, matching HWM/lifecycle/policy state. Event-sourcing topics always require delete. Operators should treat normalization and policy errors as configuration/provisioning failures and verify the effective topic policy with METADATA.

Missing values fall back to defaults in pkg/config/properties.go. The effective order is defaults, parsed flags, configuration file, supported environment overrides, then normalization and validation; --raft-peers is the documented post-file flag exception.