Cursus has a durable partition-log data path and separate coordinator control paths. The in-process subscription channels are an optimization/API for embedded consumers; network consumers use CONSUME or STREAM against partition logs.
sequenceDiagram
participant C as Producer
participant B as Broker controller
participant T as Topic/Partition leader
participant D as DiskHandler
participant R as Replicas
C->>B: PUBLISH topic=... key=... message=...
B->>B: authenticate, authorize, parse policy
B->>T: route to partition leader
T->>T: producer epoch/sequence and dedup checks
T->>D: append assigned offset
D-->>T: local write result
T->>R: replicate with leader/epoch fence
R-->>T: ISR/quorum result
T->>T: advance committed HWM
T-->>C: OK/ack with assigned offset
Keyed records use FNV-1a 64-bit hash modulo partition count when the topic policy is hash_key; unkeyed records use round-robin. round_robin policy ignores keys. Ordering is defined only within one partition.
Acknowledgement strength depends on the publish request’s acks setting and distribution mode. acks=0 emits no external response frame. acks=1 completes after the leader’s durable append and schedules follower work on a bounded, partition-ordered lane; the record is not visible to read-committed consumers and can be lost on failover until that lane commits it. acks=all and acks=-1 first require the current ISR to meet the topic-effective minimum, then complete only after the captured ISR acknowledges and the fenced HWM commit succeeds. Non-ISR catch-up runs outside that completion condition. Async enqueue, buffered-writer flush, file sync, replica append, and committed HWM are distinct milestones.
The topic-effective minimum is min_in_sync_replicas from durable topic policy when present and the broker min_insync_replicas fallback otherwise. acks itself is never persisted in topic metadata. Idempotent publishing requires all or -1.
sequenceDiagram
participant C as Consumer
participant G as Group coordinator
participant L as Partition leader
C->>G: JOIN_GROUP / SYNC_GROUP
G-->>C: member, generation, assignments
C->>G: FETCH_OFFSET per assignment
G-->>C: committed nextOffset
C->>L: CONSUME or STREAM isolation=read_committed
L-->>C: records bounded by HWM/LSO
C->>C: process records
C->>G: BATCH_COMMIT lastProcessedOffset+1
G-->>C: OK or fencing/error
The broker committed offset is authoritative. A lower request offset does not replay records before an existing commit. A missing offset follows autoOffsetReset; an offset removed by retention returns OFFSET_OUT_OF_RANGE.
read_committed is the default: it returns ordinary records and committed transaction output, skips aborted/control records, and stops at the earliest unresolved transaction. read_uncommitted exposes the raw committed partition log.
sequenceDiagram
participant C as Transactional client
participant X as Transaction coordinator
participant P as Output partition leaders
participant G as Group coordinator
C->>X: INIT_PRODUCER_ID, BEGIN_TXN
C->>X: TXN_PUBLISH
X->>P: append unresolved idempotent record
C->>X: SEND_OFFSETS_TO_TXN (one group session, multiple topics)
C->>X: END_TXN result=commit
X->>X: persist prepare_commit
X->>G: atomic fenced multi-topic offset commit
X->>P: append commit markers
X->>X: persist committed decision
X-->>C: OK state=committed
Output records remain invisible to read_committed until the partition marker and final coordinator decision agree. A restored prepared transaction is retried, while an open transaction past transaction_timeout_ms is durably aborted. Recovery work is indexed by coordinator shard and drained in bounded batches, so retained transaction history is not scanned on every monitor tick. Within this boundary, Cursus input offsets and output records form an exactly-once broker operation. Each completed epoch must be reinitialized before the next transaction; uncertain finalization retry keeps the old epoch.
APPEND_STREAM hashes the aggregate key to a partition leader, checks version = current + 1, appends/replicates the event, advances committed state, and updates the stream index. READ_STREAM and STREAM_VERSION are leader-routed and bounded by committed HWM. Snapshots are quorum-replicated optimizations; committed event replay remains authoritative.
TopicManager.RegisterConsumerGroup creates in-process partition/group/consumer channels using configured capacities. It provides low-latency fan-out inside the broker process, but it is not the dynamic network group coordinator and does not replace durable offset commits, generation fencing, or log replay.
NOT_LEADER or NOT_COORDINATOR with redirect data,