Hello MQ

外观

Apache Kafka 分区与分发 ​

本页结论：Kafka 的「路由」是 key 到分区的哈希映射——它同时决定局部顺序与负载分布；消费组在分区之上做分配，组内瓜分、组间广播。

适用场景 ​

  • 同业务键有序的事件流（key=orderId → 同分区 FIFO）。
  • 多消费者水平扩展：分区数是组内消费者的并行度上限。
  • 多个下游独立消费：每组一份完整数据。

核心模型：key → Partition → Consumer Group ​

  • 同一 key 的消息总是进同一分区 → 分区内顺序写入（ordering-replay 实验断言 samePartitionOnProduce=1）。
  • 无 key 的消息按 Sticky 策略分布到各分区（批量内轮转、批间尽量粘连），目的是摊匀负载并保留批内局部性。
  • 跨分区没有全局顺序——「Kafka 保证全局顺序」是禁止表述。

动手验证：

bash
bash demos/kafka/ordering-replay/run.sh
kafka / ordering-replaybash demos/kafka/ordering-replay/run.sh
assert.out.txt
PASS produced: 6
PASS samePartitionOnProduce: 1
PASS samePartitionOnConsume: 1
PASS observedOrder: 1,2,3,4,5,6
PASS replayed: 6
PASS replayFromOffset0: 0
PASS replayUniqueMessageIds: 6
RESULT: all assertions passed
consumer-g1.out.txt
SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".
SLF4J: Defaulting to no-operation (NOP) logger implementation
SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for further details.
[consumer] timestamp=2026-10-07T10:31:21.433006938Z level=INFO service=order-service product=kafka lab=ordering-replay destination=orders.ordering consumerGroup=orders-ordering-g1 consumer=g1 partitions=0,1,2 status=assigned
[consumer] timestamp=2026-10-07T10:31:21.576699243Z level=INFO service=order-service product=kafka lab=ordering-replay messageId=01M4AYNTWRVY565400WFNPG35Q eventType=order.created schemaVersion=1 aggregateId=order-1001 traceId=d90404379644dfb90d4c6235a5223c46 correlationId=order-1001 destination=orders.ordering partitionOrQueue=0 offset=0 seq=1 consumerGroup=orders-ordering-g1 consumer=g1 attempt=1 redelivered=false status=received
[replay-gate] timestamp=2026-10-07T10:31:21.581357009Z checkpoint=after-delivery messageId=01M4AYNTWRVY565400WFNPG35Q token=892653771a87-1-after-delivery-01M4AYNTWRVY565400WFNPG35Q-1 status=waiting
[replay-gate] timestamp=2026-10-07T10:31:26.793634890Z checkpoint=after-delivery messageId=01M4AYNTWRVY565400WFNPG35Q token=892653771a87-1-after-delivery-01M4AYNTWRVY565400WFNPG35Q-1 status=released
[consumer] timestamp=2026-10-07T10:31:26.800784393Z level=INFO service=order-service product=kafka lab=ordering-replay messageId=01M4AYNTWRVY565400WFNPG35Q eventType=order.created schemaVersion=1 aggregateId=order-1001 traceId=d90404379644dfb90d4c6235a5223c46 correlationId=order-1001 destination=orders.ordering partitionOrQueue=0 seq=1 consumerGroup=orders-ordering-g1 attempt=1 status=business_committed
[replay-gate] timestamp=2026-10-07T10:31:26.801122875Z checkpoint=before-offset-commit messageId=01M4AYNTWRVY565400WFNPG35Q token=892653771a87-1-before-offset-commit-01M4AYNTWRVY565400WFNPG35Q-2 status=waiting
[replay-gate] timestamp=2026-10-07T10:31:32.012261515Z checkpoint=before-offset-commit messageId=01M4AYNTWRVY565400WFNPG35Q token=892653771a87-1-before-offset-commit-01M4AYNTWRVY565400WFNPG35Q-2 status=released
[consumer] timestamp=2026-10-07T10:31:32.027308260Z level=INFO service=order-service product=kafka lab=ordering-replay messageId=01M4AYNV3WE4259J00JWBS1P0H eventType=order.created schemaVersion=1 aggregateId=order-1001 traceId=a29a0f51a47325e83190635b194c9332 correlationId=order-1001 destination=orders.ordering partitionOrQueue=0 offset=1 seq=2 consumerGroup=orders-ordering-g1 consumer=g1 attempt=1 redelivered=false status=received
[replay-gate] timestamp=2026-10-07T10:31:32.027704299Z checkpoint=after-delivery messageId=01M4AYNV3WE4259J00JWBS1P0H token=892653771a87-1-after-delivery-01M4AYNV3WE4259J00JWBS1P0H-3 status=waiting
[replay-gate] timestamp=2026-10-07T10:31:36.938705036Z checkpoint=after-delivery messageId=01M4AYNV3WE4259J00JWBS1P0H token=892653771a87-1-after-delivery-01M4AYNV3WE4259J00JWBS1P0H-3 status=released
[consumer] timestamp=2026-10-07T10:31:36.941830509Z level=INFO service=order-service product=kafka lab=ordering-replay messageId=01M4AYNV3WE4259J00JWBS1P0H eventType=order.created schemaVersion=1 aggregateId=order-1001 traceId=a29a0f51a47325e83190635b194c9332 correlationId=order-1001 destination=orders.ordering partitionOrQueue=0 seq=2 consumerGroup=orders-ordering-g1 attempt=1 status=duplicate_skipped
[replay-gate] timestamp=2026-10-07T10:31:36.942646233Z checkpoint=before-offset-commit messageId=01M4AYNV3WE4259J00JWBS1P0H token=892653771a87-1-before-offset-commit-01M4AYNV3WE4259J00JWBS1P0H-4 status=waiting
[replay-gate] timestamp=2026-10-07T10:31:42.152023532Z checkpoint=before-offset-commit messageId=01M4AYNV3WE4259J00JWBS1P0H token=892653771a87-1-before-offset-commit-01M4AYNV3WE4259J00JWBS1P0H-4 status=released
[consumer] timestamp=2026-10-07T10:31:42.155152381Z level=INFO service=order-service product=kafka lab=ordering-replay messageId=01M4AYNV46YGVHYV0031KF0VE7 eventType=order.created schemaVersion=1 aggregateId=order-1001 traceId=14e5b3fa597e10661a81bd3df5f832c3 correlationId=order-1001 destination=orders.ordering partitionOrQueue=0 offset=2 seq=3 consumerGroup=orders-ordering-g1 consumer=g1 attempt=1 redelivered=false status=received
[replay-gate] timestamp=2026-10-07T10:31:42.155455038Z checkpoint=after-delivery messageId=01M4AYNV46YGVHYV0031KF0VE7 token=892653771a87-1-after-delivery-01M4AYNV46YGVHYV0031KF0VE7-5 status=waiting
[replay-gate] timestamp=2026-10-07T10:31:47.263570135Z checkpoint=after-delivery messageId=01M4AYNV46YGVHYV0031KF0VE7 token=892653771a87-1-after-delivery-01M4AYNV46YGVHYV0031KF0VE7-5 status=released
[consumer] timestamp=2026-10-07T10:31:47.264335943Z level=INFO service=order-service product=kafka lab=ordering-replay messageId=01M4AYNV46YGVHYV0031KF0VE7 eventType=order.created schemaVersion=1 aggregateId=order-1001 traceId=14e5b3fa597e10661a81bd3df5f832c3 correlationId=order-1001 destination=orders.ordering partitionOrQueue=0 seq=3 consumerGroup=orders-ordering-g1 attempt=1 status=duplicate_skipped
[replay-gate] timestamp=2026-10-07T10:31:47.264642416Z checkpoint=before-offset-commit messageId=01M4AYNV46YGVHYV0031KF0VE7 token=892653771a87-1-before-offset-commit-01M4AYNV46YGVHYV0031KF0VE7-6 status=waiting
[replay-gate] timestamp=2026-10-07T10:31:52.371696899Z checkpoint=before-offset-commit messageId=01M4AYNV46YGVHYV0031KF0VE7 token=892653771a87-1-before-offset-commit-01M4AYNV46YGVHYV0031KF0VE7-6 status=released
[consumer] timestamp=2026-10-07T10:31:52.374695997Z level=INFO service=order-service product=kafka lab=ordering-replay messageId=01M4AYNV4FF47RWJ00RCEAPZ13 eventType=order.created schemaVersion=1 aggregateId=order-1001 traceId=b02b77c6685c61c60e1483e47be4fd34 correlationId=order-1001 destination=orders.ordering partitionOrQueue=0 offset=3 seq=4 consumerGroup=orders-ordering-g1 consumer=g1 attempt=1 redelivered=false status=received
[replay-gate] timestamp=2026-10-07T10:31:52.374989521Z checkpoint=after-delivery messageId=01M4AYNV4FF47RWJ00RCEAPZ13 token=892653771a87-1-after-delivery-01M4AYNV4FF47RWJ00RCEAPZ13-7 status=waiting
[replay-gate] timestamp=2026-10-07T10:31:57.382788167Z checkpoint=after-delivery messageId=01M4AYNV4FF47RWJ00RCEAPZ13 token=892653771a87-1-after-delivery-01M4AYNV4FF47RWJ00RCEAPZ13-7 status=released
[consumer] timestamp=2026-10-07T10:31:57.383684531Z level=INFO service=order-service product=kafka lab=ordering-replay messageId=01M4AYNV4FF47RWJ00RCEAPZ13 eventType=order.created schemaVersion=1 aggregateId=order-1001 traceId=b02b77c6685c61c60e1483e47be4fd34 correlationId=order-1001 destination=orders.ordering partitionOrQueue=0 seq=4 consumerGroup=orders-ordering-g1 attempt=1 status=duplicate_skipped
[replay-gate] timestamp=2026-10-07T10:31:57.384019777Z checkpoint=before-offset-commit messageId=01M4AYNV4FF47RWJ00RCEAPZ13 token=892653771a87-1-before-offset-commit-01M4AYNV4FF47RWJ00RCEAPZ13-8 status=waiting
[replay-gate] timestamp=2026-10-07T10:32:02.290345562Z checkpoint=before-offset-commit messageId=01M4AYNV4FF47RWJ00RCEAPZ13 token=892653771a87-1-before-offset-commit-01M4AYNV4FF47RWJ00RCEAPZ13-8 status=released
[consumer] timestamp=2026-10-07T10:32:02.293397502Z level=INFO service=order-service product=kafka lab=ordering-replay messageId=01M4AYNV4Q2353KF009QARRRN2 eventType=order.created schemaVersion=1 aggregateId=order-1001 traceId=270155c7f257ba6ae090593ebee53e2f correlationId=order-1001 destination=orders.ordering partitionOrQueue=0 offset=4 seq=5 consumerGroup=orders-ordering-g1 consumer=g1 attempt=1 redelivered=false status=received
[replay-gate] timestamp=2026-10-07T10:32:02.293677074Z checkpoint=after-delivery messageId=01M4AYNV4Q2353KF009QARRRN2 token=892653771a87-1-after-delivery-01M4AYNV4Q2353KF009QARRRN2-9 status=waiting
[replay-gate] timestamp=2026-10-07T10:32:07.504818027Z checkpoint=after-delivery messageId=01M4AYNV4Q2353KF009QARRRN2 token=892653771a87-1-after-delivery-01M4AYNV4Q2353KF009QARRRN2-9 status=released
[consumer] timestamp=2026-10-07T10:32:07.505596884Z level=INFO service=order-service product=kafka lab=ordering-replay messageId=01M4AYNV4Q2353KF009QARRRN2 eventType=order.created schemaVersion=1 aggregateId=order-1001 traceId=270155c7f257ba6ae090593ebee53e2f correlationId=order-1001 destination=orders.ordering partitionOrQueue=0 seq=5 consumerGroup=orders-ordering-g1 attempt=1 status=duplicate_skipped
[replay-gate] timestamp=2026-10-07T10:32:07.505894724Z checkpoint=before-offset-commit messageId=01M4AYNV4Q2353KF009QARRRN2 token=892653771a87-1-before-offset-commit-01M4AYNV4Q2353KF009QARRRN2-10 status=waiting
[replay-gate] timestamp=2026-10-07T10:32:12.713231644Z checkpoint=before-offset-commit messageId=01M4AYNV4Q2353KF009QARRRN2 token=892653771a87-1-before-offset-commit-01M4AYNV4Q2353KF009QARRRN2-10 status=released
[consumer] timestamp=2026-10-07T10:32:12.716002793Z level=INFO service=order-service product=kafka lab=ordering-replay messageId=01M4AYNV4YKDM0S500CF7WN13N eventType=order.created schemaVersion=1 aggregateId=order-1001 traceId=f176ee593016c2b4754d7a25efe74696 correlationId=order-1001 destination=orders.ordering partitionOrQueue=0 offset=5 seq=6 consumerGroup=orders-ordering-g1 consumer=g1 attempt=1 redelivered=false status=received
[replay-gate] timestamp=2026-10-07T10:32:12.716277708Z checkpoint=after-delivery messageId=01M4AYNV4YKDM0S500CF7WN13N token=892653771a87-1-after-delivery-01M4AYNV4YKDM0S500CF7WN13N-11 status=waiting
[replay-gate] timestamp=2026-10-07T10:32:17.924516289Z checkpoint=after-delivery messageId=01M4AYNV4YKDM0S500CF7WN13N token=892653771a87-1-after-delivery-01M4AYNV4YKDM0S500CF7WN13N-11 status=released
[consumer] timestamp=2026-10-07T10:32:17.925263128Z level=INFO service=order-service product=kafka lab=ordering-replay messageId=01M4AYNV4YKDM0S500CF7WN13N eventType=order.created schemaVersion=1 aggregateId=order-1001 traceId=f176ee593016c2b4754d7a25efe74696 correlationId=order-1001 destination=orders.ordering partitionOrQueue=0 seq=6 consumerGroup=orders-ordering-g1 attempt=1 status=duplicate_skipped
[replay-gate] timestamp=2026-10-07T10:32:17.925514528Z checkpoint=before-offset-commit messageId=01M4AYNV4YKDM0S500CF7WN13N token=892653771a87-1-before-offset-commit-01M4AYNV4YKDM0S500CF7WN13N-12 status=waiting
[replay-gate] timestamp=2026-10-07T10:32:22.832944437Z checkpoint=before-offset-commit messageId=01M4AYNV4YKDM0S500CF7WN13N token=892653771a87-1-before-offset-commit-01M4AYNV4YKDM0S500CF7WN13N-12 status=released
[consumer] timestamp=2026-10-07T10:32:22.835289862Z level=INFO service=order-service product=kafka lab=ordering-replay destination=orders.ordering consumer=g1 received=6 status=done
consumer-g2.out.txt
SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".
SLF4J: Defaulting to no-operation (NOP) logger implementation
SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for further details.
[consumer] timestamp=2026-10-07T10:32:24.187352546Z level=INFO service=order-service product=kafka lab=ordering-replay destination=orders.ordering consumerGroup=orders-ordering-g2 consumer=g2 partitions=0,1,2 status=assigned
[consumer] timestamp=2026-10-07T10:32:24.415778295Z level=INFO service=order-service product=kafka lab=ordering-replay messageId=01M4AYNTWRVY565400WFNPG35Q eventType=order.created schemaVersion=1 aggregateId=order-1001 traceId=d90404379644dfb90d4c6235a5223c46 correlationId=order-1001 destination=orders.ordering partitionOrQueue=0 offset=0 seq=1 consumerGroup=orders-ordering-g2 consumer=g2 attempt=1 redelivered=false status=received
[replay-gate] timestamp=2026-10-07T10:32:24.428512999Z checkpoint=after-delivery messageId=01M4AYNTWRVY565400WFNPG35Q token=957c49239d57-1-after-delivery-01M4AYNTWRVY565400WFNPG35Q-1 status=waiting
[replay-gate] timestamp=2026-10-07T10:32:34.146449900Z checkpoint=after-delivery messageId=01M4AYNTWRVY565400WFNPG35Q token=957c49239d57-1-after-delivery-01M4AYNTWRVY565400WFNPG35Q-1 status=released
[consumer] timestamp=2026-10-07T10:32:34.153612551Z level=INFO service=order-service product=kafka lab=ordering-replay messageId=01M4AYNTWRVY565400WFNPG35Q eventType=order.created schemaVersion=1 aggregateId=order-1001 traceId=d90404379644dfb90d4c6235a5223c46 correlationId=order-1001 destination=orders.ordering partitionOrQueue=0 seq=1 consumerGroup=orders-ordering-g2 attempt=1 status=business_committed
[replay-gate] timestamp=2026-10-07T10:32:34.153984583Z checkpoint=before-offset-commit messageId=01M4AYNTWRVY565400WFNPG35Q token=957c49239d57-1-before-offset-commit-01M4AYNTWRVY565400WFNPG35Q-2 status=waiting
[replay-gate] timestamp=2026-10-07T10:32:39.463700786Z checkpoint=before-offset-commit messageId=01M4AYNTWRVY565400WFNPG35Q token=957c49239d57-1-before-offset-commit-01M4AYNTWRVY565400WFNPG35Q-2 status=released
[consumer] timestamp=2026-10-07T10:32:39.468143800Z level=INFO service=order-service product=kafka lab=ordering-replay messageId=01M4AYNV3WE4259J00JWBS1P0H eventType=order.created schemaVersion=1 aggregateId=order-1001 traceId=a29a0f51a47325e83190635b194c9332 correlationId=order-1001 destination=orders.ordering partitionOrQueue=0 offset=1 seq=2 consumerGroup=orders-ordering-g2 consumer=g2 attempt=1 redelivered=false status=received
[replay-gate] timestamp=2026-10-07T10:32:39.468524305Z checkpoint=after-delivery messageId=01M4AYNV3WE4259J00JWBS1P0H token=957c49239d57-1-after-delivery-01M4AYNV3WE4259J00JWBS1P0H-3 status=waiting
[replay-gate] timestamp=2026-10-07T10:32:44.377402507Z checkpoint=after-delivery messageId=01M4AYNV3WE4259J00JWBS1P0H token=957c49239d57-1-after-delivery-01M4AYNV3WE4259J00JWBS1P0H-3 status=released
[consumer] timestamp=2026-10-07T10:32:44.380236870Z level=INFO service=order-service product=kafka lab=ordering-replay messageId=01M4AYNV3WE4259J00JWBS1P0H eventType=order.created schemaVersion=1 aggregateId=order-1001 traceId=a29a0f51a47325e83190635b194c9332 correlationId=order-1001 destination=orders.ordering partitionOrQueue=0 seq=2 consumerGroup=orders-ordering-g2 attempt=1 status=duplicate_skipped
[replay-gate] timestamp=2026-10-07T10:32:44.380621351Z checkpoint=before-offset-commit messageId=01M4AYNV3WE4259J00JWBS1P0H token=957c49239d57-1-before-offset-commit-01M4AYNV3WE4259J00JWBS1P0H-4 status=waiting
[replay-gate] timestamp=2026-10-07T10:32:49.589400895Z checkpoint=before-offset-commit messageId=01M4AYNV3WE4259J00JWBS1P0H token=957c49239d57-1-before-offset-commit-01M4AYNV3WE4259J00JWBS1P0H-4 status=released
[consumer] timestamp=2026-10-07T10:32:49.592704759Z level=INFO service=order-service product=kafka lab=ordering-replay messageId=01M4AYNV46YGVHYV0031KF0VE7 eventType=order.created schemaVersion=1 aggregateId=order-1001 traceId=14e5b3fa597e10661a81bd3df5f832c3 correlationId=order-1001 destination=orders.ordering partitionOrQueue=0 offset=2 seq=3 consumerGroup=orders-ordering-g2 consumer=g2 attempt=1 redelivered=false status=received
[replay-gate] timestamp=2026-10-07T10:32:49.593072745Z checkpoint=after-delivery messageId=01M4AYNV46YGVHYV0031KF0VE7 token=957c49239d57-1-after-delivery-01M4AYNV46YGVHYV0031KF0VE7-5 status=waiting
[replay-gate] timestamp=2026-10-07T10:32:54.804035878Z checkpoint=after-delivery messageId=01M4AYNV46YGVHYV0031KF0VE7 token=957c49239d57-1-after-delivery-01M4AYNV46YGVHYV0031KF0VE7-5 status=released
[consumer] timestamp=2026-10-07T10:32:54.804830048Z level=INFO service=order-service product=kafka lab=ordering-replay messageId=01M4AYNV46YGVHYV0031KF0VE7 eventType=order.created schemaVersion=1 aggregateId=order-1001 traceId=14e5b3fa597e10661a81bd3df5f832c3 correlationId=order-1001 destination=orders.ordering partitionOrQueue=0 seq=3 consumerGroup=orders-ordering-g2 attempt=1 status=duplicate_skipped
[replay-gate] timestamp=2026-10-07T10:32:54.805142510Z checkpoint=before-offset-commit messageId=01M4AYNV46YGVHYV0031KF0VE7 token=957c49239d57-1-before-offset-commit-01M4AYNV46YGVHYV0031KF0VE7-6 status=waiting
[replay-gate] timestamp=2026-10-07T10:33:00.032047155Z checkpoint=before-offset-commit messageId=01M4AYNV46YGVHYV0031KF0VE7 token=957c49239d57-1-before-offset-commit-01M4AYNV46YGVHYV0031KF0VE7-6 status=released
[consumer] timestamp=2026-10-07T10:33:00.035128512Z level=INFO service=order-service product=kafka lab=ordering-replay messageId=01M4AYNV4FF47RWJ00RCEAPZ13 eventType=order.created schemaVersion=1 aggregateId=order-1001 traceId=b02b77c6685c61c60e1483e47be4fd34 correlationId=order-1001 destination=orders.ordering partitionOrQueue=0 offset=3 seq=4 consumerGroup=orders-ordering-g2 consumer=g2 attempt=1 redelivered=false status=received
[replay-gate] timestamp=2026-10-07T10:33:00.035410779Z checkpoint=after-delivery messageId=01M4AYNV4FF47RWJ00RCEAPZ13 token=957c49239d57-1-after-delivery-01M4AYNV4FF47RWJ00RCEAPZ13-7 status=waiting
[replay-gate] timestamp=2026-10-07T10:33:05.042282605Z checkpoint=after-delivery messageId=01M4AYNV4FF47RWJ00RCEAPZ13 token=957c49239d57-1-after-delivery-01M4AYNV4FF47RWJ00RCEAPZ13-7 status=released
[consumer] timestamp=2026-10-07T10:33:05.044618816Z level=INFO service=order-service product=kafka lab=ordering-replay messageId=01M4AYNV4FF47RWJ00RCEAPZ13 eventType=order.created schemaVersion=1 aggregateId=order-1001 traceId=b02b77c6685c61c60e1483e47be4fd34 correlationId=order-1001 destination=orders.ordering partitionOrQueue=0 seq=4 consumerGroup=orders-ordering-g2 attempt=1 status=duplicate_skipped
[replay-gate] timestamp=2026-10-07T10:33:05.044953522Z checkpoint=before-offset-commit messageId=01M4AYNV4FF47RWJ00RCEAPZ13 token=957c49239d57-1-before-offset-commit-01M4AYNV4FF47RWJ00RCEAPZ13-8 status=waiting
[replay-gate] timestamp=2026-10-07T10:33:10.051877560Z checkpoint=before-offset-commit messageId=01M4AYNV4FF47RWJ00RCEAPZ13 token=957c49239d57-1-before-offset-commit-01M4AYNV4FF47RWJ00RCEAPZ13-8 status=released
[consumer] timestamp=2026-10-07T10:33:10.054889843Z level=INFO service=order-service product=kafka lab=ordering-replay messageId=01M4AYNV4Q2353KF009QARRRN2 eventType=order.created schemaVersion=1 aggregateId=order-1001 traceId=270155c7f257ba6ae090593ebee53e2f correlationId=order-1001 destination=orders.ordering partitionOrQueue=0 offset=4 seq=5 consumerGroup=orders-ordering-g2 consumer=g2 attempt=1 redelivered=false status=received
[replay-gate] timestamp=2026-10-07T10:33:10.055211139Z checkpoint=after-delivery messageId=01M4AYNV4Q2353KF009QARRRN2 token=957c49239d57-1-after-delivery-01M4AYNV4Q2353KF009QARRRN2-9 status=waiting
[replay-gate] timestamp=2026-10-07T10:33:15.162800333Z checkpoint=after-delivery messageId=01M4AYNV4Q2353KF009QARRRN2 token=957c49239d57-1-after-delivery-01M4AYNV4Q2353KF009QARRRN2-9 status=released
[consumer] timestamp=2026-10-07T10:33:15.163637879Z level=INFO service=order-service product=kafka lab=ordering-replay messageId=01M4AYNV4Q2353KF009QARRRN2 eventType=order.created schemaVersion=1 aggregateId=order-1001 traceId=270155c7f257ba6ae090593ebee53e2f correlationId=order-1001 destination=orders.ordering partitionOrQueue=0 seq=5 consumerGroup=orders-ordering-g2 attempt=1 status=duplicate_skipped
[replay-gate] timestamp=2026-10-07T10:33:15.163952365Z checkpoint=before-offset-commit messageId=01M4AYNV4Q2353KF009QARRRN2 token=957c49239d57-1-before-offset-commit-01M4AYNV4Q2353KF009QARRRN2-10 status=waiting
[replay-gate] timestamp=2026-10-07T10:33:20.372748762Z checkpoint=before-offset-commit messageId=01M4AYNV4Q2353KF009QARRRN2 token=957c49239d57-1-before-offset-commit-01M4AYNV4Q2353KF009QARRRN2-10 status=released
[consumer] timestamp=2026-10-07T10:33:20.375676046Z level=INFO service=order-service product=kafka lab=ordering-replay messageId=01M4AYNV4YKDM0S500CF7WN13N eventType=order.created schemaVersion=1 aggregateId=order-1001 traceId=f176ee593016c2b4754d7a25efe74696 correlationId=order-1001 destination=orders.ordering partitionOrQueue=0 offset=5 seq=6 consumerGroup=orders-ordering-g2 consumer=g2 attempt=1 redelivered=false status=received
[replay-gate] timestamp=2026-10-07T10:33:20.375941738Z checkpoint=after-delivery messageId=01M4AYNV4YKDM0S500CF7WN13N token=957c49239d57-1-after-delivery-01M4AYNV4YKDM0S500CF7WN13N-11 status=waiting
[replay-gate] timestamp=2026-10-07T10:33:25.382582296Z checkpoint=after-delivery messageId=01M4AYNV4YKDM0S500CF7WN13N token=957c49239d57-1-after-delivery-01M4AYNV4YKDM0S500CF7WN13N-11 status=released
[consumer] timestamp=2026-10-07T10:33:25.383421655Z level=INFO service=order-service product=kafka lab=ordering-replay messageId=01M4AYNV4YKDM0S500CF7WN13N eventType=order.created schemaVersion=1 aggregateId=order-1001 traceId=f176ee593016c2b4754d7a25efe74696 correlationId=order-1001 destination=orders.ordering partitionOrQueue=0 seq=6 consumerGroup=orders-ordering-g2 attempt=1 status=duplicate_skipped
[replay-gate] timestamp=2026-10-07T10:33:25.383691372Z checkpoint=before-offset-commit messageId=01M4AYNV4YKDM0S500CF7WN13N token=957c49239d57-1-before-offset-commit-01M4AYNV4YKDM0S500CF7WN13N-12 status=waiting
[replay-gate] timestamp=2026-10-07T10:33:30.291713355Z checkpoint=before-offset-commit messageId=01M4AYNV4YKDM0S500CF7WN13N token=957c49239d57-1-before-offset-commit-01M4AYNV4YKDM0S500CF7WN13N-12 status=released
[consumer] timestamp=2026-10-07T10:33:30.294195623Z level=INFO service=order-service product=kafka lab=ordering-replay destination=orders.ordering consumer=g2 received=6 status=done
producer.out.txt
SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".
SLF4J: Defaulting to no-operation (NOP) logger implementation
SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for further details.
[producer] timestamp=2026-10-07T10:31:19.932155727Z level=INFO service=order-service product=kafka lab=ordering-replay messageId=01M4AYNTWRVY565400WFNPG35Q eventType=order.created schemaVersion=1 aggregateId=order-1001 traceId=d90404379644dfb90d4c6235a5223c46 correlationId=order-1001 destination=orders.ordering partitionOrQueue=0 offset=0 seq=1 durationMs=207 status=produced
[producer] timestamp=2026-10-07T10:31:19.942291417Z level=INFO service=order-service product=kafka lab=ordering-replay messageId=01M4AYNV3WE4259J00JWBS1P0H eventType=order.created schemaVersion=1 aggregateId=order-1001 traceId=a29a0f51a47325e83190635b194c9332 correlationId=order-1001 destination=orders.ordering partitionOrQueue=0 offset=1 seq=2 durationMs=8 status=produced
[producer] timestamp=2026-10-07T10:31:19.950899104Z level=INFO service=order-service product=kafka lab=ordering-replay messageId=01M4AYNV46YGVHYV0031KF0VE7 eventType=order.created schemaVersion=1 aggregateId=order-1001 traceId=14e5b3fa597e10661a81bd3df5f832c3 correlationId=order-1001 destination=orders.ordering partitionOrQueue=0 offset=2 seq=3 durationMs=7 status=produced
[producer] timestamp=2026-10-07T10:31:19.958784134Z level=INFO service=order-service product=kafka lab=ordering-replay messageId=01M4AYNV4FF47RWJ00RCEAPZ13 eventType=order.created schemaVersion=1 aggregateId=order-1001 traceId=b02b77c6685c61c60e1483e47be4fd34 correlationId=order-1001 destination=orders.ordering partitionOrQueue=0 offset=3 seq=4 durationMs=7 status=produced
[producer] timestamp=2026-10-07T10:31:19.966595799Z level=INFO service=order-service product=kafka lab=ordering-replay messageId=01M4AYNV4Q2353KF009QARRRN2 eventType=order.created schemaVersion=1 aggregateId=order-1001 traceId=270155c7f257ba6ae090593ebee53e2f correlationId=order-1001 destination=orders.ordering partitionOrQueue=0 offset=4 seq=5 durationMs=7 status=produced
[producer] timestamp=2026-10-07T10:31:19.974045571Z level=INFO service=order-service product=kafka lab=ordering-replay messageId=01M4AYNV4YKDM0S500CF7WN13N eventType=order.created schemaVersion=1 aggregateId=order-1001 traceId=f176ee593016c2b4754d7a25efe74696 correlationId=order-1001 destination=orders.ordering partitionOrQueue=0 offset=5 seq=6 durationMs=6 status=produced
[producer] timestamp=2026-10-07T10:31:19.974155377Z level=INFO service=order-service product=kafka lab=ordering-replay destination=orders.ordering produced=6 status=done
setup.out.txt
SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".
SLF4J: Defaulting to no-operation (NOP) logger implementation
SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for further details.

消费组分配与再均衡 ​

  • 组内：每个分区至多分配给组内一个消费者；消费者数 > 分区数时，多出来的消费者空转（这就是「消费者数量上限 = 分区数」的关系）。
  • 成员变化（加入/退出/崩溃、订阅 Topic 分区数变化）触发 再均衡（rebalance）：期间消费暂停，分区重新分配；已提交 offset 决定新属主从哪继续。
  • 本仓库 consumer-group 实验用两个消费者瓜分 3 分区，随后独立组 b 再次全量接收：
bash
bash demos/kafka/consumer-group/run.sh
kafka / consumer-groupbash demos/kafka/consumer-group/run.sh
assert.out.txt
PASS produced: 3
PASS groupAReceived: 3
PASS groupAUnique: 3
PASS a1Assigned: 1
PASS a2Assigned: 1
PASS consumer-a-1.exitCode: 0
PASS consumer-a-2.exitCode: 0
PASS groupBReceived: 3
PASS groupBBusinessRows: 3
PASS groupALag: 0
RESULT: all assertions passed
consumer-a-1.out.txt
SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".
SLF4J: Defaulting to no-operation (NOP) logger implementation
SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for further details.
[consumer] timestamp=2026-10-07T10:29:40.144165071Z level=INFO service=order-service product=kafka lab=consumer-group destination=orders.group consumerGroup=orders-group-a consumer=a-1 partitions=2 status=assigned
consumer-a-2.out.txt
SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".
SLF4J: Defaulting to no-operation (NOP) logger implementation
SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for further details.
[consumer] timestamp=2026-10-07T10:29:40.139919457Z level=INFO service=order-service product=kafka lab=consumer-group destination=orders.group consumerGroup=orders-group-a consumer=a-2 partitions=0,1 status=assigned
[consumer] timestamp=2026-10-07T10:29:40.439634607Z level=INFO service=order-service product=kafka lab=consumer-group messageId=01JMQ000000000000000000001 eventType=order.created schemaVersion=1 aggregateId=order-1001 traceId=00000000000000000000000000000001 correlationId=order-1001 destination=orders.group partitionOrQueue=0 offset=0 seq=1 consumerGroup=orders-group-a consumer=a-2 attempt=1 redelivered=false status=received
[replay-gate] timestamp=2026-10-07T10:29:40.457417749Z checkpoint=after-delivery messageId=01JMQ000000000000000000001 token=08125c9aaffa-1-after-delivery-01JMQ000000000000000000001-1 status=waiting
[replay-gate] timestamp=2026-10-07T10:29:50.880586511Z checkpoint=after-delivery messageId=01JMQ000000000000000000001 token=08125c9aaffa-1-after-delivery-01JMQ000000000000000000001-1 status=released
[consumer] timestamp=2026-10-07T10:29:50.891766841Z level=INFO service=order-service product=kafka lab=consumer-group messageId=01JMQ000000000000000000001 eventType=order.created schemaVersion=1 aggregateId=order-1001 traceId=00000000000000000000000000000001 correlationId=order-1001 destination=orders.group partitionOrQueue=0 seq=1 consumerGroup=orders-group-a attempt=1 status=business_committed
[replay-gate] timestamp=2026-10-07T10:29:50.892190556Z checkpoint=before-offset-commit messageId=01JMQ000000000000000000001 token=08125c9aaffa-1-before-offset-commit-01JMQ000000000000000000001-2 status=waiting
[replay-gate] timestamp=2026-10-07T10:30:03.216425944Z checkpoint=before-offset-commit messageId=01JMQ000000000000000000001 token=08125c9aaffa-1-before-offset-commit-01JMQ000000000000000000001-2 status=released
[consumer] timestamp=2026-10-07T10:30:03.226098737Z level=INFO service=order-service product=kafka lab=consumer-group messageId=01JMQ000000000000000000002 eventType=order.created schemaVersion=1 aggregateId=order-1002 traceId=00000000000000000000000000000002 correlationId=order-1002 destination=orders.group partitionOrQueue=0 offset=1 seq=2 consumerGroup=orders-group-a consumer=a-2 attempt=1 redelivered=false status=received
[replay-gate] timestamp=2026-10-07T10:30:03.226586934Z checkpoint=after-delivery messageId=01JMQ000000000000000000002 token=08125c9aaffa-1-after-delivery-01JMQ000000000000000000002-3 status=waiting
[replay-gate] timestamp=2026-10-07T10:30:16.151521627Z checkpoint=after-delivery messageId=01JMQ000000000000000000002 token=08125c9aaffa-1-after-delivery-01JMQ000000000000000000002-3 status=released
[consumer] timestamp=2026-10-07T10:30:16.153586745Z level=INFO service=order-service product=kafka lab=consumer-group messageId=01JMQ000000000000000000002 eventType=order.created schemaVersion=1 aggregateId=order-1002 traceId=00000000000000000000000000000002 correlationId=order-1002 destination=orders.group partitionOrQueue=0 seq=2 consumerGroup=orders-group-a attempt=1 status=business_committed
[replay-gate] timestamp=2026-10-07T10:30:16.154105799Z checkpoint=before-offset-commit messageId=01JMQ000000000000000000002 token=08125c9aaffa-1-before-offset-commit-01JMQ000000000000000000002-4 status=waiting
[replay-gate] timestamp=2026-10-07T10:30:28.977137838Z checkpoint=before-offset-commit messageId=01JMQ000000000000000000002 token=08125c9aaffa-1-before-offset-commit-01JMQ000000000000000000002-4 status=released
[consumer] timestamp=2026-10-07T10:30:28.984710607Z level=INFO service=order-service product=kafka lab=consumer-group messageId=01JMQ000000000000000000003 eventType=order.created schemaVersion=1 aggregateId=order-1003 traceId=00000000000000000000000000000003 correlationId=order-1003 destination=orders.group partitionOrQueue=1 offset=0 seq=3 consumerGroup=orders-group-a consumer=a-2 attempt=1 redelivered=false status=received
[replay-gate] timestamp=2026-10-07T10:30:28.985153148Z checkpoint=after-delivery messageId=01JMQ000000000000000000003 token=08125c9aaffa-1-after-delivery-01JMQ000000000000000000003-5 status=waiting
[replay-gate] timestamp=2026-10-07T10:30:41.915484717Z checkpoint=after-delivery messageId=01JMQ000000000000000000003 token=08125c9aaffa-1-after-delivery-01JMQ000000000000000000003-5 status=released
[consumer] timestamp=2026-10-07T10:30:41.917378121Z level=INFO service=order-service product=kafka lab=consumer-group messageId=01JMQ000000000000000000003 eventType=order.created schemaVersion=1 aggregateId=order-1003 traceId=00000000000000000000000000000003 correlationId=order-1003 destination=orders.group partitionOrQueue=1 seq=3 consumerGroup=orders-group-a attempt=1 status=business_committed
[replay-gate] timestamp=2026-10-07T10:30:41.917830Z checkpoint=before-offset-commit messageId=01JMQ000000000000000000003 token=08125c9aaffa-1-before-offset-commit-01JMQ000000000000000000003-6 status=waiting
[replay-gate] timestamp=2026-10-07T10:30:54.459865179Z checkpoint=before-offset-commit messageId=01JMQ000000000000000000003 token=08125c9aaffa-1-before-offset-commit-01JMQ000000000000000000003-6 status=released
consumer-b-1.out.txt
SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".
SLF4J: Defaulting to no-operation (NOP) logger implementation
SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for further details.
[consumer] timestamp=2026-10-07T10:29:37.229317902Z level=INFO service=order-service product=kafka lab=consumer-group destination=orders.group consumerGroup=orders-group-b consumer=b-1 partitions=0,1,2 status=assigned
[consumer] timestamp=2026-10-07T10:29:37.431375769Z level=INFO service=order-service product=kafka lab=consumer-group messageId=01JMQ000000000000000000001 eventType=order.created schemaVersion=1 aggregateId=order-1001 traceId=00000000000000000000000000000001 correlationId=order-1001 destination=orders.group partitionOrQueue=0 offset=0 seq=1 consumerGroup=orders-group-b consumer=b-1 attempt=1 redelivered=false status=received
[replay-gate] timestamp=2026-10-07T10:29:37.440035947Z checkpoint=after-delivery messageId=01JMQ000000000000000000001 token=50c5d15083d6-1-after-delivery-01JMQ000000000000000000001-1 status=waiting
[replay-gate] timestamp=2026-10-07T10:29:44.255577739Z checkpoint=after-delivery messageId=01JMQ000000000000000000001 token=50c5d15083d6-1-after-delivery-01JMQ000000000000000000001-1 status=released
[consumer] timestamp=2026-10-07T10:29:44.267025629Z level=INFO service=order-service product=kafka lab=consumer-group messageId=01JMQ000000000000000000001 eventType=order.created schemaVersion=1 aggregateId=order-1001 traceId=00000000000000000000000000000001 correlationId=order-1001 destination=orders.group partitionOrQueue=0 seq=1 consumerGroup=orders-group-b attempt=1 status=business_committed
[replay-gate] timestamp=2026-10-07T10:29:44.267418787Z checkpoint=before-offset-commit messageId=01JMQ000000000000000000001 token=50c5d15083d6-1-before-offset-commit-01JMQ000000000000000000001-2 status=waiting
[replay-gate] timestamp=2026-10-07T10:29:57.099947819Z checkpoint=before-offset-commit messageId=01JMQ000000000000000000001 token=50c5d15083d6-1-before-offset-commit-01JMQ000000000000000000001-2 status=released
[consumer] timestamp=2026-10-07T10:29:57.137615836Z level=INFO service=order-service product=kafka lab=consumer-group messageId=01JMQ000000000000000000002 eventType=order.created schemaVersion=1 aggregateId=order-1002 traceId=00000000000000000000000000000002 correlationId=order-1002 destination=orders.group partitionOrQueue=0 offset=1 seq=2 consumerGroup=orders-group-b consumer=b-1 attempt=1 redelivered=false status=received
[replay-gate] timestamp=2026-10-07T10:29:57.138890108Z checkpoint=after-delivery messageId=01JMQ000000000000000000002 token=50c5d15083d6-1-after-delivery-01JMQ000000000000000000002-3 status=waiting
[replay-gate] timestamp=2026-10-07T10:30:09.760373196Z checkpoint=after-delivery messageId=01JMQ000000000000000000002 token=50c5d15083d6-1-after-delivery-01JMQ000000000000000000002-3 status=released
[consumer] timestamp=2026-10-07T10:30:09.762171404Z level=INFO service=order-service product=kafka lab=consumer-group messageId=01JMQ000000000000000000002 eventType=order.created schemaVersion=1 aggregateId=order-1002 traceId=00000000000000000000000000000002 correlationId=order-1002 destination=orders.group partitionOrQueue=0 seq=2 consumerGroup=orders-group-b attempt=1 status=business_committed
[replay-gate] timestamp=2026-10-07T10:30:09.762574782Z checkpoint=before-offset-commit messageId=01JMQ000000000000000000002 token=50c5d15083d6-1-before-offset-commit-01JMQ000000000000000000002-4 status=waiting
[replay-gate] timestamp=2026-10-07T10:30:22.484078891Z checkpoint=before-offset-commit messageId=01JMQ000000000000000000002 token=50c5d15083d6-1-before-offset-commit-01JMQ000000000000000000002-4 status=released
[consumer] timestamp=2026-10-07T10:30:22.491356362Z level=INFO service=order-service product=kafka lab=consumer-group messageId=01JMQ000000000000000000003 eventType=order.created schemaVersion=1 aggregateId=order-1003 traceId=00000000000000000000000000000003 correlationId=order-1003 destination=orders.group partitionOrQueue=1 offset=0 seq=3 consumerGroup=orders-group-b consumer=b-1 attempt=1 redelivered=false status=received
[replay-gate] timestamp=2026-10-07T10:30:22.491792961Z checkpoint=after-delivery messageId=01JMQ000000000000000000003 token=50c5d15083d6-1-after-delivery-01JMQ000000000000000000003-5 status=waiting
[replay-gate] timestamp=2026-10-07T10:30:35.313686356Z checkpoint=after-delivery messageId=01JMQ000000000000000000003 token=50c5d15083d6-1-after-delivery-01JMQ000000000000000000003-5 status=released
[consumer] timestamp=2026-10-07T10:30:35.315675776Z level=INFO service=order-service product=kafka lab=consumer-group messageId=01JMQ000000000000000000003 eventType=order.created schemaVersion=1 aggregateId=order-1003 traceId=00000000000000000000000000000003 correlationId=order-1003 destination=orders.group partitionOrQueue=1 seq=3 consumerGroup=orders-group-b attempt=1 status=business_committed
[replay-gate] timestamp=2026-10-07T10:30:35.316119038Z checkpoint=before-offset-commit messageId=01JMQ000000000000000000003 token=50c5d15083d6-1-before-offset-commit-01JMQ000000000000000000003-6 status=waiting
[replay-gate] timestamp=2026-10-07T10:30:47.937490673Z checkpoint=before-offset-commit messageId=01JMQ000000000000000000003 token=50c5d15083d6-1-before-offset-commit-01JMQ000000000000000000003-6 status=released
[consumer] timestamp=2026-10-07T10:30:47.941318921Z level=INFO service=order-service product=kafka lab=consumer-group destination=orders.group consumer=b-1 received=3 status=done
inspect-db-b.out.txt
SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".
SLF4J: Defaulting to no-operation (NOP) logger implementation
SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for further details.
[inspect] timestamp=2026-10-07T10:30:48.674563007Z level=INFO service=order-service product=kafka lab=consumer-group business_rows=3 processed_rows=3 status=snapshot
producer.out.txt
SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".
SLF4J: Defaulting to no-operation (NOP) logger implementation
SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for further details.
[producer] timestamp=2026-10-07T10:29:31.695879200Z level=INFO service=order-service product=kafka lab=consumer-group messageId=01JMQ000000000000000000001 eventType=order.created schemaVersion=1 aggregateId=order-1001 traceId=00000000000000000000000000000001 correlationId=order-1001 destination=orders.group partitionOrQueue=0 offset=0 seq=1 durationMs=269 status=produced
[producer] timestamp=2026-10-07T10:29:31.707896759Z level=INFO service=order-service product=kafka lab=consumer-group messageId=01JMQ000000000000000000002 eventType=order.created schemaVersion=1 aggregateId=order-1002 traceId=00000000000000000000000000000002 correlationId=order-1002 destination=orders.group partitionOrQueue=0 offset=1 seq=2 durationMs=9 status=produced
[producer] timestamp=2026-10-07T10:29:31.716498541Z level=INFO service=order-service product=kafka lab=consumer-group messageId=01JMQ000000000000000000003 eventType=order.created schemaVersion=1 aggregateId=order-1003 traceId=00000000000000000000000000000003 correlationId=order-1003 destination=orders.group partitionOrQueue=1 offset=0 seq=3 durationMs=8 status=produced
[producer] timestamp=2026-10-07T10:29:31.716635659Z level=INFO service=order-service product=kafka lab=consumer-group destination=orders.group produced=3 status=done
setup.out.txt
SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".
SLF4J: Defaulting to no-operation (NOP) logger implementation
SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for further details.

Offset 回放 ​

ordering-replay 同时验证组级位点：消费组 g1 首次顺序读取 6 条消息后，新消费组 g2 使用 auto.offset.reset=earliest 从 offset 0 全量读取。g1 的消费不会改变 g2 的起点，因为日志保留与消费进度彼此独立。

核心断言为：6 条消息落在同一分区、首次观察顺序为 1 到 6、回放仍读取 6 条且从 offset 0 开始。

在 Kafka 实验台查看首次消费与回放轨道。

分区数怎么选 ​

  • 分区数 ≈ 目标消费并行度；扩容消费者只能到分区数为止，扩分区不可逆（且会打乱既有 key 分布）。
  • 分区过多会增加元数据、文件句柄与端到端延迟（Producer 按分区分批）。
  • 顺序需求决定 key 的选择（orderId 而非 userId 还是二者组合），热点 key 会造成单分区倾斜。

常见误区 ​

  • 「key 相同就一定相邻消费」——只保证同分区顺序；消费端多线程拆分处理同一分区会破坏顺序。
  • 「增加消费者总能提速」——超过分区数后新消费者拿不到分区。
  • 「再均衡是无感的」——cooperative 协议减少了停顿，但 rebalance 仍意味着短暂的消费中断与位点重读风险（提交太早会重复）。
  • 「无 key 消息完全随机分布」——Sticky 分区器有批内粘连语义，不是纯随机轮转。

官方资料 ​

以统一实验验证消息系统语义边界

MIT License