Skip to main content

​
Durability

Every write to S2 is durable on object storage before it is acknowledged. This applies to both the managed service and s2-lite. Appends execute atomically — either all the records in a batch will become durable, or none. depends on the storage class for the stream:
  • Standard — writes are durable in multiple availability zones. Ack latency within 400ms.
  • Express — writes are durable across a quorum of three zonal buckets. Ack latency within 40ms.

​
Ordering

Records within streams are totally ordered. Every record gets a monotonically increasing sequence number. With an AppendSession (available via SDKs), you can pipeline batches with an ordering guarantee, and receive acknowledgements back in the corresponding order. If any batch fails, subsequent batches will not become durable.

​
Consistency

Appends, reads, and check-tail are all linearizable. This means that if an append has been acknowledged, any subsequent read or check-tail must reflect that write. We wrote about how we test for this property using the Porcupine model checker in our blog.

​
Correctness

We are huge believers in deterministic simulation testing. S2 involves many moving parts, and investing in DST setups allows us to be confident about invariants that the system needs to always uphold. In short, deterministic testing allows us to run the different processes that comprise S2 over a simulated network. We can inject faults, crash processes randomly, trigger recovery procedures, and make sure that we never violate any of our commitments to durability and strong consistency. Our setup involves both a custom Rust framework that makes use of Turmoil, which we can run both as part of CI/CD, and in much larger “nightly” searches for subtle or rare regressions. We additionally use the Antithesis platform as a second form of DST on S2.