Skip to main content
We can increase certain limits if feedback suggests they are too restrictive.

​
Basins

  • Basin names must be globally unique. The name must be between 8 and 48 characters, comprising lowercase letters, numbers and hyphens. It cannot begin or end with a hyphen.
  • Each account can create up to 100 basins.
    We plan to make the number of basins unlimited in the coming weeks. This will allow you to model, as an example, basin-per-tenant.

​
Streams

  • Streams are unlimited in number.
  • Stream names can be an arbitrary string upto 512 characters. They may be hierarchically named (e.g. using a delimiter like /) for namespacing — you are able to list streams with prefix filtering.
    Many use cases around lightweight streams find it useful to configure create stream on append / create stream on read on the basin, and set a delete on empty time threshold for the default stream configuration.

​
Records

  • Each record may be up to 1 MiB in metered bytes.
  • Batches may contain up to 1000 records, and cumulatively total up to 1 MiB in metered bytes.
S2 uses the following formula for metering record size: 
metered_bytes = lambda record: 8 + 2 * len(record.headers) \
  + sum((len(name) + len(value)) for (name, value) in record.headers) \
  + len(record.body)

​
Appends

  • 100 MiBps per stream.
  • 200 batches per second per stream per client connection. When exceeded,
    • Append requests will receive an HTTP 429.
    • AppendSession will experience throttling.

​
Reads

Reads are highly scalable on both throughput and concurrency, due to S2’s decoupled architecture and read-through caching.
The system is designed to be elastic, but there are bounds to how fast it can scale. The precise limit that will be enforced is TBD.

​
Retention

Records may stay visible for a brief period after their retention window expires, typically on the order of minutes.
On the free tier, streams can retain data for 28 days or less. If you want a longer (or infinite) retention, add a payment method in the Dashboard.

​
Reconfiguration

Storage class changes on an existing stream (i.e. from Standard to Express, or vice versa) only take effect when a stream has been inactive for up to 10 minutes. This will become a live migration in future.