Skip to main content
Clients interact with S2 at its public https endpoints, primarily with JSON bodies conforming to its OpenAPI spec.
Prefer to use an available SDK.

​
Authentication

The API requires a Bearer access token to authenticate requests. S2 SDKs take care of supplying it automatically. If you are using curl, you can provide it with -H "Authorization: Bearer ${S2_ACCESS_TOKEN}".
The first usage of an access token on a connection may experience a latency overhead for verification, typically tens of milliseconds. Subsequent requests will not incur such an overhead.

​
Compression

To optimize network usage, S2 supports compressed request and response bodies with the following algorithms:
  • zstd preferred
  • gzip
Standard HTTP semantics around Content-Encoding and Accept-Encoding apply.

​
Data Plane

There are 3 core data operations on a stream:

Append records

POST /streams/{stream}/records

Read records

GET /streams/{stream}/records?seq_num=42&count=100

Check the tail

GET /streams/{stream}/records/tail

​
Data in JSON

The following pieces of record data are stored as bytes:
  • Header name
  • Header value
  • Body
The header s2-format is used to indicate the desired encoding of these bytes when records are represented in JSON.

raw

s2-format: raw or omit header.Use when your record data is valid Unicode. Zero overhead, human-readable. Cannot handle binary data safely.

base64

s2-format: base64Use when you are working with arbitrary bytes. Always safe. 33% overhead over the wire.
You can write using one format and read with another. When reading raw, S2 is interpreting the stored bytes as UTF-8. This will be a potentially-lossy conversion if it was not also written as raw, or as base64-encoded valid UTF-8.

​
Protobuf messages

Data plane endpoints to append and read records also support protobuf bodies. This helps avoid the base64 encoding tax compared to binary data in JSON messages. To send and receive protobuf:
  • Set the Content-Type header to application/protobuf and send a protobuf-encoded payload.
  • Set the Accept header to application/protobuf to receive a protobuf-encoded response. The response will include the Content-Type: application/protobuf header if the server returns a protobuf.
Type definitions are available in git and Buf.

Append

POST ../{stream}/records AppendInput AppendAck

Read

GET ../{stream}/records?.. ReadBatch
Sending Accept: application/protobuf request header only guarantees a protobuf response in case of a success (HTTP 200). Other status codes are always accompanied by JSON bodies.

​
Sessions

S2S (S2-Session) is a minimal binary protocol used by S2 SDKs to encapsulate streaming append and read session semantics over , at the same endpoints as “unary” calls. It is meant for S2 SDKs, but documented for the adventurous.

​
Setup

  • Content-Type: s2s/proto signals that a session is being requested.
  • Accept-Encoding signals which compression algorithms are supported (service supports zstd and gzip). Content-Encoding is not sent as message-level compression is used.
  • 200 OK response establishes a session.

​
Message framing

All integers use big-endian byte order. Messages smaller than 1KiB should not be compressed.Length prefix (3 bytes): Total message length (flag + body)Flag byte (1 byte): [T][CC][A][RRRR]
  • T (bit 7): Terminal flag (1 = stream ends after this message)
  • CC (bits 6-5): Compression (00=none, 01=zstd, 10=gzip)
  • A (bit 4): Reconnect advised (1 = server suggests reconnecting, only on messages from Server → Client)
  • RRRR (bits 3-0): Reserved
Body (variable):
  • Regular message is a Protobuf
  • Terminal message contains a 2-byte status code, followed by JSON error information (corresponding to unary response behavior)

​
Data flow

Append sessions are a bi-directional stream of AppendInput messages from Client → Server, and AppendAck messages from Server → Client.Read sessions are a uni-directional stream of ReadBatch messages from Server → Client. When waiting for new records, an empty batch is sent as a heartbeat at least every 15 seconds.

​
Reconnect advice

A server that is about to terminate sets the reconnect-advised flag on regular messages.
  • Read sessions should be re-established with a fresh request.
  • Append sessions should stop sending inputs and half-close the request stream. The server acknowledges all accepted inputs and then ends the session cleanly. A new session can be established concurrently, and should start being used once all acknowledgements for in-flight appends have been received.
  • Append sessions still attached when the server drains are ended by the server: it stops reading inputs, acknowledges accepted inputs, and sends a terminal 503 with error code server_draining. Acknowledgements always precede the terminal message, so an input is either acknowledged or was never processed, and unresolved inputs can be safely resubmitted on a new session.
Read sessions are also supported over SSE which has the benefit of being usable in the browser context.