Streams are always durable and totally ordered.
Every stream has a URI: s2://{basin}/{stream}. Basins can hold unlimited streams.
Appends always land at the tail, atomically – either all the records in a batch or none.
Reads can start from any position and optionally follow for live updates.
Positions
We talk about the head and tail of a stream.
The head is the beginning, the tail is the current end — where the next record to be appended will go.
Each record on a stream has an associated sequence number and timestamp.
The tail position is (next_seq_num, last_record_timestamp).Both will be 0 for an empty stream!
Lifecycle automation
Creation
Streams can be created explicitly, or automatically on first use.
If a basin has create_stream_on_append or create_stream_on_read configured, streams will be created with the basin’s default config the first time they’re accessed. This is very useful for patterns with many dynamic streams, like one per session or job.
Deletion
Streams can be configured to automatically delete themselves once it becomes empty. The min_age_secs threshold can be set to require the stream to have been empty for a minimum duration before auto-deletion kicks in.
This is useful for ephemeral streams (e.g. one per job run) where the data is consumed and no longer needed.
See also
