gRPC API

​

AccountService

Operate on an S2 account.

Method Name	Request Type	Response Type	Description	Options
ListBasins	ListBasinsRequest	ListBasinsResponse	List basins.	idempotency_level: NO_SIDE_EFFECTS
CreateBasin	CreateBasinRequest	CreateBasinResponse	Create a new basin. Provide a client request token with the S2-Request-Token header for idempotent retry behaviour.	idempotency_level: IDEMPOTENT
DeleteBasin	DeleteBasinRequest	DeleteBasinResponse	Delete a basin. Basin deletion is asynchronous, and may take a few minutes to complete.	idempotency_level: IDEMPOTENT
ReconfigureBasin	ReconfigureBasinRequest	ReconfigureBasinResponse	Update basin configuration.	idempotency_level: IDEMPOTENT
GetBasinConfig	GetBasinConfigRequest	GetBasinConfigResponse	Get basin configuration.	idempotency_level: NO_SIDE_EFFECTS
IssueAccessToken	IssueAccessTokenRequest	IssueAccessTokenResponse	Issue a new access token.	None
RevokeAccessToken	RevokeAccessTokenRequest	RevokeAccessTokenResponse	Revoke an access token.	idempotency_level: IDEMPOTENT
ListAccessTokens	ListAccessTokensRequest	ListAccessTokensResponse	List access tokens.	idempotency_level: NO_SIDE_EFFECTS

​

BasinService

Operate on an S2 basin.

Method Name	Request Type	Response Type	Description	Options
ListStreams	ListStreamsRequest	ListStreamsResponse	List streams.	idempotency_level: NO_SIDE_EFFECTS
CreateStream	CreateStreamRequest	CreateStreamResponse	Create a stream. Provide a client request token with the S2-Request-Token header for idempotent retry behaviour.	idempotency_level: IDEMPOTENT
DeleteStream	DeleteStreamRequest	DeleteStreamResponse	Delete a stream. Stream deletion is asynchronous, and may take a few minutes to complete.	idempotency_level: IDEMPOTENT
GetStreamConfig	GetStreamConfigRequest	GetStreamConfigResponse	Get stream configuration.	idempotency_level: NO_SIDE_EFFECTS
ReconfigureStream	ReconfigureStreamRequest	ReconfigureStreamResponse	Update stream configuration.	idempotency_level: IDEMPOTENT

​

StreamService

Operate on an S2 stream.

Method Name	Request Type	Response Type	Description	Options
CheckTail	CheckTailRequest	CheckTailResponse	Check the sequence number that will be assigned to the next record on a stream.	idempotency_level: NO_SIDE_EFFECTS
Append	AppendRequest	AppendResponse	Append a batch of records to a stream.	None
AppendSession	AppendSessionRequest stream	AppendSessionResponse stream	Append batches of records to a stream continuously, while guaranteeing pipelined requests are processed in order. If any request fails, the session is terminated.	None
Read	ReadRequest	ReadResponse	Retrieve a batch of records from a stream.	idempotency_level: NO_SIDE_EFFECTS
ReadSession	ReadSessionRequest	ReadSessionResponse stream	Retrieve batches of records from a stream continuously.	idempotency_level: NO_SIDE_EFFECTS

​

AccessTokenInfo

Access token information.

Field	Type	Label	Description
id	string		Access token ID. It must be unique to the account and between 1 and 50 characters.
expires_at	uint32	optional	Expiration time in seconds since Unix epoch. If not set, the expiration will be set to that of the requestor’s token.
auto_prefix_streams	bool		Namespace streams based on the configured stream-level scope, which must be a prefix. Stream name arguments will be automatically prefixed, and the prefix will be stripped when listing streams.
scope	AccessTokenScope		Access token scope.

​

AccessTokenScope

Access token scope.

Field	Type	Label	Description
basins	ResourceSet		Basin names allowed.
streams	ResourceSet		Stream names allowed.
tokens	ResourceSet		Token IDs allowed.
op_groups	PermittedOperationGroups		Access permissions at operation group level.
ops	Operation	repeated	Operations allowed for the token. A union of allowed operations and groups is used as an effective set of allowed operations.

​

AppendInput

Input for append requests.

Field	Type	Label	Description
stream	string		Stream name. Optional for subsequent requests in the session.
records	AppendRecord	repeated	Batch of records to append atomically, which must contain at least one record, and no more than 1000. The total size of a batch of records may not exceed 1MiB of metered bytes.
match_seq_num	uint64	optional	Enforce that the sequence number issued to the first record matches.
fencing_token	bytes	optional	Enforce a fencing token which must have been previously set by a fence command record.

​

AppendOutput

Output from append response.

Field	Type	Label	Description
start_seq_num	uint64		Sequence number of first record appended.
end_seq_num	uint64		Sequence number of last record appended + 1. end_seq_num - start_seq_num will be the number of records in the batch.
next_seq_num	uint64		Sequence number of last durable record on the stream + 1. This can be greater than end_seq_num in case of concurrent appends.

​

AppendRecord

Record to be appended to a stream.

Field	Type	Label	Description
timestamp	uint64	optional	Timestamp for this record in milliseconds since Unix epoch. The service ensures monotonicity by adjusting it up if necessary to the maximum observed timestamp. A timestamp detected to be in the future will be adjusted down. If not provided, the semantics depend on the stream’s require_client_timestamps config.
headers	Header	repeated	Series of name-value pairs for this record.
body	bytes		Body of this record.

​

AppendRequest

Append request.

Field	Type	Label	Description
input	AppendInput		Request parameters for an append.

​

AppendResponse

Append response.

Field	Type	Label	Description
output	AppendOutput		Response details for an append.

​

AppendSessionRequest

Append session request.

Field	Type	Label	Description
input	AppendInput		Request parameters for an append.

​

AppendSessionResponse

Append session response.

Field	Type	Label	Description
output	AppendOutput		Response details for an append.

​

BasinConfig

Basin configuration.

Field	Type	Label	Description
default_stream_config	StreamConfig		Default stream configuration.
create_stream_on_append	bool		Create stream on append if it doesn’t exist, using the default stream configuration.
create_stream_on_read	bool		Create stream on read if it doesn’t exist, using the default stream configuration.

​

BasinInfo

Basin information.

Field	Type	Label	Description
name	string		Basin name.
scope	BasinScope		Basin scope.
state	BasinState		Basin state.

​

CheckTailRequest

Check tail request.

Field	Type	Label	Description
stream	string		Stream name.

​

CheckTailResponse

Check tail response.

Field	Type	Label	Description
next_seq_num	uint64		Sequence number that will be assigned to the next record on the stream.

​

CreateBasinRequest

Create basin request.

Field	Type	Label	Description
basin	string		Basin name, which must be globally unique. It can be omitted to let the service assign a unique name. The name must be between 8 and 48 characters, comprising lowercase letters, numbers and hyphens. It cannot begin or end with a hyphen.
config	BasinConfig		Basin configuration.
scope	BasinScope		Basin scope.

​

CreateBasinResponse

Create basin response.

Field	Type	Label	Description
info	BasinInfo		Information about the newly created basin.

​

CreateStreamRequest

Create stream request.

Field	Type	Label	Description
stream	string		Stream name, which must be unique within the basin. It can be an arbitrary string upto 512 characters. Backslash (/) is recommended as a delimiter for hierarchical naming.
config	StreamConfig		Configuration for the new stream.

​

CreateStreamResponse

Create stream response.

Field	Type	Label	Description
info	StreamInfo		Information about the newly created stream.

​

DeleteBasinRequest

Delete basin request.

Field	Type	Label	Description
basin	string		Name of the basin to delete.

​

DeleteBasinResponse

Delete basin response.

​

DeleteStreamRequest

Delete stream request.

Field	Type	Label	Description
stream	string		Stream name.

​

DeleteStreamResponse

Delete stream response.

​

GetBasinConfigRequest

Get basin configuration request.

Field	Type	Label	Description
basin	string		Basin name.

​

GetBasinConfigResponse

Get basin configuration response.

Field	Type	Label	Description
config	BasinConfig		Basin configuration.

​

GetStreamConfigRequest

Get stream configuration request.

Field	Type	Label	Description
stream	string		Stream name.

​

GetStreamConfigResponse

Get stream configuration response.

Field	Type	Label	Description
config	StreamConfig		Stream configuration.

​

Header

Headers add structured information to a record as name-value pairs.

Field	Type	Label	Description
name	bytes		Header name blob. The name cannot be empty, with the exception of an S2 command record.
value	bytes		Header value blob.

​

IssueAccessTokenRequest

Issue access token request.

Field	Type	Label	Description
info	AccessTokenInfo		Access token information.

​

IssueAccessTokenResponse

Issue access token response.

Field	Type	Label	Description
token	string		Created token.

​

ListAccessTokensRequest

List access tokens request.

Field	Type	Label	Description
prefix	string		List access tokens that begin with this prefix.
start_after	string		Only return access tokens that lexicographically start after this token ID.
limit	uint64	optional	Number of results, up to a maximum of 1000.

​

ListAccessTokensResponse

List access tokens response.

Field	Type	Label	Description
tokens	AccessTokenInfo	repeated	Access tokens information.
has_more	bool		If set, indicates there are more results that can be listed with start_after.

​

ListBasinsRequest

List basins request.

Field	Type	Label	Description
prefix	string		List basin names that begin with this prefix.
start_after	string		Only return basins names that lexicographically start after this name. This can be the last basin name seen in a previous listing, to continue from there. It must be greater than or equal to the prefix if specified.
limit	uint64	optional	Number of results, up to a maximum of 1000.

​

ListBasinsResponse

List basins response.

Field	Type	Label	Description
basins	BasinInfo	repeated	Matching basins.
has_more	bool		If set, indicates there are more results that can be listed with start_after.

​

ListStreamsRequest

List streams request.

Field	Type	Label	Description
prefix	string		List stream names that begin with this prefix.
start_after	string		Only return stream names that lexicographically start after this name. This can be the last stream name seen in a previous listing, to continue from there. It must be greater than or equal to the prefix if specified.
limit	uint64	optional	Number of results, up to a maximum of 1000.

​

ListStreamsResponse

List streams response.

Field	Type	Label	Description
streams	StreamInfo	repeated	Matching streams.
has_more	bool		If set, indicates there are more results that can be listed with start_after.

​

PermittedOperationGroups

Access permissions for a group.

Field	Type	Label	Description
account	ReadWritePermissions		Access permissions at account level.
basin	ReadWritePermissions		Access permissions at basin level.
stream	ReadWritePermissions		Access permissions at stream level.

​

ReadLimit

Limit how many records can be retrieved. If both count and bytes are specified, either limit may be hit.

Field	Type	Label	Description
count	uint64	optional	Record count limit.
bytes	uint64	optional	Metered bytes limit.

​

ReadOutput

Output of a read.

Field	Type	Label	Description
batch	SequencedRecordBatch		Batch of records. It can only be empty outside of a session context, if the request cannot be satisfied without violating its limit.
next_seq_num	uint64		Tail of the stream, i.e. sequence number that will be assigned to the next record. It will primarily be returned either because the requested starting point was larger, or only in case of a limited read, equal to the tail. It will also be returned if there are no records on the stream between the requested starting point and the tail.

​

ReadRequest

Read request.

Field	Type	Label	Description
stream	string		Stream name.
start_seq_num	uint64		Starting sequence number (inclusive).
limit	ReadLimit		Limit how many records can be returned. This will get capped at the default limit, which is up to 1000 records or 1MiB of metered bytes.

​

ReadResponse

Read response.

Field	Type	Label	Description
output	ReadOutput		Response details for a read.

​

ReadSessionRequest

Read session request.

Field	Type	Label	Description
stream	string		Stream name.
start_seq_num	uint64		Starting sequence number (inclusive).
limit	ReadLimit		Limit on how many records can be returned. When a limit is specified, the session will be terminated as soon as the limit is met, or when the current tail of the stream is reached — whichever occurs first. If no limit is specified, the session will remain open after catching up to the tail, and continue tailing as new messages are written to the stream.
heartbeats	bool		Heartbeats can be enabled to monitor end-to-end session health. A heartbeat will be sent when the initial switch to real-time tailing happens, as well as when no records are available at a randomized interval between 5 and 15 seconds.

​

ReadSessionResponse

Read session response.

Field	Type	Label	Description
output	ReadOutput	optional	Response details for a read. This will not be set in case of a heartbeat message.

​

ReadWritePermissions

Read/Write permissions.

Field	Type	Label	Description
read	bool		Read permission.
write	bool		Write permission.

​

ReconfigureBasinRequest

Reconfigure basin request.

Field	Type	Label	Description
basin	string		Basin name.
config	BasinConfig		Basin configuration.
mask	google.protobuf.FieldMask		Specifies the pieces of configuration being updated. See https://protobuf.dev/reference/protobuf/google.protobuf/#field-mask

​

ReconfigureBasinResponse

Reconfigure basin response.

Field	Type	Label	Description
config	BasinConfig		Basin configuration.

​

ReconfigureStreamRequest

Reconfigure stream request.

Field	Type	Label	Description
stream	string		Stream name.
config	StreamConfig		Stream configuration with updated values.
mask	google.protobuf.FieldMask		Specifies the pieces of configuration being updated. See https://protobuf.dev/reference/protobuf/google.protobuf/#field-mask

​

ReconfigureStreamResponse

Reconfigure stream response.

Field	Type	Label	Description
config	StreamConfig		Stream configuration.

​

ResourceSet

Set of named resources.

Field	Type	Label	Description
exact	string		Match only the resource with this exact name. Use an empty string to match no resources.
prefix	string		Match all resources that start with this prefix. Use an empty string to match all resource.

​

RevokeAccessTokenRequest

Revoke access token request.

Field	Type	Label	Description
id	string		Token to revoke.

​

RevokeAccessTokenResponse

Revoke access token response.

Field	Type	Label	Description
info	AccessTokenInfo		Access token information.

​

SequencedRecord

Record retrieved from a stream.

Field	Type	Label	Description
seq_num	uint64		Sequence number assigned to this record.
timestamp	uint64		Timestamp for this record in milliseconds since Unix epoch.
headers	Header	repeated	Series of name-value pairs for this record.
body	bytes		Body of this record.

​

SequencedRecordBatch

A batch of sequenced records.

Field	Type	Label	Description
records	SequencedRecord	repeated	Batch of sequenced records.

​

StreamConfig

Stream configuration.

Field	Type	Label	Description
storage_class	StorageClass		Storage class for recent writes. This is the main cost:performance knob in S2.
age	uint64		Age in seconds for automatic trimming of records older than this threshold. If set to 0, the stream will have infinite retention.
require_client_timestamps	bool	optional	Controls how to handle timestamps when they are not provided by the client. If this is false (or not set), the record’s arrival time will be assigned as its timestamp. If this is true, then any append without a client-specified timestamp will be rejected as invalid.

​

StreamInfo

Stream information.

Field	Type	Label	Description
name	string		Stream name.
created_at	uint32		Creation time in seconds since Unix epoch.
deleted_at	uint32	optional	Deletion time in seconds since Unix epoch, if the stream is being deleted.

​

BasinScope

Basin scope.

Name	Number	Description
BASIN_SCOPE_UNSPECIFIED	0	Unspecified basin scope.
BASIN_SCOPE_AWS_US_EAST_1	1	aws us-east-1 region.

​

BasinState

Current state of the basin.

Name	Number	Description
BASIN_STATE_UNSPECIFIED	0	Unspecified.
BASIN_STATE_ACTIVE	1	Basin is active.
BASIN_STATE_CREATING	2	Basin is being created.
BASIN_STATE_DELETING	3	Basin is being deleted.

​

Operation

API operations.

Name	Number	Description
OPERATION_UNSPECIFIED	0	Unspecified operation.
OPERATION_LIST_BASINS	1	List basins.
OPERATION_CREATE_BASIN	2	Create a basin.
OPERATION_DELETE_BASIN	3	Delete a basin.
OPERATION_RECONFIGURE_BASIN	4	Update basin configuration.
OPERATION_GET_BASIN_CONFIG	5	Get basin configuration.
OPERATION_ISSUE_ACCESS_TOKEN	6	Issue an access token.
OPERATION_REVOKE_ACCESS_TOKEN	7	Revoke an access token.
OPERATION_LIST_ACCESS_TOKENS	8	List access tokens.
OPERATION_LIST_STREAMS	9	List streams.
OPERATION_CREATE_STREAM	10	Create a stream.
OPERATION_DELETE_STREAM	11	Delete a stream.
OPERATION_GET_STREAM_CONFIG	12	Get stream configuration.
OPERATION_RECONFIGURE_STREAM	13	Update stream configuration.
OPERATION_CHECK_TAIL	14	Check tail of a stream.
OPERATION_APPEND	15	Append records to a stream.
OPERATION_READ	16	Read records from a stream.
OPERATION_TRIM	17	Trim records up to a sequence number.
OPERATION_FENCE	18	Set a fencing token for a stream.

​

StorageClass

Storage class for recent writes.

Name	Number	Description
STORAGE_CLASS_UNSPECIFIED	0	Unspecified, which is currently overridden to STORAGE_CLASS_EXPRESS.
STORAGE_CLASS_STANDARD	1	Standard, which offers end-to-end latencies under 500 ms.
STORAGE_CLASS_EXPRESS	2	Express, which offers end-to-end latencies under 50 ms.