Metrics

Usage data is available for introspection via the REST API.

​

Metric sets

Metric sets are groupings of metrics, evaluated either at the account, basin, or stream level.

Routes are:

• v1/metrics for account
• v1/metrics/{basin} for basin
• v1/metrics/{basin}/{stream} for stream

For example, the account-level active_basins metric set can be obtained with:

curl \
  --request GET \
  --url "https://aws.s2.dev/v1/metrics?set=active_basins&start=${START}&end=${END}" \
  --header "Authorization: Bearer ${S2_ACCESS_TOKEN}"

​

Common parameters

• start and end timestamps, in Unix epoch seconds
  • The time period specified should not exceed 30 days
  • The end timestamp should not be in the future (though some leeway is given to accommodate clock differences)
• set specifies the requested metric set
• interval represents a duration for use by metric sets that return timeseries
  • for accumulation, this specifies the requested bucket length, or the window of time which the relevant value is an aggregation over
  • for gauge, this specifies the unit to divide the start/end period into, and the timeseries will reflect a momentary reading at each of these divisions of time

​

Account

​

active_basins

Returns a label metric representing all basins which had at least one stream within the specified period.

This can be used to determine which basins to request basin-specific metrics for, when attempting to summarize account-level metrics for a period.

Requires a start and end timestamp.

​

account_ops

Set of accumulation metrics, one per account operation. Each account-level request is a single operation, and this endpoint will return as many timeseries as there are distinct basin operations with non-zero values within the specified period.

Requires a start and end timestamp. Accepts an optional interval parameter for minute / hour / day aggregations.

​

Basin

​

basin_ops

Set of accumulation metrics, one per basin operation. Each basin-level request is a single operation, and this endpoint will return as many timeseries as there are distinct basin operations with non-zero values within the specified period.

Requires a start and end timestamp. Accepts an optional interval parameter for minute / hour / day aggregations.

​

basin_storage

gauge of cumulative stored bytes, per hour, aggregated across all streams within the basin.

Requires a start and end timestamp.

​

append_ops

Returns accumulation of append operations aggregated across all streams within the basin, grouped by storage class.

An append operation is defined as one minute of appends to a single stream, using a single TCP connection.

For instance, 10 unary appends to stream X, each initializing a new connection, would be 10 append operations. 10 unary or streaming session appends to stream X, on a single TCP connection, within a single minute, would be 1 append operation.

Requires a start and end timestamp. Accepts an optional interval parameter for minute / hour / day aggregations.

​

append_bytes

Returns two accumulation metrics of appended throughput in bytes, aggregated across all streams within the basin: one for appends against express storage class streams, and another for appends against standard storage class streams.

Requires a start and end timestamp. Accepts an optional interval parameter for minute / hour / day aggregations.

​

read_ops

Returns two accumulation metrics of read operations, aggregated across all streams within the basin: one for reads from cold storage, and another for reads against hot storage. Reads from cold storage are guaranteed to be of records older than 20 seconds, and therefore will be priced at that rate, whereas hot reads are priced at the tailing rate.

A read operation is defined as one minute of reads from a single stream, using a single TCP connection.

For instance, 10 unary reads from stream X, each initializing a new connection, would be 10 read operations. 10 unary or streaming session reads from stream X, on a single TCP connection, within a single minute, would be 1 read operation.

Requires a start and end timestamp. Accepts an optional interval parameter for minute / hour / day aggregations.

​

read_bytes

accumulation of read throughput in bytes, aggregated across all streams within the basin.

Requires a start and end timestamp. Accepts an optional interval parameter for minute / hour / day aggregations.

​

Stream

​

stream_storage

Returns a gauge representing the cumulative stored bytes, per minute, of the specified stream.

Requires a start and end timestamp.

​

Metric types

​

scalar

Represents a single numeric value. Can represent a summary, or aggregation, if the metric set requires a start and end period, or an instantaneous value if not.

​

accumulation

Timeseries where each (epoch, value) tuple represents an aggregation value for a bucket of time beginning at the epoch, lasting for the specified bucket_length.

Buckets within the time period which have no data will be omitted (unlike with GaugeTimeseries).

​

gauge

Timeseries where each (epoch, value) tuple represents a momentary reading of a value, at the specified epoch instance.

Gauge timeseries will contain entries for every interval within the requested time period.

​

label

A set of string labels representing a set. It can represent a summary or aggregation if the metric set requires a start and end period, or an instantaneous set if not.

​

Access control

Account, basin, and stream level metric access are encoded as operations that can be access controlled. This means you could configure tokens which allow access to read metrics only for a specific basin, a stream name prefix, etc.

# Get storage, at current moment, for a stream.
curl \
  --request GET \
  --url "https://aws.s2.dev/v1/metrics/${BASIN}/${STREAM}?set=storage&start=$(date +%s)&end=$(date +%s)" \
  --header "Authorization: Bearer ${S2_ACCESS_TOKEN}" \
| jq

{
  "values": [
    {
      "gauge": {
        "name": "metered_size",
        "unit": "bytes",
        "values": [
          [
            1748473860,
            589.0
          ]
        ]
      }
    }
  ]
}