Step 2: How to Craft Telemetry Records and Send to our API

Now we will prepare our telemetry records for transmission to CloudZero. In this simple walk through, we'll capture the number of write operations to RDS performed by each product, Email and Messaging. We'll assume a simple system has been implemented in AWS to capture this metric and aggregate it daily. Once aggregated we will now form it into the JSON format for the API and send it to the CloudZero Allocation Telemetry API.

Telemetry Data Schema

Telemetry is sent to the telemetry API as records which specify:

• telemetry-stream - the name of the stream. We’ll use a different stream for each metric we’re measuring.
• element-name - the tenant associated with this usage. These values will become the elements of the dimension we create later.
• filter - this identifies the part of the cloud infrastructure being used. We’ll use the “Shared” elements of our Shared RDS dimension for this.
• value - the “amount” of activity, in this case the number of write operations performed. A good metric is a primary driver of the cost of the targeted resource.

In this walk through, we're looking to split RDS spend into our two products Email and Messaging. With the CostFormation we wrote in the previous step, we know that the costs we need to split the Shared Data Lake element of the Shared RDS dimension. The JSON payload and records we need would be formatted like so:

{
  "records": [
    {
      "timestamp": "2020-01-25T00:00:00Z",
      "granularity": "DAILY",
      "filter": { 
        "custom:Shared RDS": ["Shared Data Lake"] 
      },
      "element-name": "Email",
      "telemetry-stream": "rds-writes-by-product",
      "value": 100045
    },
    {
      "timestamp": "2020-01-25T00:00:00Z",
      "granularity": "DAILY",
      "filter": {
        "custom:Shared RDS": ["Shared Data Lake"]
      },
      "element-name": "Messaging",
      "telemetry-stream": "rds-writes-by-product",
      "value": 1000600
    }
  ]
}

Sending Telemetry Data to CloudZero

Our API documentation can be found here: Telemetry API Documentation. Follow those instructions to authenticate against the API and transmit your data.

Once the Telemetry data starts flowing and the stream's Ingest Status on CloudZero's Telemetry page is Available, then we will be able to reference this stream in a new Allocation dimension and can proceed to the final step. This can take up to 2 hours, but typically completes in about an hour.

Link to Previous Step

• Step 1: What Cost Should be Split?