Creating AnyCost Custom Adaptors

What is an Adaptor?

An adaptor is a piece of code that allows CloudZero to process cost data from any provider. You must write the Adaptor code yourself, and you can use any programming language. The Adaptor should take the following actions:

• Automatically retrieve cost data from any provider.
• Translate the cost data to Common Bill Format (CBF) for CloudZero ingestion.
• Transfer the cost data to CloudZero:
  • AnyCost Bucket Adaptors upload a gzipped CSV file of CBF data to an Amazon S3 bucket.
  • AnyCost Stream Adaptors send a JSON body of CBF data to the CloudZero AnyCost API.

After you write your Adaptor, you can create the AnyCost connection in the CloudZero UI, and CloudZero ingests the cost data. See the following documentation for more details:

• Connect an AnyCost Bucket Adaptor
• Connect an AnyCost Stream Adaptor

Guidelines for Writing Code for an AnyCost Adaptor

AnyCost adaptors give you the freedom of creating your own code to format data from any provider. This data can then be ingested into the CloudZero platform for analysis and reporting. The development of these adaptors is largely up to you, but there are some very specific guidelines you must and should follow to ensure a smooth setup with the CloudZero infrastructure.

First, you must coordinate a way to receive cost data from your vendor - whether it's through APIs or generated exports.

Second, you must write the code that will translate that data into the CloudZero Common Bill Format (CBF). CBF is our standard file format for cost data ingestion, and it is the required format for successful ingestion of cost data into the CloudZero platform.

Adaptor Requirements

When writing an Adaptor, you must do the following:

• Convert provider cost data into the CloudZero Common Bill Format.
• Preserve information whenever possible.
  • Example: if a provider gives you hourly data, don’t aggregate that data up to daily
  • Example: if a provider gives you precise information about a charge, attempt to pass all of that information along into the Common Bill Format “CBF”
• Process data by month
  • each report update in a given month is cumulative
  • adaptors must include all billing data for a given month to date
• Ensure that cost/cost and cost/discounted_cost sum to the same total
• (AnyCost Bucket Adaptors only): Write cost data files to a folder in the S3 bucket. Do not upload the files into the root of the bucket.

Adaptor Recommendations

When writing an Adaptor, we recommend that you do the following:

• Amortize to hourly
  • data is viewable in the platform according to the lowest amortization supplied
  • data supplied hourly can be aggregated to hourly and up (e.g. monthly)
  • Note: this rule can be skipped for non-usage based line items (e.g. Taxes)
• Include taxes
• Handle Discounts and Committed Use
• Allow ingesting back data
• Minimize the frequency of drops without clear business value, there is a cost to processing new data drops; don’t drop data more often than the provider is giving significant new information
• Work with cost data using appropriate data types in your programming language (e.g. decimal.Decimal instead of float in Python)
• Don't populate the bill/invoice_id field until the provider has “closed” the invoice for the month that is being processed.

FAQs

How do you DELETE a record or records?

CloudZero loads data cumulatively, so each data drop for any given billing period replaces what was previously in the CloudZero platform.

To delete or remove data from CloudZero for a given billing period, simply send us a new data drop for that period that contains all records, minus the ones you'd like to remove.

How do you DELETE an entire billing period?

To delete data for an entire billing period, send CloudZero an empty data drop, and because it replaces what is currently in the system, your data for that period will be removed.

Note: An empty data drop is one that contains a manifest file with the appropriate billing data information, and that points to a CSV with headers and no rows.

How do you UPDATE existing data?

To update data already in our system, simply send the edited data along with the rest of the data for that billing period. CloudZero replaces whatever is currently in the system with the data that is sent.

CloudZero loads cumulatively, so what you send for any particular billing period will entirely replace what is already there. Therefore, ensure the data drop for the rows you want to edit contains all the data for that billing period to avoid losing data.