Common Bill Format (CBF)

CloudZero has developed a standard data model to ingest cost data from any source, called the Common Bill Format. The CBF is modeled after the AWS CUR format, but simplier and more generic. This allows it to be used by many different cloud providers.

Data File General Guidelines

Provided as drop of gzipped CSVs and JSON manifest file (Common Bill Format (CBF) File Drop Specs)
Max 1,000,000 rows per file
datetime values should be ISO formatted strings in the UTC timezone.

Data File Columns

Category	Column	Type	CZ Dimension or Cost Type	Description	Required
Line Item	lineitem/id	string		Uniquely identifies this specific line item in this specific bill.
Line Item	lineitem/type	string	LineItemType	Broad category of this line item: Usage, Tax, Discount, etc. Values have special meaning (see below). If not provided, assumed to be Usage.
Line Item	lineitem/description	string	Description	An optional description specifying additional information about this line item.
Time	time/usage_start	datetime		The hour during which the charged usage applies. This value should be aligned to the start of the hour and will be treated as such. Note, the current version of the Common Billing Format does not currently use usage_end date.	✅
Time	time/usage_end	datetime		The end of a timespan to which the charged usage applies. Note that although this may be specified, currently all changes are treated as occurring in a single hour at usage_start. Future updates to the ingest process may take advantage of the usage_end value.
Resource	resource/id	string	Resource	Uniquely identifies the object for which this charge applies. This column is required if any `resource/tag:<key>` columns are included. Read more about Resource IDs below.	✅*
Resource	resource/service	string	Service	The category to which this resource belongs. Generally represents different kinds of services provided by the Cloud Provider for which the customer is charged.
Resource	resource/account	string	Account (if action/account is not specified)	The account or project to which this resource belongs (if applicable).
Resource	resource/region	string	Region (if action/region is not specified)	The region to which this resource belongs (if applicable).
Resource	resource/usage_family	string	UsageFamily	Commonly a subdivision of resource/service
Resource	resource/tag:`<key>`	string	Tag:`<key>`	Custom attributes associated with the resource. The `<key>` is case sensitive, and can consist of letters, numbers, and the following special characters `_ . : / = + - @.`. Using this column also requires `resource/id` to be populated. Read more about tags below.
Action	action/operation	string	Operation	The thing that was done to the resource for which you're being charged.
Action	action/usage_type	string	UsageType	Commonly a subdivision of action/operation
Action	action/region	string	Region	In what region the operation was performed. (May not match the resource/region for cross-region operations.)
Action	action/account	string	Account	In what account the operation was performed. (May not match the resource/account for cross-region operations.)
Usage	usage/amount	number		A numeric value describing an amount consumed/used. E.g. GB stored/transferred, seconds executed, credits consumed
Usage	usage/units	string		A description of the units used for usage/amount
Cost	cost/cost	number	Billed Cost	The cost associated with this line item. (May be negative for line items which represent discounts / credits.) Sometimes referred to as unblended cost, billed cost, or invoiced cost.	✅
Cost	cost/discounted_cost	number	Discounted Cost	The net cost associated with this line item after any discounts, credits, or private pricing are applied. Sometimes referred to as net cost or net unblended cost.
Cost	cost/amortized_cost	number	Amortized Cost	The effective cost associated with this line item after any "committed use" purchases (e.g. AWS RIs or Savings Plans, GCP CUDs, etc.) are amortized over all the "Usage" line items to which they apply.
Cost	cost/discounted_amortized_cost	number	Discounted Amortized Cost	The net effective cost associated with this line item after any discounts, credits, or private pricing are applied. It also includes any "committed use" purchases (e.g. AWS RIs or Savings Plans, GCP CUDs, etc.) amortized over all the "Usage" line items to which they apply. Sometimes referred to as net amortized cost, net effective cost, or fully-loaded cost.
Cost	cost/on_demand_cost	number	On-demand Cost	What the cost of this line item would have been if no mechanism for reducing costs applied. In other words, it is the cost as if there were no discounts, applicable "committed use" purchases, private pricing, etc. This is often useful for determining one's effective savings rate.
Bill	bill/invoice_id	string	InvoiceId	Uniquely identifies a particular bill. A single billing data ID for a single month may include multiple invoices. Ideally this field will not be populated until an Invoice is closed.
Kubernetes	k8s/cluster	string	Kubernetes Cluster	The Kubernetes cluster associated with the resource.
Kubernetes	k8s/namespace	string	Kubernetes Namespace	The Kubernetes namespace associated with the resource.
Kubernetes	k8s/deployment	string	Kubernetes Deployment	The Kubernetes deployment associated with the resource.
Kubernetes	k8s/labels	string	Kubernetes Labels	The Kubernetes labels associated with the resource.

*resource/id is only required when a resource/tag:<key> value is included for the charge.

Resource IDs

The resource/id is the unique identification of the resource for which the charge applies.
When resource/id is specified, you may also provide resource/tag:<key> columns to categorize the resource by <key>, which may be a purpose, owner, environment, or other criteria.
- When including resource/tag:<key> data with a charge, the resource/id column is required.
- Only one resource/tag:<key> column <value> will be associated with each unique resource/id value. For more information, see the Resource Tags section below.

Resource Tags

resource/tag:<key> columns allow you to categorize resources by purpose, owner, environment, or any other criteria.
The <key> is case sensitive, and can consist of letters, numbers, and the following special characters _ . : / = + - @..
- No spaces, tabs or returns are allowed.
- For example, resource/tag:team or resource/tag:Product-Type are valid tag keys, while resource/tag:Team Name and resource/tag:Product&Dev are not.
resource/tag:<key> will appear as Tag dimensions in the CloudZero platform, and can be used to filter or aggregate the cost of the tagged resources.
You may include as many unique resource/tag:<key> columns as you need.
Only one <value> will be associated with a resource/tag:<key> column for each unique resource/id.
- If multiple values are supplied for the same ID, the value with the latest time/usage_start will be used. If all instances for a given key have identical time/usage_start dates, then one is arbitrarily selected.
The latest <value> of a given resource/tag:<key> for each resource/id will be retroactively applied to all data in the CloudZero system with the same resource/id.

Resource ID & Tag Usage Example

B Large, Inc. is using the CloudZero platform to monitor their cloud costs, and needs to easily identify their resources by the teams that own them. To assist with this, B Large creates a Resource Tag called resource/tag:team, and populates it with the names of the various teams that own the AWS resources in question.

Team Alpha owns an AWS Lambda function with the Resource ID of making-great-calculations. Since these resources have been tagged as owned by Team Alpha using the resource/tag:team key with the value of Alpha, whenever anyone logs into CloudZero, they can easily use the team Tag dimension to view what Alpha's Lambda is costing B Large, and monitor trends over time.

Recently, B Large promoted more personalized team names, and Team Alpha voted to name themselves Team Batman. With this change, the resource/tag:team value in AWS is updated to reflect their new name, and CloudZero receives a new data drop that associates a new resource/tag:team value of Batman for the AWS Lambda making-great-calculations.

The CloudZero platform receives this updated tag information, and because only one value per resource/tag:team can be associated with a single resource/id of making-great-calculations, the platform retroactively applies this new tag to all historical data in the system for the resource/id making-great-calculations.

Now, when anyone at B Large logs into CloudZero, they can still see what Team Batman owns, and because the tag change was retroactively applied to all historical data, the monitoring of trends over time for the making-great-calculations Lambda sees no interruption, and reports continue to accurately reflect totals and trends, including those that were ingested prior to Team Batman's name change.

Line Item Types

The lineitem/type column defines how each line items should be interpreted.

The following values are supported:

Line Item Type	Description
Usage	The most common type and the default. The line item represents a charge for the use of some cloud resource. “Real Cost” includes only Usage line items and uses the first available cost type provided from: amortized_cost, discounted_cost, or cost
Tax	This line item represents any tax charges. Columns in the time, resource, and action categories should only be populated if the tax is associated with applicable Usage changes. Otherwise these columns should be left blank.
Support	Charges for support or other human services. Columns in the time category should specify the start and end of the support contract.
Purchase	Charge for a one-time purchase or non-usage based subscription; for example, software bought from the AWS Marketplace. The time category columns should represent the span of time over which the purchase applies (for subscriptions with renewal) or the time of the purchase (for one-time charges).
CommittedUsePurchase	Charges for a "committed use" (RI, savings plan, etc.) purchase. "Committed use" includes any instrument for which payment is made to receive a reduced rate on future usage. This may include one-time upfront purchases or recurring monthly charges. The time category columns should represent the span of time over which the charge applies (for example, one month for monthly recurring charges.) The amortized_cost cost column of this line item should always be zero (if used).
Discount	A negative value cost associated with some Usage line item. Columns in the time, resource, and action categories of this line item should match those of the applicable Usage line item and the cost should be negative. The discounted_cost for this line item should be zero if this discount is fully accounted for in the discounted_cost of other Usage line items. (Please note: you must still include discounted_cost if it should be zero)
Credit	A negative value cost not associated with any specific Usage line items. May represent an adjustment due to an error, rounding, refund, etc.
Fee	A positive value charge for which no other line item type applies.
Adjustment	An alteration made to the bill to correct for some error or rounding issue.

Cost Category Columns

There are five cost category columns cost, discounted_cost, amortized_cost, discounted_amortized_cost, and on_demand_cost. Only cost is required, but the others may be used to help “smooth out” charges not directly associated with the use of cloud services.

To ensure all cost columns are fully populated and may be used in the platform, blank values will "fall back" and use a different cost column. For example, if discounted_amortized_cost is left blank, discounted_cost will be used instead. The assumption is that no line items required amortization (i.e. no committed use purchases) and so the discounted cost could be used instead. Only blank or empty values trigger this behavior, if the value is set to 0 it will be used as is. The complete set of fallback logic is:

discounted_amortized_cost → amortized_cost → discounted_cost → cost
amortized_cost → cost
discounted_cost → cost
on_demand_cost → cost

The total value across all line items for each of cost, discounted_cost, amortized_cost, and discounted_amortized_cost should be the same. Each represents a redistribution of charges (discounts and/or purchases) not the inclusion or exclusion of charges. This equivalence is not currently enforced by the billing ingest system, but may be in the future.

Cost Types

CloudZero cost types are defined in terms of the lineitem/type and different cost category columns.

Real Cost - Only Usage line item types are included. The discounted_amortized_cost column is used, if present, with “fallback” logic to the other cost categories if it is not.
Discounted Amortized Cost - All line item types are included. The discounted_amortized_cost column is used, if present, with “fallback” logic to the other cost categories if it is not.
Amortized Cost [Coming Soon] - All line item types are included. The amortized_cost column is used, if present, with “fallback” to cost if not.
Discounted Cost - All line item types are included. The discounted_cost column is used, if present, with “fallback” to cost if not.
On-Demand Cost - All line item types are included. The on_demand_cost column is used, if present, with “fallback” to cost if not.
Billed Cost - All line item types are included. The cost column is used.

Examples

Simple

In this example we purchased some Compute from “Simple Cloud”. We also made an up-front “committed use” which lowered the normal hourly rate by 50%. We received the “SpecialCompute” discount for using certain Compute instance type (Simple Cloud is running a promotion.) We also received the “MVP Discount” because we are so cool.

This first version of the bill just includes the actual charges. The cost for our Compute instances is at the reduced rate because of the “committed use” purchase. (For example, instance-0000 would have otherwise cost $24 during that hour.) The sum of the values in the cost/cost column should equal exactly what I see on my bill for March: $105.30

lineitem/type	resource/service	resource/id	time/usage_start	cost/cost
Usage	Compute	instance-0000	2022-03-16T13:00:00Z	12
Usage	Compute	instance-0001	2022-03-16T13:00:00Z	20
Usage	Compute	instance-0002	2022-03-16T13:00:00Z	15.3
Purchase	CommitedUse	commit-111-222-333	2022-03-01T00:00:00Z	90
Discount	SpecialCompute	special-01010101	2022-03-16T13:00:00Z	-12
Discount	MVPDiscount	mvp-aaa-12345	2022-03-01T00:00:00Z	-20

Discounted Cost

It would be helpful if the discount we received for using the promoted Compute instances was actually represented in the cost of those instances. In this example we’ll include the cost/discounted_cost

lineitem/type	resource/service	resource/id	time/usage_start	cost/cost	cost/discounted_cost
Usage	Compute	instance-0000	2022-03-16T13:00:00Z	12	8
Usage	Compute	instance-0001	2022-03-16T13:00:00Z	20	16
Usage	Compute	instance-0002	2022-03-16T13:00:00Z	15.3	11.3
Purchase	CommitedUse	commit-111-222-333	2022-03-01T00:00:00Z	90	90
Discount	SpecialCompute	special-01010101	2022-03-16T13:00:00Z	-12	0
Discount	MVPDiscount	mvp-aaa-12345	2022-03-01T00:00:00Z	-20	-20

The SpecialCompute discount was distributed evenly over the individual instances. We also zeroed out the cost for the SpecialCompute line item because that discount has already been accounted for. We chose not to distribute the MVPDiscount since it was unrelated to the Compute instances. We copied over the values for the CommitedUse purchase and MVPDiscount, but we could have also left them blank and the cost/cost value would have been used by default.

Amortized Cost

To help our engineers better understand the real cost of those Compute instances we also want to include what we payed for the CommitedUse purchase.

lineitem/type	resource/service	resource/id	time/usage_start	cost/cost	cost/discounted_cost	cost/amortized_cost
Usage	Compute	instance-0000	2022-03-16T13:00:00Z	12	8	38
Usage	Compute	instance-0001	2022-03-16T13:00:00Z	20	16	46
Usage	Compute	instance-0002	2022-03-16T13:00:00Z	15.3	11.3	41.3
Purchase	CommitedUse	commit-111-222-333	2022-03-01T00:00:00Z	90	90	0
Discount	SpecialCompute	special-01010101	2022-03-16T13:00:00Z	-12	0
Discount	MVPDiscount	mvp-aaa-12345	2022-03-01T00:00:00Z	-20	-20

The cost of the CommitedUse purchase was evenly distributed over all the Compute instances to which it applied (which in this case was all of them.) We left the cost/amortized_cost for the Discounts blank. We could have also copied over the values from the discounted_cost (0 and -20). The important this is the the sum of the values in the cost/amortized_cost column (including the default values for the blank cells) is still: $105.30

Kubernetes-Related Columns

In this example, we're adding Kubernetes-related columns to the existing CBF data to provide more context on resource usage within a Kubernetes environment.

lineitem/type	resource/service	resource/id	time/usage_start	k8s/cluster	k8s/namespace	k8s/deployment	k8s/labels	cost/cost
Usage	Compute	instance-0000	2022-03-16T13:00:00Z	my-cluster	default	web-app	{"app": "frontend", "env": "production"}	12
Usage	Compute	instance-0001	2022-03-16T13:00:00Z	my-cluster	default	web-app	{"app": "frontend", "env": "production"}	20
Usage	Compute	instance-0002	2022-03-16T13:00:00Z	my-cluster	default	worker-app	{"app": "backend", "env": "production"}	15.3

In this example, we're including Kubernetes-related information such as cluster, namespace, deployment, and labels. This allows us to track usage and costs within specific Kubernetes contexts.