GCP Billing Data Integration
The following delineates the process of mapping GCP Billing Data into CloudZero's standard Common Billing Format (CBF). This ensures consistency, accuracy, and ease of interpretation when you are examining GCP-based billing records within the CloudZero platform.
Sample GCP billing record
{
"adjustment_info": null,
"billing_account_id": "00AABB-CCDD12-998877",
"cost": 1.987900000000000e-02,
"cost_type": "regular",
"credits": [],
"currency": "USD",
"currency_conversion_rate": 1.000000000000000e+00,
"export_time": 1653012525407000,
"invoice": {
"month": "202205"
},
"labels": [
{
"key": "airflow-version",
"value": "v1-10-6-composer"
},
{
"key": "goog-dataproc-cluster-name",
"value": "cust-datalake-generate-v2-20220518"
},
{
"key": "goog-dataproc-cluster-uuid",
"value": "c3f3a73c-6f27-4620-bae9-e9ba228e6a1a"
},
{
"key": "goog-dataproc-location",
"value": "us-central1"
}
],
"location": {
"country": "US",
"location": "us-central1",
"region": "us-central1"
},
"project": {
"ancestors": [
{
"display_name": "cz-project-id",
"resource_name": "projects/123420799999"
},
{
"display_name": "cloudzero.com",
"resource_name": "organizations/123429371234"
}
],
"ancestry_numbers": "/123429371234/",
"id": "cz-project-id",
"labels": [],
"name": "CZ Project",
"number": "123420799999"
},
"resource": {
"global_name": "//compute.googleapis.com/projects/123420799999/zones/us-central1-c/disk/6082010960894652432",
"name": "cust-datalake-generate-v2-20220518-w-14"
},
"service": {
"description": "Compute Engine",
"id": "6F81-5844-456A"
},
"sku": {
"description": "SSD backed PD Capacity",
"id": "B188-61DD-52E4"
},
"system_labels": [],
"tags": [],
"usage": {
"amount": 3.865465515688619e+14,
"amount_in_pricing_units": 1.344084270000000e-01,
"pricing_unit": "gibibyte month",
"unit": "byte-seconds"
},
"usage_end_time": 1652943600000000,
"usage_start_time": 1652940000000000
}
Complete details on the elements of this record can be found in the Google Cloud documentation
Mapping GCP data to CloudZero Dimensions
The following table documents the alignment between GCP billing data and CloudZero Dimensions.
Dimension | GCP Record Element | Notes |
---|---|---|
LineItemType | cost_type | cost_type values are mapped to equivalent line item types like: Usage, Tax, Discount |
Account | project.id | |
Service | service.description | |
UsageFamily | sku.description | |
Resource | resource.global_name | Elements of the global_name are extracted and converted to a CZRN. When the global_name is not provided, other elements of the line item (like account / service) are used to generate a CZRN. A local resource ID is created from a hash of all available labels. |
Region | location.region | |
UsageAmount | usage.amount_in_pricing_units | |
unit | usage.pricing_unit | |
UsageStartDate | usage_start_time | |
UsageEndDate | usage_end_time | |
K8s:Cluster | labels.value | Relevant if GKE is active. Consolidates values from specific label keys including (goog-k8s-cluster-name , gke-cluster ) |
K8s:Namespace | labels.value | Relevant if GKE is active. Consolidates values from specific label keys including (k8s-namespace , goog-k8s-namespace ) |
K8s:Pod | labels.value | Relevant if GKE is active. Consolidates values from specific label keys including (app , k8s-app , app.kubernetes.io/name , service , dag_id , task_id , name , statefulset.kubernetes.io/pod-name , job-name ) |
Tag:gcp_cz:ancestry_numbers | project.ancestry_numbers | |
Tag:gcp_cz:organization_name | project.ancestors.display_name | Selects the display_name linked to a resource prefixed with "organizations". |
Tag:gcp_cz:resource_name | resource.name | Currently only available for “Compute Engine” service |
Tag:gcp_cz:project_name | project.name | |
Tag:gcp:* | system_labels | |
Tag:project:* | project.labels | |
Tag:* | labels | Key/Value pairs are expended into “user tags” |
Updated 2 days ago