Connections are how CloudZero manages the various Cost Sources that bring Billing, Resource, and other types of data into the platform.

How the GCP Connection Works

Connecting to a GCP account will show GCP cost data alongside other Cost Sources in the Explorer, as well as enable anomaly alerts on GCP spend.

The CloudZero platform will ingest GCP Cost data by using the GCP Cloud Billing Data Export to BigQuery feature. This export will enable the CloudZero platform to be able to get accurate cost information. Once the export is created, you will then need to grant a CloudZero service principal access to the data.

In the following steps, you will find instructions on how to create these exports and configure CloudZero's access to that cost data.

📘

About CloudZero's Access to your GCP Accounts

All of CloudZero's permissions are Read-Only

We have no access to data except where explicitly authorized (for example, the data you explicitly share with CloudZero).

Summary of Permissions:

• BigQuery Data Viewer: grants read access only, to allow loading data from Cloud Billing Export files stored in BigQuery

Connect a GCP Billing Account

Step 1: Configure Cloud Billing Data Exports

The first step is to configure your Cloud Billing Data Exports in the Google Cloud console. You may already have completed these steps for other purposes, but please confirm the steps here match what you have previously set up.

1. Setup a Detailed usage cost data export in GCP BigQuery by following the Set up Cloud Billing data export to BigQuery instructions.

   • You may have already created usage cost data exports for other purposes, but CloudZero requires the Detailed usage cost data exports, so please confirm which type of export you have already configured and create additional Detailed exports if necessary.

2. Once the table is accessible, make note of its fully qualified name for use in later steps. This name consists of is the Project ID, Dataset and Table name of the table you configured.

   • For example: billing-administration-123456.all_billing_data.gcp_billing_export_resource_v1_123456_123456_123456
   • In that example:
     • Project ID: billing-administration-123456
     • Dataset: all_billing_data
     • Table name: gcp_billing_export_resource_v1_123456_123456_123456
     • Tip: The fully qualified table name will always include the word resource. If this is missing, please reconfirm your export type is Detailed usage cost data.
   • This name can be copied easily from the table details, listed next to the Table ID:
     

📘

First time setting up Cloud Billing exports?

If this is the first time you've setup Cloud Billing data exports in BigQuery, please be aware that it can take up to 24-48 hours for your export table to appear. It does not appear until GCP does it's first data drop, which can take a day or two. You can not continue the rest of the connection process until the table has been created.

Step 2: Configure Your Connection in the Platform

Once your Cloud Billing Data Export table is accessible, you will want to create your connection in the CloudZero platform.

📘

Note: Admin Role Required

You must be a CloudZero Admin to add new Connections to the platform.

1. Open the Cloud Integrations page. This can be found by selecting the gear icon from the top navigation and selecting Cloud Integrations, or alternatively going to https://app.cloudzero.com/organization/connections

2. Select the Add Connection button.
3. On the following page you'll be presented with a list of possible connection types. Select the GCP tile to begin setting up a GCP connection.
4. On the subsequent page, review the information and when you're ready, select Get started.
5. The Connection Details page for a GCP connection will be displayed, and you'll want to enter the information as follows:
   • Connection Name: This is the name you will see throughout the CloudZero platform, in addition to your GCP Account ID.
   • Service Principal: This is the Service Principal we use to access your GCP data. Make note of this value, as you will need it for Step 3: Grant Access to CloudZero.
   • Fully Qualified Table Name: This is the billing data table that you setup in Step 1. It should include the project, dataset, and table name.
     • For example: billing-administration-123456.all_billing_data.gcp_billing_export_resource_v1_123456_123456_123456
     • In that example:
       • Project ID: billing-administration-123456
       • Dataset: all_billing_data
       • Table name: gcp_billing_export_resource_v1_123456_123456_123456
     • This can be copied easily from the table details, listed next to the Table ID:
       
6. Select the Save button to save your connection.
7. You will see your connection on the Connection Details page. Please Note: Your connection may be in error until you complete Step 3 below to grant us access to your GCP billing data.

Step 3: Grant Access to CloudZero

At this point, you need to grant CloudZero access to your GCP Billing Data. You do this using the Service Principal saved from Step 2 above. In the instructions below, replace anywhere you see <cz-service-principal> with the Service Principal value you copied.

1. Log into the GCP Console
2. Select the Project which holds your billing account data and go to BigQuery
3. Find the billing data export table. It should look like: gcp_billing_export_resource_v1_<billing_account_id>
   
4. Select that table and click [+SHARE]:
   
5. Click [+ ADD PRINCIPAL]:
   
6. Add the service account and role, then click [SAVE]
   • Service Account: <cz-service-principal>
   • Role: BigQuery Data Viewer
     

Step 4: Return to the Cloud Integrations Page

Once your connection saves, it will appear in Billing Connections table of the CloudZero Cloud Integrations page with a status of Pending Data and a Last Checked status of Pending First Ingest.

At this time, the CloudZero platform will attempt to connect to your BigQuery table using the Service Principal assigned to your organization.

Once the connection has been verified, the Health column will update from Pending Data to Healthy.

If there are issues with your connection, you may see an Error status instead of a Healthy one. If this happens, You can hover over the status button to get additional information, but you may also want to verify your GCP table name saved to the connection, and that your Service Principal was properly granted access to the table (Step 2 above).

📘

Please Note

Discovery (the act of switching from a Pending First Ingest to a Healthy status) can take up to an hour.

It can take up to 24 hours to synchronize new accounts and being to see cost data in Explorer.

How to Limit Billing Data Ingestion into CloudZero

If you want only a subset of your billing data (e.g., specific projects, or a certain time range) ingested into CloudZero, you can create a view on top of the Billing Table. Follow these steps to use a view instead of a table:

1. Open a new Query Editor tab in the GCP BigQuery console.
2. Write the SQL for the view definition.
3. Alias _PARTITIONTIME as PARTITIONTIME.
4. Save the view to an existing BigQuery dataset.
5. Grant BigQuery Data Viewer permissions to the new view (see Step 3: Grant Access to CloudZero).
6. Create a new GCP Billing Connection using the fully qualified view name in lieu of the table name.

Here is an example view definition that restricts billing data from May 2024 onwards:

SELECT
    *,
    _PARTITIONTIME AS PARTITIONTIME
FROM `billing-administration-******.all_billing_data.gcp_billing_export_resource_v1_******_******_******`
WHERE TIMESTAMP_TRUNC(_PARTITIONTIME, DAY) >= TIMESTAMP("2024-05-01")

For more information on GCP views, see GCP Views.

Connecting Other Cost Sources

Additionally, you may want CloudZero to help with your other AWS, Snowflake, or Azure costs, or the custom costs you can bring in using an AnyCost Adaptor.

Start by learning about CloudZero Connections, or choosing a Cost Source from the What's Next section below.