Installation of CloudZero integration

Installation of CloudZero Container Cost Solution

The CloudZero CloudWatch agents collects, aggregates, and summarizes metrics from containerized applications and microservices running in AWS EKS or native Kubernetes on EC2 in the form of a CloudWatch performance log. CloudZero uses metrics from the Performance log to determine how to allocate your container costs.

Solution Overview with Needed Permissions

An agent is installed as a Deamonset and writes performance metrics to CloudWatch Logs. The CloudZero platform aggregates and pulls the data into CloudZero. Access is achieved via an cross account role created by connecting the account to CloudZero as a Resource Account. This image provides a high level overview.

Summary of Permission needed

1. Kubernetes RBAC
2. Node Permission to write to CloudWatch Logs.
3. Cross Account Role access from CloudZero to CloudWatch Logs

CloudZero Agent Installation Instructions

Here is an outline of the steps to complete the installation:

1. Verify prerequisites. e.g. Adding the AWS Managed policy arn:aws:iam::aws:policy/CloudWatchAgentServerPolicy to the cluster nodes
2. Use Helm to install the CloudZero Helm Chart

Prerequisites

The agent must have permission to create and write to a CloudWatch LogGroup and LogStream. Adding the AWS Managed policy arn:aws:iam::aws:policy/CloudWatchAgentServerPolicy to the cluster nodes allows this. Review how you manage your nodes to see how best to attach this managed policy to the node role before installation.

CloudZero platform requires permission to read the LogGroup the agent is writing to. This is accomplished by connecting the AWS account the cluster resides to the CloudZero platform as a "Resource Account". Follow the same process to connect your management account in the Account Connection section of our documentation.

Helm Chart

For reference, the Chart is located here: https://github.com/Cloudzero/cloudzero-k8s-charts

Considerations Before Installing

The agent consumes CPU and memory in relation to your cluster density. The density is defined by counting resources like number of nodes, pods, endpoints, and replicasets. The chart values defined for CPU and memory limits/requests are suitable for a cluster density of 300 nodes, 5000 pods, and 70,000 ReplicaSets.

Installation

Add the Cloudzero repository to Helm:

helm repo add cloudzero https://cloudzero.github.io/cloudzero-k8s-charts

Install the latest version:

helm upgrade --install cloudzero-cloudwatch-metrics           \
             cloudzero/cloudzero-cloudwatch-metrics           \
             --namespace cloudzero-metrics --create-namespace \
             --set clusterName=<Your Cluster>

Note: these are helm3 commands that creates a namespace for this deployment or you can use an existing namespace.

Validation

First, determine if the agent is running. (ensure your kubectl focus is on the cluster you deployed too)

kubectl get pods -A | grep cloudzero

Second, validate the agent has created the performance log group in AWS CloudWatch logs. (note: it might take 5 minutes for the LogGroup to show up. Get a coffee and then validate)

The LogGroup will have the following format

"/aws/containerinsights/<Cluster Name>/performance"

You can validate this in the console or via the AWS CLI.

aws logs describe-log-groups | grep <Cluster Name>

You can checkout the logs too. Grab one of the pod names for the kubectl get pods listed above.

kubectl logs --namespace cloudzero-metrics <Pod Name>

You can validate the number of agents running matches the number of nodes in the cluster.

kubectl get node --no-headers --selector='!node-role.kubernetes.io/control-plane' | wc -l | awk {'print "node  count = "$1'} && kubectl get pods --namespace cloudzero-metrics --no-headers | wc -l | awk {'print "agent count = "$1'}

Additional Information

Presently, the CloudZero Agent is unable to set the retention period of the log group that is created. It is on our road map to make this configuration at installation time. For now, we suggest you set the retention period to 7 days. This can be done in the AWS Console or via the AWS CLI

aws logs put-retention-policy \
                --log-group-name /aws/containerinsights/<Cluster Name>/performance \
                --retention-in-days 7

📘

Please Note

Once set up, the EKS integration will produce a log line every minute for each pod in the cluster.

Supported Platforms

Amazon Elastic Kubernetes Service (Amazon EKS) on Amazon EC2
Self-managed Kubernetes infrastructure on Amazon EC2
Amazon ECS on Fargate can be supported in limited situations
Amazon ECS on EC2 can be supported in limited situations
NOTE: AWS EKS on AWS Fargate is not supported at this time