(Legacy) Installation of CloudZero integration
Install and configure CloudZero Kubernetes agent on AWS
Legacy method
This method of container metric data ingestion to the CloudZero platform is no longer the preferred approach. If you are looking to bring new container metrics into CloudZero, please refer to Installation of CloudZero Agent for Kubernetes.
Installation of CloudZero Container Cost Solution
The CloudZero CloudWatch agents collects, aggregates, and summarizes metrics from containerized applications and microservices running in AWS EKS or native Kubernetes on EC2 in the form of a CloudWatch performance log. CloudZero uses metrics from the Performance log to determine how to allocate your container costs.
Solution Overview with Needed Permissions
An agent is installed as a Deamonset and writes performance metrics to CloudWatch Logs. The CloudZero platform aggregates and pulls the data into CloudZero. Access is achieved via a cross account role created by connecting the account to CloudZero as a Resource Account. This image provides a high level overview.
Summary of Permission needed
- Kubernetes RBAC
- Node Permission to write to CloudWatch Logs.
- Cross Account Role access from CloudZero to CloudWatch Logs
CloudZero Agent Installation Steps
Here is an outline of the steps to complete the installation:
- Verify the AWS account containing the cluster is configured in CloudZero
- Grant Kubernetes permissions to write to AWS CloudWatch
- Use Helm to install the CloudZero Helm Chart
- Validate the installation
Considerations Before Installing
The agent consumes CPU and memory in relation to your cluster density. The density is defined by counting resources like number of nodes, pods, endpoints, and replicasets. The chart values defined for CPU and memory limits/requests are suitable for a cluster density of 300 nodes, 5000 pods, and 70,000 ReplicaSets.
Verify AWS account is configured in CloudZero
CloudZero platform requires permission to read the LogGroup the agent is writing to. This is accomplished by connecting the AWS account the cluster resides to the CloudZero platform as a "Resource Account".
If you have already configured AWS account using the preferred "Automated" method, the requisite policies and roles have already been created for you.
If you have not yet configured this account follow the same process to connect your management account in the Connecting to AWS section of our documentation.
If you have already configured this AWS account using the manual method, verify in the AWS IAM console that cross account access role exists and contains the permissions provided by the account setup wizard in the CloudZero console.
Grant Kubernetes Permissions
The agent must have permission to create and write to a CloudWatch LogGroup and LogStream. There are several ways to grant this:
- Adding the AWS Managed policy
arn:aws:iam::aws:policy/CloudWatchAgentServerPolicy
to the instance role assigned to the cluster nodes. - Use a Kubernetes service account to assume an IAM role. https://docs.aws.amazon.com/eks/latest/userguide/associate-service-account-role.html. NOTE: To use this method, you must manually create a new IAM policy and Role. The policy must include all of the allows from managed policy
arn:aws:iam::aws:policy/CloudWatchAgentServerPolicy
Review how you manage your Kubernetes cluster IAM permissions to see which of the above options is best to attach this managed policy.
Helm Chart
For reference, the Chart is located here: https://github.com/Cloudzero/cloudzero-k8s-charts
Installation
Add the Cloudzero repository to Helm:
helm repo add cloudzero https://cloudzero.github.io/cloudzero-k8s-charts
Install the latest version:
helm upgrade --install cloudzero-cloudwatch-metrics \
cloudzero/cloudzero-cloudwatch-metrics \
--namespace cloudzero-metrics --create-namespace \
--set clusterName=<Your Cluster>
Note:
--set clusterName
should always be usedIf
--set clusterName=
is not specifified the helm chart will set a default cluster name ofcluster_name
. If you plan to configure the agent on more than one cluster and do not set this you will not be able to distinguish between clusters in CloudZero.
Note: these are helm3 commands that creates a namespace for this deployment or you can use an existing namespace.
Validation
First, determine if the agent is running. (ensure your kubectl focus is on the cluster you deployed too)
kubectl get pods -A | grep cloudzero
Second, validate the agent has created the performance log group in AWS CloudWatch logs. (note: it might take 5 minutes for the LogGroup to show up. Get a coffee and then validate)
The LogGroup will have the following format
"/aws/containerinsights/<Cluster Name>/performance"
You can validate this in the console or via the AWS CLI.
aws logs describe-log-groups | grep <Cluster Name>
You can checkout the logs too. Grab one of the pod names for the kubectl get pods listed above.
kubectl logs --namespace cloudzero-metrics <Pod Name>
You can validate the number of agents running matches the number of nodes in the cluster.
kubectl get node --no-headers --selector='!node-role.kubernetes.io/control-plane' | wc -l | awk {'print "node count = "$1'} && kubectl get pods --namespace cloudzero-metrics --no-headers | wc -l | awk {'print "agent count = "$1'}
Additional Information
Presently, the CloudZero Agent is unable to set the retention period of the log group that is created. It is on our road map to make this configuration at installation time. For now, we suggest you set the retention period to 7 days. This can be done in the AWS Console or via the AWS CLI
aws logs put-retention-policy \
--log-group-name /aws/containerinsights/<Cluster Name>/performance \
--retention-in-days 7
Please Note
Once set up, the integration will produce a log line every minute for each pod in the cluster.
Supported Platforms
- Amazon Elastic Kubernetes Service (Amazon EKS) on Amazon EC2
- Self-managed Kubernetes infrastructure on Amazon EC2
- Amazon ECS on Fargate can be supported in limited situations
- Amazon ECS on EC2 can be supported in limited situations
NOTE: AWS EKS on AWS Fargate is not supported at this time*
Updated about 2 months ago