Kubernetes Istio Configuration
Installing the CloudZero Agent for Kubernetes in Istio-Enabled Clusters
When installing the CloudZero Agent for Kubernetes Helm chart in a Kubernetes cluster that uses Istio, additional steps may be required to ensure proper functionality. This is because Istio automatically configures sidecars to use mutual TLS (mTLS), which can interfere with the agent’s existing TLS communication.
Why These Steps Are Needed
The CloudZero Agent for Kubernetes includes a webhook server component responsible for handling admission review requests from the Kubernetes API server. These requests use TLS, and when intercepted by an Istio sidecar, Istio may attempt to apply its mTLS policies. These policies are not always compatible with the webhook’s TLS configuration.
While this does not block pod deployments, it prevents the insightsController
from collecting critical pod labels, which are necessary for accurate cost allocation.
To ensure the CloudZero Agent for Kubernetes works correctly in Istio-enabled clusters, you can choose from the following options:
- Disable sidecar injection for the webhook-server pods only — Completely removes the Istio sidecar from the webhook server, avoiding any interference.
- Disable envoy for webhook ports only — Keeps the sidecar but excludes webhook traffic, preserving Istio functionality for all other traffic.
- Disable mTLS for the webhook-server pods — Keeps the sidecar but disables mTLS enforcement specifically for webhook-server traffic.
Option 1: Disable Sidecar Injection for the CloudZero Agent for Kubernetes webhook-server Pods Only
To disable the sidecar injection only for a subset of the pods deployed by the cluster, update the chart input values with the following annotation:
insightsController:
server:
podAnnotations:
sidecar.istio.io/inject: "false"
This annotation prevents the Istio sidecar from being injected into the webhook-server
pods, while the rest of the cluster retains normal Istio functionality.
Option 2: Disable Envoy for Webhook Ports Only
To prevent only requests to a single port on the webhook-server pods from being routed through envoy, apply the following annotation:
insightsController:
server:
podAnnotations:
traffic.sidecar.istio.io/excludeInboundPorts: "8443"
In this case, the pods will still have an Istio sidecar injected, but traffic to port 8443 (the webhook port) will bypass envoy.
For more details, see here.
Option 3: Disable mTLS for the CloudZero Agent for Kubernetes
To disable mTLS for the CloudZero Agent for Kubernetes service, apply the following PeerAuthentication
resource:
apiVersion: security.istio.io/v1beta1
kind: PeerAuthentication
metadata:
name: cloudzero-agent-mtls
namespace: <cloudzero-namespace>
spec:
selector:
matchLabels:
app.kubernetes.io/component: webhook-server
mtls:
mode: DISABLE
Steps to Apply:
- Replace
<cloudzero-namespace>
with the namespace wherecloudzero-agent
is deployed. - Apply the resource:
kubectl apply -f cloudzero-agent-mtls.yaml
- Deploy the
cloudzero-agent
chart as instructed in the documentation.
This configuration disables mTLS for cloudzero-agent
webhook-server pods only, while keeping it enabled for the rest of the cluster.
Updated 4 days ago