AKS spend: the node pool decisions that set your bill

Azure charges almost nothing for the AKS control plane on the Free tier, which leads teams to believe AKS is the cheap option. Then the invoice arrives and it is all virtual machines, managed disks, load balancers and Log Analytics ingestion. None of that is AKS. All of it is decided by how you built the node pools.

Here is the order we work through a cluster.

Measure before you touch anything

Requests versus actual usage, per namespace. The ratio of requested CPU and memory to used CPU and memory is the single most useful number in the cluster. Above 3x you are paying for three nodes to run one node's worth of work. Container Insights gives you this, or Azure Monitor managed Prometheus if you want it properly, comparing kube_pod_container_resource_requests against real consumption.

Allocatable versus requested, per node. AKS reserves a meaningful slice of each node for the kubelet and system daemons, and that reservation is proportionally brutal on small SKUs. On a 2 vCPU node you can lose a quarter of the machine before a single workload pod schedules. If your pools are built from Standard_D2s, that alone is a large, invisible tax.

Cost per namespace. OpenCost or Kubecost, or a manual split of the node bill by requests. Without a per-team number nobody accepts that the waste is theirs, and nothing changes.

Log Analytics ingestion from the cluster. Container Insights at default settings collects stdout and stderr from every container in every namespace, plus kube events and inventory. On a busy cluster this is routinely a bigger line than a whole node pool. We cover the fix in the Azure Monitor bill.

The four changes that usually halve it

Split system and user pools, and stop paying for the wrong SKU. The system pool runs CoreDNS, metrics-server and the rest. It needs to be reliable, not large. Give it a small pool with CriticalAddonsOnly taint, then size the user pools for what the workloads actually request. Mixed-purpose single pools are how clusters end up with memory-optimised nodes running stateless web services.

Set requests from observed usage. Requests should sit near the p95 of real consumption, not at a round number someone typed when the service was new. Vertical Pod Autoscaler in recommendation mode gives you the numbers without acting on them. Run it a fortnight and apply the output by hand. Set memory limits, because the alternative is a node-wide out-of-memory event, and be sparing with CPU limits, which throttle at p99 while the node sits idle.

Turn on node autoprovisioning instead of hand-built pools. Cluster autoscaler scales the pools you defined in advance, so your bin packing is capped by the SKUs you guessed at months ago. Node autoprovisioning, which is Karpenter underneath, picks the VM size that fits the pending pods and consolidates workloads onto fewer nodes as they shrink. Consolidation alone typically removes 20 to 30 percent of the nodes from a cluster that has been running a year. Set a disruption budget and annotate the stateful workloads with karpenter.sh/do-not-disrupt before you enable it.

Put the right things on Spot pools. Stateless, replicated, restart-tolerant workloads belong on a Spot node pool with a diversified SKU list and an on-demand core for anything control-plane-adjacent. The discount is deep. What hurts teams is a single-replica stateful service on Spot and an eviction at three in the morning. Spot pools carry the kubernetes.azure.com/scalesetpriority taint by default, so you have to opt workloads in deliberately, which is the right way round.

The things people forget

• Idle dev and staging clusters. Scale user pools to zero outside working hours. The system pool has to stay up, so the saving is partial, but on a large non-production estate it is still real money. An automation account and a schedule; one afternoon.
• A Standard Load Balancer and public IP per cluster, plus a NAT gateway for egress. Four small clusters where one with namespaces would do is pure fixed overhead.
• Over-provisioned DaemonSets. A logging agent requesting 500m CPU across 40 nodes is 20 cores you are paying for and not using, and it schedules on every new node autoprovisioning brings up.
• Orphaned managed disks from PersistentVolumeClaims deleted with a Retain reclaim policy. Azure keeps them and bills for them forever.
• Premium tier you did not need. The Standard tier buys a financially backed uptime SLA and is the right default for production. Free is fine for dev. Premium is for long-term support on an old Kubernetes version, which is a reason to upgrade, not a reason to pay.

Scale to zero properly

Horizontal Pod Autoscaler scales on CPU and memory, which is the wrong signal for most of what actually needs scaling. KEDA, which ships as an AKS add-on, scales on queue depth, Service Bus message count, Event Hub lag or a Prometheus query, and it scales to zero. A worker deployment that runs two replicas overnight to process an empty queue is a small, permanent, entirely avoidable cost.

What good looks like

A healthy AKS cluster runs at 55 to 70 percent average CPU allocation, keeps requests within 1.5x of usage, brings up a node in under two minutes, and survives losing any single node without paging anyone. Reaching that is two or three weeks of work and rewrites nothing.

Only then does a commitment make sense. AKS nodes are ordinary virtual machines, so reservations and the Azure savings plan for compute apply to them exactly as they do elsewhere — which is why this work belongs before the purchase, in the order set out in the Azure bill. We do this in the cost phase of a cloud engagement.

What to do this week

Compute the requests-to-usage ratio for your three largest namespaces, and pull the last 30 days of Log Analytics ingestion attributed to Container Insights. One of those two numbers is almost certainly your answer, and neither takes more than an hour to get.