S3 and data transfer: the two AWS lines nobody audits

The bucket costs 23 dollars per terabyte per month. That number is the reason nobody looks at S3, and it is also the least interesting number in the entire service. What we find on real accounts is a storage line inflated by objects that were tiered too aggressively, versions nobody knows exist, and uploads that never finished, sitting next to a data transfer line that is often larger than the storage itself and appears under a completely different heading on the bill.

Storage and transfer are one problem, because the decisions that shape one shape the other. A lifecycle rule that moves ten million small objects to Glacier is a transfer-and-request event before it is a storage saving. A private subnet without a gateway endpoint pays NAT charges to talk to a bucket in the same region. Here is the audit we run.

Storage classes charge for time and for size, not just for bytes

Every colder S3 class carries two penalties that the per-gigabyte price hides.

The first is minimum billable duration. Standard-IA and One Zone-IA bill a minimum of 30 days per object, Glacier Instant Retrieval 90, Glacier Flexible Retrieval 90, Deep Archive 180. Delete or transition an object earlier and you pay the remainder anyway. A lifecycle policy that tiers on day 1 and expires on day 45 is paying a penalty on every single object, and the more objects, the worse it gets.

The second is minimum billable object size. The infrequent-access and Glacier classes bill small objects as if they were 128 KB. If your bucket is forty million thumbnails averaging 12 KB, moving them to Standard-IA multiplies the billed volume by roughly ten. That transition makes the bill go up, not down, and we have watched it happen.

The rule that follows: colder classes are for large objects with long lives. Before you write any lifecycle rule, get the object count and the average object size from S3 Storage Lens. If the average is under about 128 KB, the answer is Standard, or it is a packaging change so the objects stop being tiny.

Intelligent-Tiering is right until the monitoring fee eats it

Intelligent-Tiering solves the guessing problem: it moves objects between access tiers based on real access patterns, with no retrieval charges and no early-deletion penalty across the frequent, infrequent and instant-access tiers. For a bucket with an unpredictable read pattern, it is usually the correct default.

The catch is a small monthly monitoring and automation charge per object. For a hundred thousand large objects it is noise. For eighty million tiny ones it can exceed what you would have paid in Standard. Objects below 128 KB are not monitored or auto-tiered at all, so you are paying attention to the class for the wrong half of the bucket. Do the arithmetic on object count first; the crossover is a division, not a judgement call.

Transitions cost money per request

Lifecycle transitions are billed per thousand objects, and the Glacier classes cost noticeably more per transition than the IA ones. Tiering forty million objects is a real, one-off charge that shows up in the month you turn the rule on, and it will scare somebody. Model it before you enable the rule, and never write a lifecycle configuration that transitions through several classes in sequence: each hop bills again.

The things that accumulate

• Non-current versions. Versioning is on, nobody set an expiry, and the bucket is quietly storing every revision of every object since the day it was enabled. On write-heavy buckets the non-current tier is frequently larger than the current one. Add a NoncurrentVersionExpiration rule.
• Delete markers with no versions behind them. Deleting in a versioned bucket creates a marker, not a deletion. Orphaned markers cost little but they wreck listing performance and hide the real size. ExpiredObjectDeleteMarker cleans them.
• Incomplete multipart uploads. A failed 5 GB upload leaves parts that bill as storage and appear in no ls. Every bucket should have an AbortIncompleteMultipartUpload rule at 7 days. This is the single most common finding we have, and it is one line of Terraform.
• Request cost on small-object workloads. A million GET requests against a bucket of 5 KB objects costs more than storing them. If a service reads the same small objects constantly, the fix is a cache, not a storage class.
• Replication you forgot. Cross-region replication pays for the transfer, the destination storage and the requests, forever, for every new object. Check what is replicating and why.
• Logging into the bucket being logged. Server access logs written to the same bucket generate objects that generate logs. Point them elsewhere with a lifecycle expiry.

Then the transfer, in the order it bites

Inbound is free. Everything after that is about where the boundary sits, and the boundaries are the same ones that cause the design problems in the VPC mistakes that cost real money.

• Internet egress is the headline rate, tiered down by volume, with a monthly free allowance. CloudFront in front of a bucket usually lowers it twice: the CDN's own egress rate is lower, and origin fetches from S3 to CloudFront are free.
• NAT gateway data processing is charged per gigabyte on top of the hourly price, in both directions, and it applies to traffic that never leaves AWS. A private subnet talking to S3 or DynamoDB through NAT is paying for nothing. Gateway VPC endpoints for S3 and DynamoDB are free and remove that traffic from the NAT path entirely. If you fix one thing this month, fix this one.
• Interface endpoints (PrivateLink) for the other services cost an hourly fee per endpoint per availability zone plus a per-gigabyte charge, which is still usually cheaper than NAT for high-volume service traffic, and gives you a real security boundary as a side effect.
• Cross-availability-zone traffic inside a region is billed in both directions. In a multi-AZ Kubernetes cluster with topology-unaware services, most pod-to-pod traffic crosses a zone, the effect we describe in EKS spend.
• Cross-region transfer for replication, chatty multi-region applications and backups. Usually an architecture finding rather than a knob.

What to do this week

Open S3 Storage Lens on the account, sort buckets by non-current version bytes and by incomplete multipart upload bytes, and take the top three of each. Then list every VPC in the account and check which ones have a gateway endpoint for S3. Both lists take twenty minutes and both are usually longer than anyone expects. We do exactly this in the cost phase of a cloud engagement, alongside the five layers of an AWS bill.