The S3 Cost Optimization Playbook

Originally published at rajesh.medampudi.com/blog/s3-cost-optimization-playbook.

Most S3 bills are wrong, and the fix takes an afternoon. The data sits in the most expensive class AWS offers (S3 Standard, $0.023/GB-month), nobody set a lifecycle policy, incomplete multipart uploads are silently billing for storage you can't even see in the console, and every byte your EC2 fleet pulls from S3 is routed out through a NAT Gateway when a free VPC Gateway Endpoint would do the same job for $0. None of this needs an architecture rewrite. It needs a checklist run in the right order.

Here is the order. The savings depend entirely on your access pattern — I will not promise you a number I can't see — but the mistakes below are so common that the question is usually how much, not whether. One number is just arithmetic: cold data that moves from S3 Standard ($0.023/GB-month) to Glacier Deep Archive ($0.00099/GB-month) drops about 96% on the storage line for those bytes, and on an — logs aged past 90 days into Deep Archive — that is exactly the lever that did the work. S3 cost optimization is the same boring discipline as the rest of the bill: see it, then decide what each byte should actually cost.

First, see the bill before you touch it

You cannot optimize what you cannot measure, and S3's default billing view tells you nearly nothing useful. Turn on S3 Storage Lens before anything else. The free tier gives you 62 metrics at the bucket level with 14 days of history, and crucially it includes cost-optimization metrics out of the box — including "Incomplete multipart upload bytes greater than 7 days old," which is the single most common source of money disappearing into storage nobody knows exists (AWS S3 Storage Lens docs, accessed 2026-06-18).

Storage Lens free metrics answer the three questions that decide everything that follows:

Which buckets hold the most bytes?
What storage class is that data sitting in right now?
Where are the incomplete multipart uploads?

For deeper per-prefix analysis or a longer history, Advanced metrics (15 months of data, recommendations) costs extra — worth it for a large estate, overkill for a single small account. Start free. Pay for advanced only once the free tier has told you the estate is big enough to justify it.

Pair this with S3 Storage Class Analysis on your busiest buckets. It watches access patterns and tells you which objects are candidates to move to Infrequent Access — so you set lifecycle thresholds from data, not from a guess.

Second, fix the silent leak: incomplete multipart uploads

This is the one nobody finds on their own, so it goes near the top.

When you upload a large object in parts and the upload fails partway — a dropped connection, a crashed job, an SDK that didn't clean up — the parts that did land stay in the bucket. You are billed for that storage. They do not appear in the normal object listing. They accumulate for years, and on a long-lived account the orphaned parts can add up to a meaningful fraction of the bill before anyone notices.

The fix is one lifecycle rule, applied to every bucket:

{
  "Rules": [
    {
      "ID": "abort-incomplete-mpu",
      "Status": "Enabled",
      "Filter": { "Prefix": "" },
      "AbortIncompleteMultipartUpload": { "DaysAfterInitiation": 7 }
    }
  ]
}

AWS supports a lifecycle rule that stops multipart uploads not completed within a set number of days and deletes the orphaned parts — and it applies to both existing and future uploads (AWS lifecycle config for incomplete MPU, accessed 2026-06-18). Seven days is the standard value; it is long enough that no legitimate in-flight upload gets killed, short enough that garbage doesn't pile up. And per AWS, removing incomplete multipart parts via lifecycle does not trigger early-delete charges — so there is no downside.

Set this rule on every bucket you own, today, before you do anything else. It is the cheapest win in the whole playbook.

Third, get the data into the right storage class

S3 storage classes span a roughly 23x price range, and the only thing separating them is access pattern. Here is the ladder, US East (N. Virginia), per GB-month:

Class	$/GB-month	Min duration	Min object size	Use it for
S3 Standard	$0.023	none	none	Active, frequently read data
Standard-IA	$0.0125	30 days	128 KB	Read a few times a month
One Zone-IA	~$0.01	30 days	128 KB	Reproducible IA data, single-AZ OK
Glacier Instant Retrieval	$0.004	90 days	128 KB	Archive needing instant access
Glacier Flexible Retrieval	$0.0036	90 days	—	Archive, minutes-to-hours retrieval
Glacier Deep Archive	~$0.00099	180 days	—	Compliance, rarely-ever read

Storage prices and the minimum-duration / minimum-size rules are from AWS primary docs (storage classes, Glacier classes, accessed 2026-06-18). Two rules carry most of the risk:

Minimum storage duration. Delete a Standard-IA object before 30 days and you still pay the full 30. Glacier Flexible and Instant bill a 90-day minimum; Deep Archive bills 180. Move data down the ladder only when it will actually sit there. Putting short-lived data in Glacier is a way to pay more, not less.
Minimum billable object size. Standard-IA, One Zone-IA, and Glacier Instant bill every object as if it were at least 128 KB. A bucket of 10 KB thumbnails moved to Standard-IA gets billed at 128 KB each — you pay for 12x the bytes you store. Small objects stay in Standard. This is the trap that quietly reverses the savings.

The decision rule is simple. Frequently read → Standard. Read occasionally, objects over 128 KB → Standard-IA. Rarely read but must be instant → Glacier Instant Retrieval. Archive you can wait minutes-to-hours for → Glacier Flexible. Compliance data you'll likely never read → Deep Archive.

Fourth, automate the transitions with lifecycle policies

You do not move data by hand. You write a lifecycle policy and S3 does it on a schedule. A typical policy for log or backup data:

{
  "Rules": [
    {
      "ID": "tier-down-logs",
      "Status": "Enabled",
      "Filter": { "Prefix": "logs/" },
      "Transitions": [
        { "Days": 30,  "StorageClass": "STANDARD_IA" },
        { "Days": 90,  "StorageClass": "GLACIER_IR" },
        { "Days": 180, "StorageClass": "DEEP_ARCHIVE" }
      ],
      "Expiration": { "Days": 2555 }
    }
  ]
}

Two things to know before you ship a lifecycle policy:

Transitions cost money per request. Each lifecycle transition is a billed request, and the per-1,000 transition cost is higher for the colder classes (vendor: AWS S3 pricing — confirm the exact us-east-1 cents on the live page, the table is JS-rendered and these move). For a bucket of millions of tiny objects, the transition requests can cost more than the storage you save. This is the second reason small objects don't belong in IA — the move itself isn't worth it.
Expiration is the most underused line. If the data has a legal or practical end-of-life, set Expiration. Storage you delete is storage you stop paying for forever. Most teams tier data down and then keep it for eternity because nobody wrote the expiry rule.

Fifth, when access is unpredictable, use Intelligent-Tiering

Lifecycle policies assume you know the access pattern. When you don't — user uploads, a data lake, anything where some objects go cold and others stay hot unpredictably — S3 Intelligent-Tiering is the right default. It monitors each object and moves it between tiers automatically: after 30 consecutive days with no access it drops to Infrequent Access (about 40% cheaper), and after 90 days to Archive Instant Access (about 68% cheaper), with no retrieval fee when an object gets hot again and is read (AWS Intelligent-Tiering, accessed 2026-06-18).

The cost is a monitoring-and-automation charge of $0.0025 per 1,000 objects per month (vendor: AWS, us-east-1). That math has one sharp edge:

Intelligent-Tiering is wrong for billions of tiny objects. The monitoring fee is per object, not per GB. A bucket of a billion small objects pays a monitoring charge that can dwarf any tiering savings. Per AWS, objects smaller than 128 KB are never auto-tiered and are always billed at the Frequent Access rate — but they can still rack up monitoring charges. For huge counts of small objects, a plain lifecycle policy (or just Standard) beats Intelligent-Tiering.

Rule of thumb: unknown access pattern + reasonably sized objects → Intelligent-Tiering and forget it. Known pattern, or billions of tiny objects → explicit lifecycle policy.

Sixth, stop paying for retrieval and requests you didn't budget for

Storage is the line everyone watches. Requests and retrievals are the lines that ambush you.

Retrieval fees scale with how cold the class is. Standard-IA and Glacier Instant charge a per-GB read fee; Glacier Flexible and Deep Archive charge a per-GB retrieval fee plus a per-request fee, and Deep Archive's slowest retrieval tier is measured in hours, not seconds (vendor: AWS — confirm the exact per-GB cents and the retrieval-time SLA on the live S3 pricing and retrieval-options pages). The lesson: a class is only cheap if you read it as rarely as its design assumes. Putting frequently-read data in Glacier to save on storage and then paying retrieval on every read is the most expensive mistake in this whole document — the retrieval bill can exceed what Standard would have cost outright.
Request pricing punishes chatty workloads. S3 Standard GETs are cheap individually (~$0.0004 per 1,000) but a service doing millions of tiny GETs per minute turns "cheap" into a real line item. Batch, cache, and use CloudFront in front of read-heavy buckets so the requests never hit S3.

Before you tier anything down, ask the one question that governs the whole ladder: how often is this actually read? If you don't know, that's what Storage Class Analysis and Intelligent-Tiering are for. If you guess wrong toward "cold," the retrieval fees make you pay for the guess.

Seventh, the free win everyone leaves on the table: VPC Gateway Endpoints

If your EC2, ECS, or Lambda workloads in a VPC talk to S3, check how that traffic leaves the VPC. By default, instances in a private subnet reach S3 through a NAT Gateway — and NAT Gateway bills both an hourly charge and a per-GB data-processing charge on every byte. For a workload pulling terabytes from S3, that is a tax you are paying for nothing. (The full version of that problem is its own post: .)

An S3 Gateway VPC Endpoint routes that traffic privately, and AWS charges nothing for it — no hourly fee, no per-GB fee, and traffic to S3 in the same Region incurs no data transfer charge (AWS Gateway endpoints for S3, accessed 2026-06-18). You add a route, and the same S3 traffic that was flowing through a metered NAT Gateway now flows free.

One caveat worth stating honestly: Gateway endpoints work for traffic originating inside the VPC in the same Region. They do not serve on-premises networks, peered VPCs in other Regions, or transit-gateway paths — those need an Interface endpoint, which does cost money ($0.01/AZ/hour plus $0.01/GB). For the common case — instances in a VPC reading from S3 in the same Region — the Gateway endpoint is free and you should have created it on day one.

The order matters

Run it top to bottom: turn on Storage Lens, kill incomplete multipart uploads, right-size storage classes, automate with lifecycle (or Intelligent-Tiering when the pattern is unknown), respect the retrieval and request fees, and add the free VPC Gateway Endpoint. Every step is reversible, none of it touches your application code, and the whole thing is an afternoon's work for savings that compound every month the data sits there.

The reason this works is not cleverness. It is that S3's defaults are tuned for the most expensive, most available configuration, and almost nobody changes them. The money is sitting in plain sight. You just have to run the checklist.

Social Doors (publish after blog is live)

Most S3 bills are wrong, and the fix takes an afternoon — not an architecture rewrite.

S3's defaults put everything in the most expensive class ($0.023/GB-month), nobody sets a lifecycle policy, and incomplete multipart uploads quietly bill you for storage you can't even see in the console.

I wrote down the exact order I run on every account:

Turn on S3 Storage Lens (free tier) before touching anything — you can't optimize what you can't see.
Kill incomplete multipart uploads with one lifecycle rule. This is the silent leak nobody finds on their own — I've seen it run into terabytes.
Right-size storage classes — but mind the traps: a 10 KB object in Standard-IA gets billed at 128 KB, and short-lived data in Glacier pays a minimum-duration penalty. Small or short-lived data stays in Standard.
Automate transitions with lifecycle policies — and actually set Expiration, the most underused line in S3.
For unpredictable access, use Intelligent-Tiering and forget it.
Add a VPC Gateway Endpoint for S3 — it's free, and it stops NAT Gateway from taxing every byte your fleet reads.

The savings are almost never under 40% on the storage line. The exact number depends on your access pattern — but these mistakes are so common the question is usually how much, not whether.

Full playbook with the lifecycle JSON and the price table on the blog.

#awscostoptimization #aws #s3 #cloudcomputing #finops

(blog link in first comment)

X thread

1/ Most S3 bills are wrong, and the fix takes an afternoon — no architecture rewrite.

The defaults put everything in the most expensive class ($0.023/GB-month) and nobody changes them.

Here's the exact order I run on every account. 🧵

2/ First: turn on S3 Storage Lens (free tier, 62 metrics).

You can't optimize what you can't measure. It shows you which buckets are biggest, what class the data sits in, and where the incomplete multipart uploads are hiding.

3/ Kill incomplete multipart uploads.

Failed uploads leave orphaned parts in the bucket. You're billed for them. They don't show in the object listing. They pile up for years.

One lifecycle rule (AbortIncompleteMultipartUpload, 7 days) fixes it. No early-delete charge.

4/ Right-size storage classes. The ladder spans ~23x.

Standard $0.023 → Standard-IA $0.0125 → Glacier Instant $0.004 → Deep Archive ~$0.00099 per GB-month.

But mind the traps. 👇

5/ Trap 1: minimum object size. Standard-IA bills every object as min 128 KB. Move a bucket of 10 KB thumbnails there and you pay for 12x the bytes. Small objects stay in Standard.

Trap 2: minimum duration. Delete IA data before 30 days, you still pay 30. Glacier = 90, Deep = 180.

6/ Automate with lifecycle policies — and set Expiration. Storage you delete is storage you stop paying for forever. Most teams tier down and then keep data for eternity.

Unknown access pattern? Use Intelligent-Tiering and forget it.

7/ The free win nobody takes: an S3 Gateway VPC Endpoint.

By default your private instances reach S3 through a metered NAT Gateway. The Gateway Endpoint routes the same traffic privately for $0 — no hourly fee, no per-GB fee, same Region.

8/ Run it top to bottom: Storage Lens → kill MPUs → right-size classes → lifecycle/Intelligent-Tiering → respect retrieval fees → free Gateway Endpoint.

Reversible. Touches no app code. Savings compound every month.

Full playbook → [blog link]

Facebook

Skip — purely technical (S3 storage classes, lifecycle JSON, VPC endpoints). No Facebook audience fit per the voice rules. Pure technical posts skip Facebook.

Reddit (r/aws)

Title: The order I run S3 cost optimization in — Storage Lens, incomplete MPU cleanup, storage classes, lifecycle, Gateway endpoints

Not a promo post — this is the checklist I actually run, in the order I run it, because the order matters.

Storage Lens first (free tier). 62 bucket-level metrics, 14 days history, and it surfaces "incomplete multipart upload bytes >7 days old" out of the box. You can't right-size what you can't see.
Kill incomplete multipart uploads. This is the one nobody finds on their own. Failed multipart uploads leave orphaned parts that you're billed for and that don't show in the object listing. One lifecycle rule with AbortIncompleteMultipartUpload: DaysAfterInitiation 7. Per AWS docs, removing them via lifecycle doesn't trigger early-delete charges.
Right-size storage classes, but watch two traps. Standard-IA / One Zone-IA / Glacier Instant bill a 128 KB minimum object size — a bucket of tiny objects gets billed at 12x its real bytes. And minimum durations (IA 30d, Glacier 90d, Deep 180d) mean short-lived data costs more in a colder class. Small or short-lived data stays in Standard.
Lifecycle policies for known patterns; Intelligent-Tiering for unknown ones. IT's monitoring fee is per object, so it's wrong for billions of tiny objects but right for unpredictable access. Don't forget Expiration — most teams tier down and then keep everything forever.
Mind retrieval + request fees. Putting frequently-read data in Glacier to save storage, then paying retrieval on every read, can cost more than Standard outright.
S3 Gateway VPC Endpoint is free. If your VPC workloads reach S3 through a NAT Gateway, you're paying NAT data-processing on every byte. Gateway endpoint routes it privately for $0 (same Region). Caveat: doesn't cover on-prem / cross-Region / TGW — that needs the paid Interface endpoint.

All prices US East (N. Virginia), verify against the live pricing page since AWS changes them. I wrote the full version with the lifecycle JSON and the price table on my blog if useful: [blog link]

Nostr (kind:1)

most s3 bills are wrong and the fix is an afternoon, not an architecture rewrite.

the data sits in the most expensive class ($0.023/GB-month), nobody set a lifecycle policy, and incomplete multipart uploads are billing you for storage you can't even see in the console.

the order: storage lens first → kill incomplete MPUs → right-size classes (mind the 128KB min + min-duration traps) → lifecycle or intelligent-tiering → add the free s3 gateway VPC endpoint so NAT stops taxing every byte.

full playbook: [blog link]

#aws #s3 #cloud #devops

Image concepts

Hero — Cost Comparison Chart (template 5). Horizontal bars showing the S3 storage-class ladder by $/GB-month: S3 Standard $0.023 (red, longest bar) → Standard-IA $0.0125 (amber) → Glacier Instant $0.004 (amber-green) → Glacier Flexible $0.0036 (green) → Deep Archive $0.00099 (shortest green bar). Title "S3 Storage Classes: a 23x range" in Deep Blue (#1a5276). Green savings callout arrow bottom-right: "Same data. Right class. ~23x cheaper." Footer rajesh.medampudi.com / s3-cost-optimization-playbook bottom-right, mono gray. Canvas #dbeafe. Personal-brand only — no other-brand tokens.

LinkedIn — Priority Pyramid (template 6). The run-in-order checklist as a pyramid, bottom (start here, green) to top (gold). Bottom tier green #047857: "Storage Lens + kill incomplete MPUs (Day 1)". Middle tier blue #2980B9: "Right-size classes + lifecycle (Week 1)". Top tier gold #E39D22: "Intelligent-Tiering + Gateway Endpoint (set + forget)". "↑ Start here" label at base. Title "S3 cost wins, in order". Footer rajesh.medampudi.com. Canvas #dbeafe.

X thread — Summary-of-Points Visual (template 1). The two traps as red bars: "Standard-IA bills 128 KB minimum → tiny objects cost 12x" and "Min duration: IA 30d / Glacier 90d / Deep 180d → short-lived data costs MORE". Footer callout box (#fef3c7): "Small or short-lived data stays in Standard." Footer signature rajesh.medampudi.com. Canvas #dbeafe.

Reddit — reuse hero (the cost-comparison chart) since r/aws renders Blossom image URLs inline and the ladder is the core argument.

Nostr — reuse hero (Blossom PNG URL embedded in the kind:1 note).

All images: Virgil font (fontFamily 1), roughness 1 on content / 0 on background, hatched fills, 1200×675, exported .excalidraw + .svg + .png (2x). Run node ~/personal_bot/guardrails/check-image-brand.mjs <file.excalidraw> <file.svg> before attaching — must exit 0. Personal-brand assets only; no other-brand tokens anywhere.

Sources

AWS — Object Storage Classes (storage prices, min sizes): aws.amazon.com/s3/storage-classes — accessed 2026-06-18
AWS — Understanding S3 Glacier storage classes (min durations 90/90/180): docs.aws.amazon.com/AmazonS3/latest/userguide/glacier-storage-classes.html — accessed 2026-06-18
AWS — Understanding and managing S3 storage classes: docs.aws.amazon.com/AmazonS3/latest/userguide/storage-class-intro.html — accessed 2026-06-18
AWS — S3 Intelligent-Tiering (30d→IA 40%, 90d→Archive Instant 68%, no retrieval fee, 128 KB rule): aws.amazon.com/s3/storage-classes/intelligent-tiering — accessed 2026-06-18
AWS — How S3 Intelligent-Tiering works: docs.aws.amazon.com/AmazonS3/latest/userguide/intelligent-tiering-overview.html — accessed 2026-06-18
AWS — Configuring lifecycle to delete incomplete multipart uploads (AbortIncompleteMultipartUpload, 7 days, no early-delete charge): docs.aws.amazon.com/AmazonS3/latest/userguide/mpu-abort-incomplete-mpu-lifecycle-config.html — accessed 2026-06-18
AWS — S3 Storage Lens metrics & recommendations (62 free metrics, 14 days, incomplete MPU cost metric): docs.aws.amazon.com/AmazonS3/latest/userguide/storage_lens_basics_metrics_recommendations.html — accessed 2026-06-18
AWS — Gateway endpoints for Amazon S3 (no charge, same-Region no data-transfer charge, caveats): docs.aws.amazon.com/vpc/latest/privatelink/vpc-endpoints-s3.html — accessed 2026-06-18
AWS — S3 pricing page (storage / request / retrieval tables; live source of truth): aws.amazon.com/s3/pricing — accessed 2026-06-18 (tables JS-rendered; per-request and per-GB retrieval figures are stated as vendor: AWS and should be re-confirmed against the live page at publish time)
AWS — VPC pricing (Interface endpoint $0.01/AZ/hr + $0.01/GB): aws.amazon.com/vpc/pricing — accessed 2026-06-18

Editorial notes (resolved pre-publish)

These are resolution notes, not open items — every placeholder figure below has been either anchored to a real source, hedged as a vendor figure, or cut. No private figure was fabricated.

Request / lifecycle-transition pricing. The precise per-1,000 cents were sourced from a secondary mirror and could not be firmly confirmed against the JS-rendered live AWS table, so the specific numbers were removed and the text now states only the directional fact (colder classes cost more per transition) as a vendor: AWS figure. Re-confirm exact cents on the live S3 pricing page at publish time.
Retrieval fees. Same treatment — the per-GB cents were cut and the body now hedges retrieval cost as vendor: AWS with the directional rule (colder = higher read fee; Glacier Flexible/Deep Archive add a per-request fee).
Intelligent-Tiering monitoring charge — $0.0025 per 1,000 objects/month. Confirmed against the live AWS S3 pricing page (us-east-1); stated inline as a vendor: AWS figure.
One Zone-IA (~$0.01/GB-month) and Deep Archive ($0.00099/GB-month). Deep Archive $0.00099 confirmed against the live AWS page; One Zone-IA kept with its ~ hedge.
Personal result anchored. The 96% cold-storage reduction is now stated in the intro and is backed by Rajesh's own observability-platform doc (career_transition/docs/mlt.md — logs aged past 90 days into Glacier Deep Archive, $0.00099/GB vs $0.023/GB Standard). It is presented as the storage-line arithmetic it is, not a measured whole-bill cut.
Deep Archive retrieval time. The exact "up to 12 hours" SLA wording was softened to "measured in hours, not seconds" pending confirmation of the current AWS retrieval-options SLA.