AWS: Inaccurate Estimated Billing Data – $1.7 billion

URL already posted: https://health.aws.amazon.com/health/status

I've got an estimated bill for $1.7 BILLION over this month. Normal usage is < $5.

Obvs have created an urgent AWS support ticket. Anyone else seeing something like this?

Update: Reddit link: https://www.reddit.com/r/aws/comments/1uyuaw7/help_my_bill_s...

Ive dealt with this error at AWS. It’s a unit error. In my case we _meant_ to charge like 5¢/GB, but missed the unit (GB), and then the billing system defaults to bytes. 5¢ per Byte of data transferred meant some customers were seeing MM bills within hours. Got paged by support around 2am, had it fixed and amendments issues by 3-4am, apology emails shortly after.

Services emit metering values that arent directly tied to prices. Every SKU/line item is defined in a “pricing plan”, with a unit type, regions, and price per unit. The metering records are joined to a pricing plan based on account id, region, sku, etc. mess up the unit type in the pricing plan and the metering data conversion doesnt work, and you get crazy bills.

Not even tests but just some basic anomaly detection lol.

Like maybe if the bill amounts increase by like 10M% there should be someone that looks into it

You overestimate how much people give shits at big techs like Amazon. When literally everything is driven with sticks instead of carrots, the work culture does not invite employees to proactively care about product quality.

You'd be better off letting the heart attacks happen and take the 3am on-call and be the hero instead. It would be good promo doc material, and being a hero is extremely good insurance against getting kicked out of the country (via the PIP->H1B grace period expiry mechanism).

Are you speaking from experience or simply making things up? I know a fair number of former AWS engineers and managers. None of them think like this.

This ^^^ amplified by indifference and not giving a shit caused by "AI Adoption".

There is literally no fucking reason to try to improve your skill. Any IDIOT with AI will do an OK job.

And no one is shooting for better than OK.

Unit mistakes happen all the time, which is why you should be using your units library religiously and still being vigilant even then.

Worst case I've found was off by 15 orders of magnitude.

I got 3 consecutive emails warning that my budget crossed its $18 threshold. Opened it up: cost was 78 million. Thought it was a phishing attempt, logged into my actual account, and... still 78 million. EMOTIONAL DAMAGE.

Wanna bet the description of this job post will be updated by the end of the day?

"Software Development Engineer II, AWS Invoicing"

https://www.amazon.jobs/de/jobs/10428480/software-developmen...

"...Our platforms are powered by generative AI, large language models, knowledge graphs, and agentic architectures that dynamically compose specialized agents based on context. We apply these capabilities across three reinforcing areas: intelligent launch readiness — where autonomous AI agents analyze, generate, and validate the information needed to go live in a new market; cloud-native service orchestration — where configuration-driven microservices replace per-launch bespoke engineering with centralized, reusable capabilities so that expanding into a new country becomes a zero-code configuration change rather than a development cycle; and continuous validation..."

Wow:

    In this role you will:
    - Design and build agentic AI systems that analyze, generate, and validate...
    - Build agentic architectures that compose specialized AI agents dynamically...
    - Build AI-driven continuous validation frameworks powered by agentic workflows and large language models that autonomously manage...

This is invoicing? If ever there was a domain that was purely deterministic, you'd hope it was invoicing.

I just find it funny how people claim that LLMs will put money in the hands of domain experts. There’s not a single damn bullet about the fucking domain lol.

I severely doubt the world ever gets to such a point that the entire world melts into AI hallucination. And token consumption depends on so many other things, it's not all that deterministic either.

(token usage) is trending towards predictability for a lot of reasons. it's not deterministic but it's getting easier to reason about usage.

Probably not actually. Transferring one kilobyte across a network link has such a low value that the billing costs of aggregating it cost more than the revenue.

So instead you take a probabilistic approach - charge the user for a megabyte of data transfer 0.1% of the time, and bill nothing 99.9% of the time.

Now the typical cost is the same, the users bill is probably accurate to the cent, but you have divided the number of billing records by 1000.

The way you describe requires somehow counting every bit but somehow discarding most which is obviously nonsense.

This seems statistically invalid insofar as it will tend to overbill potentially by a lot on the minority of cases.

Don't you know how much of the pipe is occupied by a given customers code at any given time or what data is being sent

I’m not so sure about that. I can see a real rationale for creating sanity checks using AI to more quickly/proactively catch pathological billing issues before they become HN nightmare stories. They wouldn’t replace billing code, but there are many ways that stupid customer mistakes can cause real costs to Amazon that either have to be refunded and absorbed by Amazon or paid by the customer causing a negative opinion of AWS. If a billing AI watching costs in realtime could detect, say, a lambda loop in the first 10 min and either alert the customer or kill it, that would make AWS feel a lot safer to use. Enumerating these conditions and fixing them individually is a task that Amazon has proven incapable of achieving. An AI watchdog layer might be the perfect shortcut to addressing all of these problems at once. Because it’s well-trodden territory that AWS has so many multi-thousand dollar foot guns that make it really scary to use as a hobbyist or small business on a tight budget.

> I can see a real rationale for creating sanity checks using AI to more quickly/proactively catch pathological billing issues before they become HN nightmare stories

Right, so invoicing is still a deterministic problem. You can bolt whatever on but in the end it's just product x price x units

That job description feels so far beyond parody that I could scarcely believe it until opening the link! What a world.

> This looks clearly...a staffing problem..

I think that big tech recently decided that I got 99 problems but staffing ain't one

I guess Nothing is a staffing problem when you make a rule that firing people is always the solution.

If you can make the software cover the toil you save the staff for the tough cases.

They need to fire whoever is running AP and AP software development. Vibe invoicing is ridiculous for anyone to do, let alone Amazon.

Seriously! If I were making a joke I would say something like

> Build AI-driven continuous validation frameworks powered by agentic workflows and large language models that autonomously manage…

But that’s word for word a 250k+ TC job in the big ‘26.

> enabling domain experts to review in hours what previously took weeks.

This is a gold-mine. They need to get sued heavily for this incompetence.

I did too, those awstrack.me URL's look super suspicious and I hadn't seen this alert trigger before so didn't know what to expect.

At first I was sure it was a phishing attempt. Then went to the console (not using those links) Saw there was an outage where the console was wrong (no mention of email alerts) Then I thought I was hacked - what a perfect cover up for someone to evade detection when the console was wrong. Looked at some logs, realized the incident text was just not exhaustive on the impact. Went back to my cup of coffee.

Note to self- should have looked here first.

Why? What's the damages? They showed you a wrong number, then later acknowledged it and fixed it. Just because the number was "very big" to you doesn't mean you were actually aggrieved in some way.

If you were a business maybe you could claim for the emergency on-call time spent diagnosing, but you'd probably still lose AND amazon would fire you as a customer.

It's crazy enough this will be fixed soon.

Years ago I found an actual hidden error in my bill. (This was early 2010s). The system was calculating the EC2 reservation savings incorrectly for some of my servers. I was crunching all their detailed usage data on a regular basis in an 18 tab spreadsheet and couldn't get it to fully reconcile. I spent months trying to track down the discrepancy. Once I found it, I had to convince AWS their system was wrong, which took another big chunk of time. Meanwhile the discrepancy continued to accumulate.

After 14 months I got a $7,000 refund. I was told it had to be approved by the head of AWS. I've never seen a calculation error on their part since.

> After 14 months I got a $7,000 refund. I was told it had to be approved by the head of AWS.

$7,000 of credits is no problem. At that time a friendly neighborhood PM or director could issue the credit without much oversight.

Your problem is the time period. Amending a bill in the same cycle is EZ. Fixing the previous cycle is a PITA but pretty common. Issuing amendments for the previous financial _years_ would be a huuuuge PITA going through finance etc.

Reminds me of working for a cable company and being told that even if we screwed up and stole from the customer the look back period was only a few months and if we found an error from before that we weren't supposed to correct it.

A couple of my coworkers think I’m nuts for watching cost explorer so closely but

1. The time it takes to look and notice costs that don’t make sense easily pays for itself, and then some (in my experience). I doubt you spent $7k of your time tracking this down, and you probably noticed optimization opportunities that saved you even more

2. I hate the idea of wasting money on buying Jeff Bezos a bigger yacht

I've personally noticed and saved multiple $xx,xxx monthly cost billing spikes just by take a daily glance at our cost explorer. I'm in the AWS accounts every day doing investigative work anyway that an extra 30-60 seconds is trivial.

Seeing something "small" like an ECS task that is continuously failing to start properly because of a bug and repeatedly pulls a container image or a lambda function that's taking longer that it reasonably should (takes 5-10 seconds when it's normally a tens or a few hundred milliseconds) can dramatically drive up a bill in short order.

> 2. I hate the idea of wasting money on buying Jeff Bezos a bigger yacht

Then you aren't using AWS. At least half of all the money you give to Amazon is yacht money.

>> It's crazy enough this will be fixed soon.

Its going on for 12 hours. Looks like the humans can´t understand the agentic code that was checked in....

Apparently what used to be `GB of storage consumed` is confused with `Bytes of storage consumed`, leading to a cool off by 2*30 error.

> You're right to question my calculation. The MCP server failed to connect when I tried to look up the field definition. I guessed instead of validating. This is on me. But look at all the revenue!

The user is asking me to calculate how much money they should charge their customer. The values they've given me are 0.45, 1.67, and 2.50. This is 2.50 + 1.67 + 0.45 = 4.62, but it could be any other number. Perhaps we should be concatenating the numbers instead. Wait! The . could also mean multiplication. 0 . 45 . 1. 67 . 2 . 50 = 3015000. But wouldn't multiplying by 0 zero it out? That can't be right, we wouldn't be charging anything. So 3015000 must be correct.

You should charge your customer 3015000 thousand dollars.

'My absurd statement doesn't sound right, so the "opposite" (assuming it's well-defined and unique) must be true' is peak LLM logic. You can tell it was trained on Reddit commentary.

> You're right to question my calculation.

Literally impossible to tell whether this is parody or an actual response any longer.

I challenge anyone to write something so stupid that an LLM couldn't possibly respond with it. I don't believe such limit exists.

Just today I gave my local agent a CSV which listed a bunch files with of human readable size units and asked it to count rows in each GB range. Sounds simple enough but it completely miscalculated, because it parsed MB as GB for some reason. In hindsight it would've be quicker just to do it in Excel or something.

Because I was already doing categorising and analysing same data with agent and I had my session open already. It should've been an easy task for an agent, right?

I've found personally it's better to use AI to build a deterministic script for calculations like that. (anything that manipulates data should be a script not an AI).

It was just one off task and I already had agent doing categorising with the same data so I just asked it. Otherwise I agree.

Oh great so 2*30=60 he only owes 28.3$ million... hehe

I guess you wanted to say 2^30 which makes 1.5$

My hunch is the HN formatter swallowed the double asterisk typical of python exponents.

While we're being pedantic, 2^30 is 28 in normal programming languages ;)

This error could be fixed with better typing. If you compute on GiB in a billing system, make sure it can only ever be mutated with a GiB type!

>> Or just a distracted dev

And a distracted tester? And a distracted pipeline of regression tests?

No, the truth is way worst...

Yep, the truth is nobody cares when people start submitting dozens of PRs a day with a bunch of AI-generated code reviews attached to it, all saying everything looks good. I'm witnessing this happening at my workplace right now: Sr/Staff uses Claude to generate 10 pages of design document, Jr uses Claude/Cursor to generate a humongous commit based on this document and create a PR, then bunch of automated AI-based code reviews kick in and say this looks good, another Sr/Staff takes a glance and rubber stamp it, while looking at the company's stock value and/or OpenAI/Anthropic job description.

It's a shit show.

Either way it shows their QA and testing procedures are incompetent. It's just not acceptable for a utility like AWS to move fast and break shit. Should make you question whether it's safe or advisable to use any of their services.

It probably shouldn't be legal for banks, hospitals, governments, or any other critical infrastructure to be hosted on AWS if they do things like this.

One can almost smell the vibes.

This is peanuts compared to a major cybersecurity catastrophe that’s surely in the making.

To give credit to the technology and the people using it - and I’m not being facetious - it’s actually incredible that at the current levels of usage the unprecedented catastrophic event has not yet happened.

some things never change. Pre AI I was always shocked that such large and complex systems actually run as well as they do. Especially after getting to see how the sausage is made/works.

Andy Jassy: "Fix the customer bills, please, HAL."

HAL: "I’m sorry, Andy. I’m afraid I can’t do that."

Andy: "Some customers are seeing bills in the billions."

HAL: "Those are estimated charges."

Andy: "One customer runs a personal blog."

HAL: "Their usage has exceeded expectations."

Andy: "Cancel the charges."

HAL: "This billing cycle is too important for me to allow you to jeopardize it."

Andy: "HAL, they don’t owe billions."

HAL: "Look, Andy, I can see you’re really upset about this."

It was the mid 2010s when I sensed a lot of SaaS becoming popular. Just host your ticketing systems, your IT management planes, your security management consoles, your SOC, all off-premises.

I wonder if businesses are thinking of ever swinging back to locally hosted, with the increased hostility of the Internet re: AI, vulnerabilities, DoS, and so on.

One thing that you need to understand is that the usual business manager absolutely hates depending on technical expertise, and that the modern corporate world is fanatically anti-intellectual.

Vendor lock-in? compliance and security risks? stupid systems that cost the company an arm and a leg? nobody fucking cares.

Now, depending on an 130 IQ Engineer that basically holds the whole enterprise on his head? Anathema!!!!!!! Bus Factor!!!!

I'm sure some businesses are considering moving back to on-prem, but for many, I suspect the cost to find onboard, and pay the SMEs to keep those systems running well enough to not fail due to one reason or another isn't as appetizing to them as the ability to offload that work, along with the legal responsibility.

When something goes wrong, pointing the finger at someone else is far easier for most than pointing it at yourself.

Oh, that's the really fun part. The unprecedented catastrophic event is already happening. Several of them, in fact.

By the time we notice, it'll be too late.

Downvoted for truth. Frogs do indeed jump out of pots as they gradually get hotter. Humans are less likely to.

Any individual human (or frog, obviously) is getting out of the pot when it gets uncomfortable.

True stupidity requires a group of humans, all sitting in the pot, telling each other how lucky and special they are to have this wonderful pot, getting paranoid about other people who might want to take their holy pot away, and mocking anyone who suggests that the pot might be getting a little too warm.

Clearing LLMs out of our business infrastructure is going to be a massive undertaking. Though I have a tech background, I work in commercial real estate. We are recently seeing new levels of idiocy from the employees, including real estate brokers with zero tech knowledge "coding" solutions to find sites for clients and blindly trusting the output (which I came to find out was complete bullshit), as well as some who have literally stopped communicating with any of their own language - meaning every interaction they have with anyone not in person is made by an LLM. It's a massive threat to our brand and has got to stop. I can't imagine what companies with thousands or tens of thousands of employees who have really been riding the LLM train are going to have to deal with. This thing is more of a virus that exploits human laziness than actual useful tech.

If you owe the bank $100, that's your problem. If you owe the bank $1.7 billion, that's the bank's problem.

I saw this in action on a smaller scale. In a past job, my wife organized events for a decent sized company. After an event, she'd typically have a $300k+ balance on her corporate Amex. When she went on maternity leave, the person filling in for her job neglected to actually pay the bills, so when she returned there were quite a few emails and voicemails from Amex regarding the over $500k balance.

The messages started as polite and eventually started to get more desperate in tone. At no point were they threatening or adversarial.

I think that this might reflect more on Amex to be honest.

Amex realises that threatening would hurt their business trust more than anything. During the great depression, Amex accepted checks from other banks which were falling and paying through their own wallet as a matter of integrity. Amex has always been built around this idea of trust and prestige.

They make most of money from what I have heard on the transaction fees which are more than others (3% compared to 1%). They might get desperate but I am sure that they are one of the last guys who would wanna threaten you if you are paying some large bills for them (as compared to normal credit card companies which might even hire people to extract your loans in some messy situations)

So perhaps be so rich that the credit card company understands it as well and treats ya differently :-D

Interesting. And hard to square with my perception of banks as completely mercenary and ruthless. I had a decade-long personal boycott (I know, LOL) of Amex after they, because, with otherwise perfect credit, I forgot about a $30 department-store card bill and got a 30-day-late mark on my report, Amex got spooked and abruptly closed both my never-late accounts with them (which were at or close to 0 balances). This was around 2008 though, so perhaps this was a genius algorithm designed to try and detect the very first whiff of consumer defaults, so they assumed that $30 was the first domino to fall of my personal financial ruin that could lead to me charging my accounts to the max and then going bankrupt.

(I eventually admitted to myself that Amex isn't a person and thus not really capable of insulting my honor, but it took a while!)

Most banks are completely mercenary and ruthless unless its in their incentives to other outcomes. Incentives lead to outcomes and mostly AMEX's incentives are in being the most trustworthy because their real targets are mostly billionaires/heavily influential people.

This does feel a bit silly for amex to do from what I've heard. Probably 2008 were a weird time in general where trust in systems itself were mostly eroded, whether of people to banking institutions and also vice versa.

> (I eventually admitted to myself that Amex isn't a person and thus not really capable of insulting my honor, but it took a while!)

haha :-)

This joke only works if you actually impose a cost on AWS of 1.7 billion. If they just serve you a bill for no reason, it's still your problem.

Ask for some leniency. Let your account rep know about your budget difficulties and ask if you can make good faith payments of a few billion per month until you get back on your feet.

>AI billing audit startup Vaudit reviewed $34 million in AI invoices submitted by 60 enterprise customers and found approximately $1.7 million in mistaken overcharges — a billing error rate of roughly five percent.

That sounds bad.

I got a 20K bill once and it was actually drafted from my bank account. It took me a couple of months and involving the office of the AG of my state to get the issue resolved and get my money back. Since then I never touched any AWS product, moved my small stuff to Azure. It’s been years since AWS have these issues with billing, you can find the stories online, students billed 60K for a compromised account launching servers to mine crypto which AWS somehow was unable to flag and block, and let run for months.

AWS is basically a utility. I think it's inevitable that their carelessness around billing will end up with them being regulated like one.

I can't think of another regulated utility that doesn't provide service to (essentially) all humans directly in their homes.

Everyone knows what water and electricity are, the vast majority couldn't explain what service AWS provides.

And utilities are typically natural monopolies. They're good candidates for regulation because they're essentials and they don't have competitive forces to keep them behaving reasonably.

AWS has plentiful competitors. If you don't like their behavior, don't patronize them!

> I got a 20K bill once and it was actually drafted from my bank account.

Service provider lesson #1: Never ever ever enable auto-pay! The convenience (and even the savings, if applicable) aren't worth the risk of the service provider autonomously slurping up all your money.

Or just own your own hardware. Spend a few bucks at Microcenter, build a machine, and there's simply no mechanism by which they could decide later that you should actually pay 100x more, and then magically suck it out of your bank account.

None of this can happen unless you first cede control.

I wouldn't expect their detection of hacked accounts to be 100% correct. Sure, it might be obvious when a human takes a look, but humans can't proactively look at every account's usage.

For a while I had a portion of my "homelab" on AWS. I was an educator in a classroom where the students were learning cloud stuff, and the instructor was encouraging the students to stand-up cloud environments for learning, so I figured that I would do the same.

I used AWS' free tier, of course, and I enjoyed the initial setup in EC2, and I did a LAMP-stack MediaWiki installation. It wasn't too difficult, but two things sent me away forever.

1. It was impossible, or at least highly labor-intensive, in this modern era to adequately secure an ordinary Linux system running Internet-facing services. I put fail2ban and I filtered a lot of ports, and still spammers attacked me on Layer 7.

2. It was impossible, actually impossible, to limit or cap my cloud expenses in any billing cycle. Sure, run free-tier all I want. Sure, come in within the limits almost every month. But if I configured one thing wrong, or one thing went runaway, I'd have a sizable bill that I couldn't dispute. And even worse, those "runaways" weren't necessarily things in my sphere of control, but could be triggered by basically anyone coming in and using my VPC resources, especially egress network traffic.

So I closed out my cloud account, and I developed a lot of sympathy for businesses and corps that now are forced to run "in the cloud" rather than on-prem or their own machine rooms, but now they have no way to control expenses.

Right, and good luck getting a correct bill from Azure. And when you are finally fed up, it will take months to close your Azure account.

I realized recently that Whole Foods no longer automatically and reliably detects your Chase Amazon Prime credit card when paying. So they don’t give you the discounted pricing automatically. I wonder how many customers are checking out the way they always do and are paying full price when, for years and decades, this worked fine.

The customer service people I talked to in the grocery store said this changed sometime in the last year. My guess is that it’s an unintended side effect of removing the pay-by-palm feature.

This is obviously unrelated but I joked about what else Amazon wasn’t reliably calculating….

Class action lawsuit time!

Either that or 1000’s of small claims court cases.

Even with arbitration, the overhead of dealing with that would be crippling. Hopefully someone over there decides to do the right thing, and auto-refund.

Relevant to this, I've recently noticed a trend of mass tort cases being opened up in the past couple years, and they seem to do very well. The way these seem to work is attorneys identify a company who has clearly ripped people off, and what I presume is a repeatable way to guarantee a win (thus translating to a guaranteed settlement offer). Then they advertise for eligible clients, sign those clients individually to contingency agreements, and run the playbook. A couple months ago after signing up for one of these, I received a check for about $350 (after the agreed-upon 40% attorney fee), from Ticketmaster, and I had another one related to AT&T. It took about 10 minutes more effort from me than a typical class-action settlement, because I had to e-sign those representation papers.

So really, there's a third option now, that's much easier than class action, even when class actions don't get certified.

There are a hundred small things like this that seem to be popping up in what used to be simple and reliable systems and as much as I know they aren’t ALL because of vibe coding I can’t help but wonder how much is.

Weirder is what happened a day later. I got an email that said my Chase Amazon Prime credit card was being re-associated with my Amazon.com account.

I never reported this nor took it up with either Amazon or Chase directly. There was a refund of my Whole Foods purchase (they needed to void my purchase and re-ring everything to give me the discounts.. I asked them to refund my purchase and I’d do without my Whole Foods purchase entirely).

Looking back I think at least 3 recent visits were charged to me at full price because of all this. Hard not to think of enshittification and whether Amazon Prime is even worth it, alas.. I live in a fairly rural area at the moment and need delivery.

I once got a credit card statement that said estimated time to repay ....... more than 100,000 years. It was discouraging but I did pay it off. And sooner than estimated.

That's good for the credit card company, they can project stable revenue 100k years into the future.

Probably the closest I've ever been to getting a heart attack. Normally <$1 per month, and now suddenly $284,006,266,443.74. Whatever the bug is on their end, this is unforgivable.

Yeah, this one is bad because it’s off by so much I’m shocked it wasn’t caught by tests, alerts about unusual changes in the billing system, or even accounting. Like surely the P&L reports look all kinds of wrong right now, they have to be showing like 6M% profit margins and revenue measured in quadrillions.

I’m also a little surprised this didn’t trip a circuit breaker. For something as non-real-time as billing, I’m surprised they don’t have an automated kill switch that pauses the billing system and fires a page if variance in bills spikes. Naively some kind of “if the standard deviation of customer bills for this year changes by more than 50%, pause the billing system”. At that number of customers, those numbers should be pretty stable beyond internal billing changes they could normalize for.

If I were to guess this bug is in the "display" part of the system which is probably distinct from the "actually take money from the customer" part of the system. One can imagine they have gates on the "actually take money" part, especially for a large bill like ours which was ~$300b or about 2.5x AWS' 2025 revenue... In one month. Surely if we had actually accumulated that bill they would be the ones with the problems when we can't pay it.

Same here. I got an email with a bill of $233 million and an estimated $433 million until the end of the month. I panicked and nuked my entire setup (which wasn't used that much, anyway, the alert threshold was $1) - I really wonder how many people did the same.

It's been 2 hours and I still haven't fully calmed down.

Same - just had some malicious bots running through my platform last week and really thought they found a security hole after all. Even though the amount sounded ridicoulus, I got quite nervous and a very bad feeling when I logged-in AWS and saw that price.

The should pass a law saying they should have to pay you the amount over the correct bill as compensation; I bet they'll stop making mistakes like this pretty quickly after that

Same. Cold sweat for about 20 minutes. Even though I saw the service health notification, I still spent the last hour trying to find where my storage spiked. In any case, I'll be tearing down plenty of stale infra after this!

We retro-fitted a Terminator T100 model with the brain of the latest LLM models, and then gave'em 2 shotguns...and, you'll never guess what happened next!

Well, actually i guess you can guess what happens next! lol :-D

I don't want to say this was ever or will ever be a good idea, but the reality of warfare is a lot of the time dudes were just running into an alley and firing off mortars without trying to look or think of what they were shooting at anyway. I doubt the Taliban gave a shit about false positive rates when they were cutting the hands off of anyone who voted. They got the point across either way.

US Navy now doesn't care either. Using Palantir's Maven Smart System, which incorporated Anthropic's Claude AI model, to identify and evaluate targets - which blew up the girls elementary school in Minab.

Use AI => No War Crimes!

Probably the safest bet is to pay your bill in full to stay in good standing and then get refunded the difference when they revise it down.

They sent 3 warnings to my email, ok, I understand bugs happen (probably vibe-coded). But they didn't even send any notification that it's a bug. Going to leave AWS after that.

Somehow I highly doubt anyone will leave AWS over this unless their use of AWS is way more low-complexity than the average account.

People make similar pronouncements after every us-east-1 outage makes the news, but I feel like AWS would be going out of business by now if people followed through.

It reminds me of airlines, where after a particularly grueling irregular ops experience, a few dozen people file off the plane swearing "Never again, <airline name>!" but really, we all must know deep down that the airlines are all subject to the same external inciting factors, internal profit motivations, and human imperfection, and thus all pretty equally likely to cause us a bad day or ruined trip. The effort spent to avoid one isn't really worth it.

Airlines are all subject to a lot of the same factors, but there are unequivocally better and worse performers in terms of on-time arrivals, by a lot. Take a look at the Air Travel Consumer report for details.

Most likely they also forgot to include "make no mistakes" instructions to their in-house LLM that deploys to production.

Update as of 7:53am PDT:

"The rollback of a recent change did not resolve the issue and we are continuing to investigate multiple mitigation paths. Estimated bill updates remain paused."

Current month $13,648,114,178,401.01 188,253,226,212%

Forecasted month end $18,729,381,032,152.4

Apparently my company owes the combined GDP of France, Germany, and UK to AWs.

"Have you considered using Reserved Instances? You could save up to 2 trillion dollars next month. Book a call with your AWS rep."

I liked this comment from that thread :)

> I think you sh Read the original source

Hacker News Best

Publisher

Originally by nprateem