return to table of content

Alternative clouds are booming as companies seek cheaper access to GPUs

apitman
181 replies
6d

Over the weekend a coworker was helping someone in our lab prepare data for an important conference talk.

We accidentally ran up a rather large bill because while the EFS storage pricing was simple enough, the usage pricing bit us.

It seems like AWS' entire business model is making the pricing so confusing that you don't know what it will cost until after you've used it. It feels weirdly similar to the US healthcare/insurance situation.

More competition in this space can only be a good thing.

ryandrake
85 replies
6d

It seems like AWS' entire business model is making the pricing so confusing that you don't know what it will cost until after you've used it. It feels weirdly similar to the US healthcare/insurance situation.

Not just the healthcare situation, but everything. There's nothing much more stereotypically American than "not knowing what you're going to pay for something until you're billed." Dozens of little fees on your cable or ISP bill, resort fees in hotels, service charges on your restaurant bill, fees on car rentals, fees from your bank when you so much as breathe on your account, and of course sales taxes which for some reason are never listed on the tag in any store.

doctorpangloss
41 replies
5d23h

People complain about this relentlessly, but never change.

Additionally, every single business owner I know complains that whatever it is they're selling, (1) it isn't worth the brain damage to sell to customers looking for the lowest price, (2) as long as people comparison shop in a harebrained way, hook pricing (aka up front pricing that looks low and turns out high) is only rational. It's not like they're providing a bad service for the cost.

thfuran
26 replies
5d23h

hook pricing (aka up front pricing that looks low and turns out high) is only rational.

Yes, many deceptive business practices are rational acts on the part of the businesses. That doesn't mean they should be tolerated.

wongarsu
18 replies
5d23h

Which is exactly why they should be banned to keep the market overall healthy. Preventing deceptive practices in a market economy this is a prime example of useful government "intervention", just like supporting contract enforcement or preventing theft

AnthonyMouse
17 replies
5d21h

They can always justify it as not knowing the price.

The sales tax rate is different in different places. It costs a different amount to ship to Florida than Alaska and picking it up at the factory is free (even if nobody does). The advertised price is if you have your own modem, renting one from the cable company is more.

None of these are inherently wrong. You should be paying more if you're having it shipped to a remote location with high shipping costs, and the cost of that shouldn't be dumped on every other customer. But it's kind of a loophole if you want the advertised price to be lower than what people are actually going to be paying in practice.

wongarsu
14 replies
5d21h

There are some cases where not knowing the price is reasonable. As you say, shipping can often only be calculated if you know all items and their destination. If you advertise on national TV you can't name a price that includes sales price (unless the company eats the difference). But these could be treated as tightly regulated necessary evils, not as a justification that showing final prices is always impossible.

There is no reason you can't show final price including tax on the label in a physical shop. There is no reason why a restaurant should be able to charge a 20% service charge instead of increasing regular prices by 20%. If you are buying a concert ticket or airline ticket the displayed price should include all mandatory fees. They can upsell you on additional services, but they can't suddenly notice in the last checkout step that your price is higher because the website you are using is charging a fee; that fee was known to them at the beginning of the transaction and should have to be disclosed at that point in time at the latest. If you want to go even further you can also dictate that shipping and handling fees are only allowed to include reasonable costs of actual shipping and handling.

All of these are normal common-sense regulations in most first-world countries.

AnthonyMouse
13 replies
5d20h

There is no reason you can't show final price including tax on the label in a physical shop.

Sure there is. When the price label is affixed by the factory/warehouse then you would have to track where everything is going and be unable to share inventory. Also, if the sales tax rate changes then all the labels become wrong. These would ultimately increase costs for consumers.

Adding sales tax is also not at all misleading because the customer is not going to be surprised by it and there isn't going to be a competing merchant across the street who can avoid charging it.

There are also business customers with their own sales tax ID and they can buy things without paying sales tax when they're being incorporated into a product where they collect the sales tax themselves.

There is no reason why a restaurant should be able to charge a 20% service charge instead of increasing regular prices by 20%.

This is actually true. A mandatory undisclosed fee is BS. But it doesn't help much, because if they want to do it then they just make it "optional" where the way to avoid it is more of an inconvenience than paying the fee.

They can upsell you on additional services, but they can't suddenly notice in the last checkout step that your price is higher because the website you are using is charging a fee; that fee was known to them at the beginning of the transaction and should have to be disclosed at that point in time at the latest.

The last step is where you disclose your address. Before that they may not even know which country you're in, much less the city/state, and there are a thousand legitimate reasons to have different prices or fees in different jurisdictions.

If you want to go even further you can also dictate that shipping and handling fees are only allowed to include reasonable costs of actual shipping and handling.

That doesn't really help, they're typically charging the actual cost. They just don't include it in the advertised price because it makes you inclined to make the purchase online instead of saving $10 by picking it up for the same price but no shipping charge the next time you go to the competing local store.

There's a reason Amazon's major competitive advantage is free two day shipping, derived from having the scale to achieve low shipping costs themselves.

cogman10
12 replies
5d20h

When the price label is affixed by the factory/warehouse then you would have to track where everything is going and be unable to share inventory.

Price tags are pretty much universally handled at the stores. There are some goods where the price tag is attached, but that's more the exception and not the rule.

This is a solvable problem. So much so that if you've traveled in most nations you'll see that all prices include tax. Not including tax is a particularly weird aspect of US culture that simply doesn't exist in other nations, even those with a large amount of imported goods.

AnthonyMouse
10 replies
5d14h

Price tags are pretty much universally handled at the stores.

Advertising isn't. If the same ad is viewable by people in different states, what price are they supposed to put on it? Shouldn't this be the same price as what they show in the stores, or else people will have trouble knowing if the advertising was deceptive?

This is a solvable problem. So much so that if you've traveled in most nations you'll see that all prices include tax.

Don't most of those other nations have a uniform tax rate?

withinboredom
9 replies
5d12h

I have no idea what you are going on about. If you advertise 9.99, it’s 9.99. Sales tax isn’t THAT different between jurisdictions, so we are talking about the seller having to lose a small percentage of margin. Sales are mostly about moving items, so they are already losing huge margins at that point anyway.

Anyway, you can always advertise 9.99 + tax and show the actual price with tax in the store tags.

AnthonyMouse
8 replies
5d8h

Sales tax isn’t THAT different between jurisdictions, so we are talking about the seller having to lose a small percentage of margin.

California has a 7.25% sales tax. Oregon has a 0% sales tax. A 7.25% difference is larger than the entire net margin on many products.

Sales are mostly about moving items, so they are already losing huge margins at that point anyway.

It seems like we've now gone from "there is no reason they can't do this" to "the merchant can just eat the sales tax and go out of business"?

Anyway, you can always advertise 9.99 + tax and show the actual price with tax in the store tags.

Which is basically what they do now. You see the final price at the register before you pay. You also know that there is going to be sales tax, no one is being misled.

Putting a different price on the tags than there is on the ad would just confuse people and make it harder to tell when you're not getting the advertised price because you have to do fractional arithmetic on every soup can and candy bar to see if it matches.

withinboredom
7 replies
5d7h

Don’t advertise California prices in Oregon?

I was referring to city/county taxes, not state taxes. Tor example, a local city to me in the US had a total of 14% sales tax, while outside the city was 10%.

Anyway, a sale’s motivation is to move products off the shelf and make room for more profitable products; they’re already losing money.

AnthonyMouse
6 replies
4d8h

Don’t advertise California prices in Oregon?

California borders with Oregon. Radio waves cross state lines. People see highway billboards and then drive into another state. People use the internet and you don't know where they are.

Anyway, a sale’s motivation is to move products off the shelf and make room for more profitable products; they’re already losing money.

A sale's purpose is frequently to bring customers into the store to buy other products, or get customers to try a product for the first time to drive repeat business.

There is also plenty of advertising that isn't discounts. Maybe your product is great and you want to inform the customer that they can get something better than the competition for the same price. Maybe you're just publishing a comprehensive price list for all your products.

And in all cases, "losing money" is not a binary result. Losing a small amount, or not making as much as you wanted to, is very different from losing so much that you go out of business.

withinboredom
5 replies
4d7h

If businesses go out of business from paying taxes, maybe they shouldn’t be in business. This isn’t a problem in the EU where countries border other countries and sales tax varies from 6-19-24%… so clearly this is a delusional problem that doesn’t actually exist.

AnthonyMouse
4 replies
3d22h

If businesses go out of business from paying taxes, maybe they shouldn’t be in business.

This statement has always been BS. If you set the tax rate to a percentage of revenue which exceeds the business's margins, all businesses go out of business. "Maybe grocery stores shouldn't be in business" is so absurd that the statement seems designed to deliberately shut down reasoned debate by stunning people into silence or making them so angry they respond rashly.

And in this context it doesn't even apply. The issue here is that customers in Oregon are going to pay a lower price than customers in California because Oregon has lower sales tax and the tax is part of the price they pay. The business can't eat the tax in California so their only alternative would be to advertise the higher price. But if they have to advertise the higher price then many of them will just charge the higher price everywhere so all you're doing is screwing customers in the jurisdiction with lower taxes out of paying lower prices. This also reduces competition by making it harder for customers to compare prices, because a store that only operates in a state with lower taxes would then seem to have lower prices than one that operates in multiple states and has to advertise the highest price, even if it didn't charge you the highest price where you actually are.

This isn’t a problem in the EU where countries border other countries and sales tax varies from 6-19-24%

Or it is a problem and you're just eating the cost of getting screwed by the law while defending it.

withinboredom
3 replies
3d11h

If you set the tax rate to a percentage of revenue which exceeds the business's margins, all businesses go out of business.

Unless I misunderstand, isn't this EXACTLY what the US does to ensure certain businesses are or aren't profitable in their smaller jurisdictions? This is how they entice businesses to come to their cities/states and they even compete on giving bigger tax breaks. I assume they also do the exact opposite to keep out businesses they don't want.

The issue here is that customers in Oregon are going to pay a lower price than customers in California because Oregon has lower sales tax and the tax is part of the price they pay.

They're already doing this ... this just makes it so you can do simple addition in your head while you walk around the store instead of throwing in some multiplication -- and if you live in a state where certain kinds of items have different tax rates, knowing what those tax rates are.

This isn't rocket science, I don't understand why you are making it seem so complicated.

AnthonyMouse
2 replies
2d

Unless I misunderstand, isn't this EXACTLY what the US does to ensure certain businesses are or aren't profitable in their smaller jurisdictions?

Traditionally this is called fines rather than taxes. The ostensible purpose of general sales tax is not to bankrupt every grocery.

this just makes it so you can do simple addition in your head while you walk around the store instead of throwing in some multiplication

Why do you have to do any math in your head at all? The register will do it for you at the end.

withinboredom
1 replies
1d23h

Surely you’ve been poor with 40 bucks in your pocket and trying to get as many items as possible?

AnthonyMouse
0 replies
1d19h

When the prices are listed pre-tax then use the amount of money you have in pre-tax dollars. If you have $40 and the sales tax is 7% then use your phone to divide your $40 by 1.07 and that's how much you have to spend. You don't have to do the multiplication for each item individually and items with different tax rates are typically entirely different classes of product which are sold in different stores.

ryandrake
0 replies
5d18h

Yep, this is one of those "This problem can't be solved - Says the only nation that has this problem" situations.

thfuran
1 replies
4d23h

They can always justify it as not knowing the price.

You could as easily say that the marketing department not knowing the features of the product justifies them making whatever claim they want in an advertisement. Why should a company be permitted to advertise something they don't know to be true, let alone something they know to be false?

AnthonyMouse
0 replies
3d21h

It's not that they don't know what the price is, it's that the price is differs based on factors that aren't the same for every person viewing the advertisement. Meanwhile most of these factors (like the local sales tax rate) are known by the customer, so they already know what the final price will be given the base price on the ad.

thaumaturgy
5 replies
5d23h

I have begun to think of this as a "large population error", and I'm noticing it in a lot of different markets.

At small scales, annoying your customers is bad business. You only need to lose a few before it begins to hurt. Customer complaints are more likely to be a consideration in business decisions.

At larger scales, a business can begin to preferentially adopt practices intended to drive away some customers. Perhaps you don't want the "pathological" customers, to borrow one of Patrick McKenzie's terms. You can make more money with less effort by being more selective about your customers.

At extremely large scales, you largely stop thinking about groups of customers altogether. All of your decisions are driven by aggregates -- did all sales go up this quarter, or down? The effort required to do a deep dive into the behaviors and preferences of any individual market segment may not make sense anymore on a quarterly basis. At this scale, you might be able to afford to annoy tens of thousands of customers and still have a very nice graph next quarter.

So, when someone says a business practice shouldn't be "tolerated", that's a perfectly reasonable position, except it doesn't actually work for businesses operating at extremely large scales. It's too difficult for customers to organize a protest in a way that will influence that business's decision-making.

So much business has moved online in the last 20 years, while the US has leveled off at 80% urbanization over the same time period, along with more and more businesses congealing into BigCos, combined with the recent domination of private equity: lots and lots of things are now operating at a scale where customer concerns just aren't a part of the business model anymore.

Coffee shops, fast food, big-box retail, online retail, SaaS, PaaS: all of these can thrive while running on exorbitant pricing and abusive customer policies, because their volume of customers is so large that it's nearly impossible to be so bad that you'll piss off enough customers to impact your decision-making. (Unless you're Sony.)

s1artibartfast
2 replies
5d20h

The intuitive answer is competition, where customers move from user-hostile companies to new ones.

The problem in my mind is that the largest companies have too much efficiencies of scale to compete with on price.

When competition can't undercut on price, it is hard to argue that customers aren't being served by monopoly mega corps.

thaumaturgy
1 replies
5d17h

Yeah, this is a pretty standard answer to the same market failures that so many people have been talking about recently. It assumes though that low prices are the best possible benefit consumers might get from competition, and it also assumes that megacorps necessarily lead to the lowest possible prices for consumers.

Chokepoint Capitalism is a pretty okay book that kicks the legs out from under both assumptions.

s1artibartfast
0 replies
5d15h

I don't think I'm making any assumptions about what's best. I bring it up because the most common argent levied against mega corps is using monopoly power to drive up prices, which I don't think is always true.

If people specifically want to break them up to drive down prices, I don't think that will always be successful.

thfuran
0 replies
5d15h

So, when someone says a business practice shouldn't be "tolerated", that's a perfectly reasonable position, except it doesn't actually work for businesses operating at extremely large scales.

I'm not really sure what you mean. The EU has managed to put together much better consumer protections in a lot of areas. There are large businesses there.

staunton
0 replies
5d22h

It's too difficult for customers to organize a protest in a way that will influence that business's decision-making.

It's actually very easy (in theory). You (vote for someone who will) ban fraudulent and anti-competitive behavior, sue the offenders and have them pay huge fines. The fact that this very rarely works out is a failure of the political system.

satvikpendem
0 replies
5d22h

This argument reminds me of pre-ordering games. People are simply too dumb to stop doing so, even if it's in their best interest. But those that do are often rewarded for not pre-ordering in the form of lower prices and more content, hen the game eventually goes on sale. It doesn't mean that pre-ordering should be banned, however.

krisoft
5 replies
5d21h

People complain about this relentlessly, but never change

What do you mean by this? How would “people change” to get out of resort fees or confusing pricing systems?

vanviegen
0 replies
5d21h

Vote for politicians that haven't been bought by big corp (yet).

tpm
0 replies
5d21h

Ban them. Here in the EU such behavior is mostly illegal.

s1artibartfast
0 replies
5d19h

people could stop buying from sellers with confusing pricing systems.

People are capable of change.

joshstrange
0 replies
5d21h

I believe they are saying that people fall for the hook pricing and won't go with an alternative that is upfront about their price (because it looks higher). If companies aren't rewarded for doing the "right" thing then why would they do it?

jimjimjim
0 replies
5d21h

"people" as a group need to stop shopping solely on price. Btw, You may look at other factors but as a general rule "people" look at price.

autoexecbat
1 replies
5d20h

Is this just restaurants?

justinc8687
0 replies
5d19h

No, it’s everything. Restaurants somehow thought they’d be exempt and the article is referencing recent explicit guidance that they will not be.

theoldlove
0 replies
4d15h

Too bad DoorDash is exempt

pixl97
0 replies
5d23h

I mean this is why a lot of first world countries have consumer protections that demands the retailer/service publishes prices up front. Course in the US there is quite a counter lobbyist group that prevents just that from happening.

duxup
0 replies
5d23h

it isn't worth the brain damage to sell to customers looking for the lowest price

The cheapest customers are always the most expensive to work with. It’s a sad reality.

I worked at several companies who for some foolish reason saw the cheapo customers as some untapped market and when they raced to the bottom they lost every time.

diob
0 replies
5d21h

This is why other countries just have laws for this, it evens the playing field and removes the bs.

antisthenes
0 replies
5d22h

It's not like they're providing a bad service for the cost.

When it comes to healthcare, they absolutely are. The US spends more than double for the same outcomes as other developed countries.

And no, this isn't because of the cost to develop new and novel drugs (which aren't used in 99.9% of routine health care)

silverquiet
25 replies
5d22h

I have to say that my response to "not knowing what you're going to pay for something until you're billed" is basically, "well I guess we'll see if I can pay it then". You sort of start asking yourself, "what if I just don't pay?". I have a (deadbeat) buddy who's entire medical plan is essentially to give a fake name to the ER, and if you're poor in America, what else can you do?

pjlegato
20 replies
5d22h

give a fake name to the ER, and if you're poor in America, what else can you do?

The American government spends something approaching a trillion dollars annually on Medicaid, "a government program that provides health insurance for adults and children with limited income and resources."[1]

Seperately, Obamacare[2] created a private health insurance market where low income people can obtain free or heavily discounted private health insurance coverage, according to their income.

[1] https://en.wikipedia.org/wiki/Medicaid [2] https://en.wikipedia.org/wiki/Affordable_Care_Act

silverquiet
7 replies
5d22h

We live in Texas.

pjlegato
6 replies
5d21h

Sounds like your issue is with Texas, then, not with America.

Why perpetuate crude and inaccurate stereotypes that smear and disparage America in general? America is spending a huge amount of money every year precisely on providing healthcare for the poor.

Besides which, it looks like Medicaid does indeed operate in Texas, which is discoverable in less than 10 seconds of Internet searching: https://www.hhs.texas.gov/services/health/medicaid-chip/abou...

And, seperately, Obamacare's marketplace works nationally, including in Texas: "More Texans than ever before enrolled in ACA health plans in 2024"[1]

[1] https://www.texastribune.org/2024/01/24/texas-aca-health-ins...

silverquiet
5 replies
5d21h

Have you considered perhaps that because I live in Texas, I know a bit more about those programs than you who have just googled and linked the top results? And that perhaps because I live in Texas, I indeed have issues with Texas (still a part of America in spite of the wishes of some on the right wing)?

From your own article (I knew this stat would be in there):

Currently, Texas leads the nation in the number of uninsured residents with nearly 5 million people living here without health insurance coverage, nearly double the national average.

Why do you think so many are uninsured if it is so simple for them to get health coverage?

pjlegato
2 replies
5d21h

You said giving a fake name at the ER was the only option for poor AMERICANS to obtain healthcare.

That is categorically false, and testifes to some sort of deep seated bias against our own country profoundly embedded in your worldview -- one which extends all the way to glib counterfactual promotion of verifiably false information about the country and its supposed moral shortcomings.

2) Texas has a much larger population than 49 of the other 50 states -- about 30 million, higher than any state but California. Of course the number of uninsured residents will be higher than the national average. You didn't control for uninsured per capita.

Why are so many uninsured if it's possible (I didn't say easy) for them to get coverage? Probably, a large part of the reason is people who go around promoting the (completely false) common trope that "poor Americans just can't get health coverage, the government does nothing for them."

silverquiet
1 replies
5d21h

Pretty sure my buddy is an American - I've known him almost all my life. I'm quite sure he's poor - he asks me for money a bit more than I'd like. The fake name thing was just a strategy he developed after going to the ER due to a blow to the head which caused him to forget his name; I'm sure there are many creative ways to get healthcare, but most of the of poor Texans (whom I'm also pretty sure are Americans considering I'm related to a few of them) I've known just go to the ER and ignore the bills. It's probably not the best system, but it's the one we've got I guess.

As to point 2) I thought about including the info in my previous comment, but it wasn't in your own source material and I'm a bit lazy. So here, I'll do it in this one.

Texas is still the state with the highest percentage of uninsured residents, at nearly 17 percent, according to the most recent U.S. Census Bureau survey released Thursday. [0]

[0]https://www.texastribune.org/2023/09/14/census-bureau-texas-...

Speaking as a Texan (also pretty sure I'm American) who lives with some disability, I can tell you that I've looked at the systems and it's a real fear of mine that I'll end up as one of these statistics.

grepfru_it
0 replies
4d1h

Well hi, also fellow Texan here. State law requires hospitals to take in and treat any patient with injuries “incompatible with life”. This means a fake name is unnecessary, you can give the same name. You may be asked about a previous bill but that will in no way preclude you from getting treatment.

I’m pretty sure there is a moral to this story..

vel0city
1 replies
5d20h

Why do you think so many are uninsured if it is so simple for them to get health coverage?

High numbers of people who have illegally immigrated and are worried about getting into programs subsidized by the government which might result in questions about residency status leading to deportation?

High rates of misinformation surrounding the costs of insurance and availability of welfare programs in the state?

I just looked up getting a plan in Texas. 40 year old male non-smoker in Texas earning $45k/yr can get insurance for $128/mo. They're eligible for an HSA, so they can put tax-free savings that roll over every year into an investment account to help cover the $7,400 deductible. PCP and preventative care visits are free. Generic drugs are $10. Urgent care out of pocket is $160. Other plans have slightly higher premiums but much lower deductibles, some have different co-pays.

silverquiet
0 replies
5d20h

Yes - my plan if ever separated from employer insurance is to manipulate my income and expenses (I have significant investments I can draw upon and access to a family real estate portfolio that allows me to live rent-free) in order to get the full subsidy. Most people are not like me however; my buddy has several times had his car impounded for not paying car insurance. I don’t think health insurance will make the cut in his budget.

rmah
4 replies
5d20h

Most Americans don't realize just how huge and extensive medicare/medicaid is. They cover 65mil and 88mil people respectively, including 40mil children. That's 153mil people out of 330mil population -- that's 46% of the USA's population. Total spending is $1.9T for both. A cost of $12,400/person covered. $1.9T is 7.5% of the total US GDP of $25.44T. Medicare/medicaid is a massive program. Bigger than social security at $1.4T. It's crazy huge.

silverquiet
3 replies
5d20h

In that case, why not just go all the way to Medicare for All?

jonfromsf
1 replies
5d19h

Because with the current medical cost, it would be absolutely ruinous. The root problem is American health care is far too expensive. The entire system is massively wasteful and complex.

silverquiet
0 replies
5d19h

Seems like unifying it under a single, already-extant federal system would greatly reduce administrative overhead though, don’t you think?

What’s funny is I’m probably one of the people whose paycheck is dependent on me not understanding that to paraphrase Upton Sinclair. When I started at my current employer, one of the devs straight up said the government should be doing what we do and we shouldn’t have a business.

quickthrowman
0 replies
5d19h

Unfortunately, several million people work in medical billing and insurance. No politician that is aiming to get re-elected wants to double the employment rate. It would cure inflation almost instantly though!

notaustinpowers
4 replies
5d22h

Obamacare is (largely) a failure due to insurance meddling and laws preventing government insurance programs from negotiating prices with drug companies.

I'm a healthy individual, not even 30 yet, never smoked, never broken a bone, and never even had stitches in my life. And when I last checked their prices, I was making only $45k/yr. For coverage with Obamacare, I would be paying $350/month for the "catastrophic" plan. Which includes no prescription copay, no dental, no vision, and only kicks in after I've spent $100,000 in one year, and it takes 6 months to take effect after signing up. It's only there for serious issues like losing a leg or cancer.

dangrossman
3 replies
5d22h

Obamacare is not a plan, it's a law that set minimum standards for health insurance plans, created marketplaces to cross-shop plans across providers on a single website, and created a system where you can receive government subsidies towards their cost if your income is low enough. At $45K, you would receive subsidies towards your insurance premiums.

I've bought my insurance on the ACA marketplace since it opened ten years ago. There's no difference I can see between the plans offered on the health insurance marketplace and those offered directly from the websites of the same insurers that offer coverage in my state (Blue Cross, CVS Aetna, United, Ambetter, etc).

The very highest deductible "catastrophic" plan offered on the marketplace in California for a 29 year old has a $9450/year deductible, which is also the maximum out-of-pocket expense for the year if you have this plan. A $100K deductible plan does not exist, and when you enroll during the annual enrollment period or after a qualifying life event, plans take effect the day you make your first payment, not months later.

I'm 10 years your senior and pay less than $350 per month with a lower deductible than the plan quoted above, with no government subsidies.

notaustinpowers
2 replies
5d21h

I use Obamacare in the rural southern way, referencing ACA and healthcare.gov, lol.

I'm also in Georgia which may have different regulations regarding the deductibles. I remember looking at the plans around 2 years ago and realizing that there was no way I could afford the premium, let alone the yearly deductible.

I just took another quick look at a non-Healthcare.gov site. Insurance for me would be $313/month with a $9,100 deductible. But it does not cover doctor visits, generic drugs, or specialist visits until after I pay the full $9,100.

Why would I want to pay $313/month for essentially no coverage until I spend 20% of my income towards a deductible before I see any benefits?

phonon
0 replies
5d21h

This is the cheapest plan I could find in GA. It said it would subsidize the $357/m cost down to $113 for $45k 40 year old Male non-smoker, single. It seems quite a bit better than what you are suggesting. (I used https://www.healthsherpa.com/ to more easily check out available plans.) It covers preventative care, and after the $9100 deductible, it seems to cover pretty much everything with $0 copay, as long as its in-network.

(You also get their negotiated rates when you go to the doctor, I assume.)

https://d3ul0st9g52g6o.cloudfront.net/2024/GA/sbc/2024_58081...

dangrossman
0 replies
5d21h

You get an annual doctor visit for free with any insurance plan. There is no deductible or copay. There are other categories that are covered with no deductible. My wife had a $13,000 IUD insertion under anesthesia at a hospital due to complications, and this cost her $0 with insurance, without having met her deductible, since reproductive health is covered with no out of pocket cost under all ACA plans.

I'm on healthcare.gov looking at Georgia's plans this year for someone with $45K of income. You have options starting at $129/month. Many of these sub-$200 plans get you doctors visits for $40-60, prescription drugs for under $25 each, mental health treatment for under $60 per visit. This is all without hitting your deductible at all, they're day 1 prices.

If you paid the cash prices for many of these doctors, specialists, therapists, they'd be many times higher than the insurance negotiated costs. Look under the "covered costs" estimates for things like mental health treatment, diabetes maintenance, broken bone treatment, etc and you'll see that the estimated annual cost for the insured is often half or less the plan's deductible -- which tells you that hitting the deductible is not when the savings start. I don't think my wife or I have ever hit our out-of-pocket maximums in a year, yet the insurance has saved us more than it's cost in most years.

You're going to start interacting with the healthcare system a lot more than you have in your 20s once you're in your 30s. We all do at that age. And if you have even the worst ACA plan, you'll start to understand what it's doing for you regardless of the deductible.

softsound
0 replies
5d22h

It's still expensive, I worked as a contractor full time for a tech company for a year which basically means I pay for my own insurance through Obamacare (though they did suggest insurance at work but they didn't pay anything towards it so it is just is in case you want a 4th party to help you help yourself). Still long before that I basically just said I didn't have insurance and would get the price knocked down considerably. Half the time it's cheaper not to even have insurance... But now I'm "responsible" in the hopes it might bring down the price for other things in the future. For planned events I recommend health insurance, despite how often it's not all that worth it. Honestly though my current insurance copay cost as much as being uninsured in other countries so I kinda laugh at this crazy idea. I pay about $100 to visit any specialist doctor and it only cost $80 to be uninsured in some other countries on top of the $350 a month I already pay.

heavyset_go
0 replies
5d20h

Medicaid wasn't expanded in several states with tens of millions of people living in them[1]. For example, in Texas, nobody is eligible for Medicaid based solely on their income alone.

Where Medicaid is expanded, the income requirements for Medicaid are far from sane. If you make over 100% to 138% of the federal poverty line, which is $15,060/year for an individual, you are not eligible for coverage. For example, someone who makes $16k to $21k a year, depending on where they live, is ineligible for coverage despite making poverty wages.

[1] https://www.kff.org/affordable-care-act/issue-brief/status-o...

danielvaughn
3 replies
5d22h

I worked in a couple of emergency rooms for a total of 6 years, and you aren't wrong. Assuming the care you need can be entirely performed within the ER, then that's a viable strategy so long as you don't intend to re-visit the hospital.

Hospitals are obligated to provide care regardless of your ability to pay in the moment, and of course they often take care of indigent and foreign patients, with the understanding that they won't be able to recoup their costs.

If you don't have your identification on you, then they simply give you a number to call and they usually ask to respond within a few business days because after that, it's harder to get insurance approval.

This all changes if you're admitted to the hospital and need surgery, because I think there's pre-approval required from your insurance carrier. Though since I worked in the ER, my memory is hazy on that.

If they believe you were deceptive and simply refused to be identified, then that is technically illegal, so they will put up your picture and will alert the police if they see you again. But if you never intend to visit the hospital again, it doesn't really matter. No one is going to hunt you down unless you're doing it on such a large scale that it can't be ignored.

This all said, I want to say that I still think it's an unethical thing to do.

explaininjs
1 replies
5d22h

I have a friend that incurred massive medical debts in college when their (college-provided) insurance refused to cover services rendered at the (college) hospital. The college then made all sorts of threats like “you won’t be able to graduate until this $$,$$$ bill is paid!!!”. Keep in mind my friend was only able to attend college on a full scholarship as a result of coming from a poor ESL family.

They ignored all the threats, the department in charge of threatening seemingly didn’t talk to the department in charge of graduation, and to this day (5 years later) they still receive near-daily letters in the mail requesting payment. I can’t say I find my friend’s actions unethical in the slightest.

Ah! The UC system…

simonbarker87
0 replies
5d21h

As the old saying goes “If I owe you $100 it’s my problem, if I owe you $100,000 … it’s your problem”

silverquiet
0 replies
5d22h

alert the police if they see you again

Aren't inmates technically supposed to get healthcare? Could be a last-ditch option I suppose if you can't get something covered any other way. Personally I'm trying to stay employed to keep my coverage, but sometimes you can't outrun the layoffs.

jameshart
5 replies
5d21h

Then pay for it with a credit card that gives you a random amount of cash back, and pay your credit card bill using your tax refund that came as a nice surprise at the end of the year, because you over withheld all year so you don’t actually even know what your real take home pay is supposed to be.

No wonder Americans have no idea how the economy is doing.

TechDebtDevin
3 replies
5d21h

I'm always surprised by people who expect and are excited for their refund. Half the country is pitching about politicians taxation plans when all the while they're volunteering to overpay out of every paycheck.

tatpacc
0 replies
5d21h

people who expect and are excited for their refund.

and then pay % of your refund as a fee to CPA, so they help you maximize your refund.

jkingsman
0 replies
5d21h

When you lack the self-control to save, over-withholding + refund becomes an ersatz year-long savings account. It's terrible, lacking any interest at all, but when it's all you've got, breaking open the piggy bank to get your money back feels good even if it doesn't make financial sense.

gadflyinyoureye
0 replies
5d21h

I know how many guns I have. I know the Feds have more. Therefore give them their pound of flesh before they show up with the myriad of guns.

_factor
0 replies
5d20h

Even paying cash. Unless the establishment offers a cash discount, you’re paying inflated credit card induced compensatory pricing.

It’s a real shame banks received 2-5% of most transaction for what costs them pennies. Sure, there are benefits, but their ask isn’t covering it.

wongarsu
1 replies
5d23h

Which is especially weird for the country that fought the cold war to show the supremacy of capitalism; a system that is based on market actors knowing the price and value of every product and service offered and making rational purchase decisions based on that.

heavyset_go
0 replies
5d20h

Capitalism can't work without information asymmetry when it comes to pricing.

godelski
1 replies
5d22h

Don't get me started on dark patterns! I think the fact that we create them should be something we discuss.

But I think we need to up our game in this cat and mouse game a bit. For example, in aggregators -- like Expedia or Google Flights, etc -- why not try to capture some of these fees in the price? I can search for hotels with parking but what about sorting hotels by price and including the parking price? It's hard to compare when I see a $120 hotel that has a $50 valet vs a $150 hotel that includes parking. But that's the thing I'm actually after a lot of times. Or similar with flights and baggage fees. We should be able to collect a lot of this type of information and properly present it to the users and try to make these types of dark patterns ineffective (still will be cat and mouse and this is only a specific type of pattern, but still, I think there are things we can do)

mikepurvis
0 replies
5d22h

Probably an opportunity there to gather and expose that data, same as how flight aggregators like orbitz twenty years ago started showing prices inclusive of taxes and fees.

alex_lav
1 replies
5d23h

Hiding blatant fraud behind the label of “convenience”. Every industry. Pay for Amazon prime to get 2 day shipping “for free”. Item arrives in six days. Prime does not get refunded.

Pay +10% for “priority” rides in Lyft, supposed to arrive in 1-5 minutes, whereas “regular” is 7-15. Car shows up in 18 minutes. Priority payment does not get refunded.

Honestly what’s even the point in caring anymore. Living in America is about getting grifted until you can hopefully figure out your own grift. The irony of posting this thought on this website is intentional.

lapphi
0 replies
5d22h

Yes. Frank reynolds in the popular television show it’s always sunny in Philadelphia lays it out in plain terms on one of the episodes. “In America you’re either the duper or the dupee”. PT Barnum also knew this essential truth about our nation.

127
1 replies
5d21h

It's strange from a foreigners perspective because all of this looks exactly like a dark pattern made to distract and confuse gambling addicts. Why does American government allow direct predation on its own citizens?

bee_rider
0 replies
5d20h

Did they manage to get AWS to provide some mechanism to get an up-front price in your country? That’s pretty neat if so.

I think lots of people go along with these as-you-go services because they’d rather deal with an unexpected bill, than having their servers shut off.

xkcd-sucks
0 replies
5d23h

Less advanced economies have discovered this pricing innovation, but are generally not able to scale it beyond hostess bar scams and the like :)

datavirtue
0 replies
5d18h

Dont forget the runaway interest that accumulates on the credit you used to pay for it. It really never ends. The IRS even has a few pitfalls that allow you to pay taxes you don't really owe, that can't be refunded.

TeMPOraL
0 replies
5d22h

Maybe that's why credit cards are so extremely popular in the US? I suppose if you never know how much you're going to pay for anything, it helps to have a buffer between the vendor and your bank account, allowing you to review the charges after the fact, and should you dispute some, be on a somewhat even playing field with the vendor.

matsz
52 replies
6d

This is precisely why I stick to dedicated servers for my own personal projects.

$40/month for a machine, doesn't get any more predictable than this.

zeroxfe
34 replies
6d

There's no way you're getting a machine for $40/month. :-)

Did you mean a dedicated VM or VPS?

(I have a bunch of actual dedicated machines with different providers, and this would save me a lot of money.)

(Edit: holy moly those prices are fantastic!)

dinvlad
22 replies
5d23h

Yeah, Hetzner has 20 vCPU 64GB RAM i5-13500 servers for $40/mo:

https://www.hetzner.com/dedicated-rootserver/matrix-ex/

This is ~10x cheaper than the closest AWS option, and without the extra fees.

Also of note that Hetzner is profitable, which means AWS has been operating at an insane markup.

thorncorona
11 replies
5d23h

Also of note that Hetzner is profitable, which means AWS has been operating at an insane markup.

You can also evidence of this on their 10-Q :^)

ikiris
10 replies
5d23h

If you think offering a cloud service is at all the same as offering someone a box in a rack, I invite you to compete in the space.

TacticalCoder
5 replies
5d22h

It's more than a box in a rack. These providers do actively monitor and fix these boxen. They can all be rebooted remotely as if you physically hit the button, you've got interfaces to access the machine as if you were logging in physically with a DB-9/RS232/ethernet/whatever console, etc.

It's not just "space in a rack and you deal with the servers yourself and you come to fix them if they break".

These companies know what they're doing.

ikiris
4 replies
5d19h

You're missing the point. There's a massive difference between getting a box service, and getting a highly available regionally distributed service with a semblance of a SLA of bandwidth to anywhere on the planet. To quote a former manager, its not even apples and oranges, is apples and pumpkins. They simply aren't in any way the same scope.

exceptione
3 replies
5d18h

What do you mean? The big bare metal providers have multiple datacenters with fat pipelines and peering. You can put those geo-separated servers in the same subnet. Europe and US is covered, it looks like Asia a black spot for Hetzner, I will give you that. I doubt that so many business have such size and global reach, that world wide latency is a priority. If it would be, the average site would not connect to 20 domains to load megabytes of javascripts, trackers and what not.

(Also, at Hetzner you can even rent their network hardware if you want to make custom solutions.

ikiris
2 replies
5d17h

Sure, go build your own less reliable cloud. But don't pretend that single box pricing is comparable to single cloud service as a result.

withinboredom
1 replies
5d12h

I don’t know why you are suggesting that people will build unreliable things. My company’s first day of AWS was the first time AWS had a major outage. We were sold on more reliable, but I can tell you AWS is just as reliable as home grown BS, if not less reliable. The biggest difference is what you do during downtime: in AWS, you refresh status pages. In your actual hardware, you’re actually problem solving and able to build/deploy workarounds to get back working within 30m.

surgical_fire
0 replies
4d7h

Having worked for one of the major cloud providers in the past, I second this.

In fact, I have no idea why people go for those. You pay a massive premium for the "privilege" of not owning the infrastructure, and being subject to opaque pricing and outages that are completely beyond your control.

And in terms of actually managing the stuff, now you have to pay staff to manage your cloud things too.

It makes no sense to me.

dinvlad
2 replies
5d23h

Hetzner does have a cloud VPS option, which is still very affordable: https://www.hetzner.com/cloud

This is pretty heads-to-heads with EC2, in terms of how it works behind the scenes.

ikiris
1 replies
5d23h

So go with hetzner?

dinvlad
0 replies
5d23h

Already do :-)

benterix
0 replies
5d9h

Hetzner is actually offering their Hetzner Cloud and it's a joy to work with because of its simplicity (think about the early days of AWS). You can do everything in Terraform or via their CLI if you prefer. Setting up a full k8s cluster takes maybe 10 minutes including all configuration.

intelVISA
5 replies
5d17h

Yeh but a pragmatic decision like a $40pcm box won't go down well with your now very bored team of SREs, DevOps and distributed systems engineers(tm) who demand more playtime with the magic cloud toybox (for a 100DAU internal app).

bionsystem
1 replies
5d12h

You guys have teams that decide the infra they run ?

chronid
0 replies
5d12h

If they manage it in my company at least of course, the real question is if they want to manage it (and know what that means). Usually they don't, in my experience.

robben1234
0 replies
5d14h

Is there anything stopping those teams from deploying k8s and every possible apache product onto these machines?

There's also middle ground in the form of Digital ocean's kubernetes which runs noticeably cheaper than big tech cloud offerings.

chronid
0 replies
5d12h

Those people will actually be grateful they don't have to deal and spend weeks to debug yet another cloud provider hidden gotcha or bug and convince support they are right. They will now very happy as they can now deliver value (at relative scale) and really speed up processes. I know I am.

Good ol' bare metal is real nice, but it won't save you from application complexity, security requirements, and so on - you still need to manage it somewhat. If you're not a startup looking for market fit at least.

benterix
0 replies
5d9h

I'm not sure about that. We get paid well for working with (or around) public cloud complexity, but I bet many of us would gladly manage much simpler setups like the Hetzner Cloud.

notarealllama
2 replies
5d23h

Dang, that is a fantastic deal. The €100 / month is even better - DDR5 RAM, 2TB NVME raid 1, and it's all customizable too. Just have to wait for Ubuntu 24.04 to be available and might have to make this switch.

yashg
0 replies
5d23h

Just setup two Ubuntu 24.04 servers in last couple of days. One at Hetzner and another on AWS.

dinvlad
0 replies
5d23h

Yeah, they're all pretty fantastic. Can even upgrade existing servers with more RAM/disk, or store ~250 TB of data for $400/mo etc.

Not to mention unlimited free 1 Gbit egress/server

margorczynski
0 replies
5d22h

And this is just for the "dumb" EC2 instance, the markup on their "smarter" stuff is probably much higher. In general I understand why one would want to start off in the cloud but staying there for 10+ years is quite absurd given the costs.

distantsounds
4 replies
6d

OVH and Hetzner have offerings at that price range. you can use a tool like serverhunter.com to find all sorts of economically-priced servers.

Scarbutt
2 replies
5d16h

Does OVH asks for passport verification too, like Hetzner?

HeatrayEnjoyer
1 replies
5d16h

Why would they ask for a passport? Many people don't even have one.

Scarbutt
0 replies
5d15h

The worst part is they ask for it after you have enter your credit card details. Maybe they have policies per geographic region for this.

matsz
0 replies
5d23h

Yep, I'm renting two servers from Hetzner; have one in Germany and another in Finland. Both cheap (€40/month) and over the few years I've been using their services, I have nothing to complain about.

eddd-ddde
1 replies
5d23h

Also a dedicated VM is /literally/ a dedicated machine. It's in the name!

jameshart
0 replies
5d21h

A dedicated virtual machine is a kind of dedicated machine, sure. Like a private community swimming pool is a kind of private swimming pool.

toast0
0 replies
5d23h

I've got a dedicated machine for $30/month. It's ancient, a xeon L5640 with 16 GB ram, and 1 TB spinning disk, but it's dedicated and it works great. Well actually, the first one stopped working well, and I got a replacement with double those specs for the same price; and the second one is working great. I also run with full disk encryption, because I don't trust their opsec on wiping drives, so that's a bit of a hassle to reboots, I have to get a console with IPMI and put in the disk password, although I saw something [1] last night that inspires me to consider the possibility of automation.

I recommend shopping at https://lowendbox.com/ and https://lowendtalk.com/categories/offers

[1] https://github.com/emtiu/freebsd-outerbase

javchz
0 replies
5d23h

OVH's so you start and server4you offer dedicated machines for around that price. For personal projects they work great

buildbot
0 replies
5d21h

I'm renting a box from Hetzner - 4x 22tb HD, 2x 1.5TB NVME SSD, Ryzen 3600, 64gb ecc ram... for 100$ per month! It's nice :)

TacticalCoder
0 replies
5d22h

There's no way you're getting a machine for $40/month. :-)

Oh you can!

I've got several dedicated servers at OVH. My absolute cheapest one is an "ECO" / Kimsufi (Kimsufi is a company which spun out of OVH then, a few years later, back into OVH) which I pay... 5 EUR / month. 6 EUR / month with VAT (so 6.5 USD per month).

Sure, it's not beefy at that price: an Atom N2800 with 4 GB or RAM but it is a dedicated server with its own IPv4 IP (yup, there can be uses for that).

I mostly use it as a jump host / reverse-ssh-with-a-known-fixed-IP thinggy.

They've got great dedicated servers at very good price and they're not the only ones in that space.

These can be rebooted/reinstalled remotely and they're monitored: OVH shall deal with hardware failure, if any, for you (never had any so far).

baobabKoodaa
16 replies
6d

If you rent a dedicated server from AWS, you will be hit for various additional fees which will likely dwarf that $40/month.

So the issue here is not really shared vs dedicated instances. The issue here is that a particular cloud provider (namely AWS) has set up an opaque fee structure.

oooyay
6 replies
5d23h

I suspect they're not getting a dedicated server from Amazon. Anyone doing that is either foolish or is so locked into AWS infrastructure that it cost-wise makes more sense than to expand a network to another provider. Fun times, indeed.

What I started doing is just running my own "cloud" out of my house for personal projects. I have all of the things I need. There's some overhead in terms of maintenance and up-front setup cost in terms of time and equipment, but after that it's pretty smooth sailing.

yolovoe
3 replies
5d15h

I suspect they're not getting a dedicated server from Amazon.

If it's literally called a dedicated server, why would you suspect otherwise?

Source: I worked in the core EC2 dataplane for a couple years. PEs and leadership there would not be happy with misleading customers. We constantly thought of the customer experience there.

StackRanker3000
1 replies
5d11h

I think you misread the comment you responded to.

It didn’t mean ”I suspect that what they are getting from Amazon isn’t a dedicated server”.

Rather it meant ”I suspect that they’re getting a dedicated server from somewhere else that isn’t Amazon”.

The original comment was talking about a dedicated machine for personal projects with a fixed cost of $40/month.

yolovoe
0 replies
4d16h

Ah, this makes sense. Thanks for clarifying!

oooyay
0 replies
5d15h

Generally you get a "dedicated host" from AWS, not a "dedicated server" so that was kind of my first tip off. There's a very big difference; the only usecase I'd really consider for dedicated hosts is security related and, frankly, they're far more expensive than any colo box. Dedicated Hosts are also far more restrictive than what you get at a colo DC. Second, even if AWS offered to let you have full control of a rack box you'd have to be economically out of your mind with AWS' network costs compared to a colo DC.

whiplash451
1 replies
5d23h

You still have network performance limitations + the security aspects to run by yourself (as in: not getting your data drive encrypted by a ransomware)

oooyay
0 replies
5d17h

I probably wasn't explicit enough with "cloud" but I mean running your own isolated workloads with at minimum virtual machines and virtual isolated storage (block or otherwise). From a topological perspective my home DC resembles most major clouds other than the fact that there are not two sites.

drdaeman
4 replies
6d

If you rent a dedicated server from AWS

Who, in their right mind, goes for a bare metal to AWS, when there are so many decent and time-tested options out there?

jedberg
1 replies
5d23h

Data Gravity. If you already have all your data in AWS, and your app that is generating new data is in AWS, it makes a lot of sense to get bare metal in AWS to do batch workloads on that data, so there is no egress fees.

benterix
0 replies
5d9h

We did a calculation in my previous company and it turned out even with the egregious egress costs it was way cheaper to host these huge workloads outside of AWS.

smabie
0 replies
5d23h

crypto hft

mywittyname
0 replies
5d23h

Anyone who has experience with AWS or is looking to hire candidates with known skills. AWS is the industry standard. A lot of quality candidates know it and use it, because it pays the best and has the most job opportunities. And most companies use it because they were first to market and its easy to find candidates.

I'd argue that startups should have a good reason for not using AWS. The costs for their basic services is not that much compared to the cost of development.

ElevenLathe
1 replies
6d

In my experience, when people say "dedicated server" they typically mean something like OVH rather than a real "cloud" provider (AWS, GCP, Azure, etc.). In other words, something morally equivalent to colo but without having to ship servers around.

baobabKoodaa
0 replies
6d

Yes, that is likely the case here as well. But if someone is not well versed in cloud providers and reads these comments, they might be misinformed without clarifying that this issue is AWS vs other providers, as opposed to dedicated vs shared resources.

whiplash451
0 replies
5d23h

You can get much cheaper dedicated servers with alternative providers such as OVH.

matsz
0 replies
5d23h

Oh no, I would never go AWS for dedicated. Should've specified that in my comment so people don't get disappointed if they try to get that from AWS.

superfrank
15 replies
5d23h

It seems like AWS' entire business model is making the pricing so confusing that you don't know what it will cost until after you've used it.

Maybe I'm naive, but I don't think that's intentional. I think it's just a byproduct of trying to build something that works for everyone and every use case.

As you make your target market bigger and bigger, you continually hit edge case after edge case that you try and solve with "just one more" rule or option. Eventually the system becomes so complex that no layman can understand it.

zb3
5 replies
5d23h

So you're saying that they can't implement a hard limit, yet are able to provide hard-limited trials / student credits? It's impossibly hard to believe they couldn't implement the pre-paid model

mtnGoat
4 replies
5d15h

would you want your services to just turn off when a certain threshold is hit? If so, this problem is solved, set an alarm when cost hits X, use that alarm to trigger a Lambda to turn off the unwanted infra. AWS provides all the tools you need, you just need to use them.

withinboredom
0 replies
5d13h

Yes. Turning off is exactly what I want.

nitwit005
0 replies
5d12h

Generally, using cost reporting directly doesn't work due to the delay in reporting of the costs.

dns_snek
0 replies
5d10h

This problem should be solved with a simple form in the billing area, not a solution that probably requires upwards of 40h of work to set up and properly test to a level I would trust it with anything. Even then it's dependent on delayed cost reporting with unknown failure modes. It merely makes a best-effort at containing the costs and it doesn't actually limit them.

benterix
0 replies
5d10h

Yes, but I want this lambda to be written and run by Amazon. This was requested already in the first days ow AWS, they literally said they understand it is important for us and will work on it, and then stopped replying about this issue for many years, in spite of people asking for it and voting for it all the time. They know perfectly well what is going on and decided not to do it for business reasons.

ikiris
4 replies
5d23h

Was it "I didn't understand it and/or put in the effort to understand it first" ? Nah it must be "theres a conspiracy to make it confusing so we pay more"

AWS has one of the nicer to comprehend billing systems.

zb3
3 replies
5d23h

Is there a pre-paid option?

cyberax
2 replies
5d22h

Yes. AWS Advance Pay.

zb3
1 replies
5d22h

I couldn't find a list of "eligible charges", also that doesn't seem to mean advance payment would be the only payment option so I couldn't be billed and the service would just stop.. did I miss anything?

benterix
0 replies
5d9h

No, you didn't miss anything the advance payment is just an additional feature but you still need to give in your CC details so they can charge you if you make a mistake. The usual course of action is to stress out and beg them to forgive you, which they sometimes do, and sometimes don't.

tjoff
3 replies
5d23h

That would make some sense if the edge-cases didnt have 10.000% margin.

superfrank
2 replies
5d23h

I've worked at companies that were spending 6 figures a month on AWS and we had dedicated AWS employees who helped us understand our bill and keep pricing in check. We also had GCP and Azure reps constantly reaching out to see if they could win us over by showing us how they could lower our bill. Overcharging enterprise customers for things they're not using is actually a risk to AWS as it gives customers a reason to jump ship.

The whales on AWS aren't overspending because AWS uses the granularity of their billing to make sure they aren't. That's how they keep the whales happy.

For the little guys, I'm not even sure it's worth AWS's time to nickel and dime them. AWS hands out thousands to ten of thousands of dollars in credits like candy. I'm pretty sure it's more important to them to lock in whales when they're still minnows than to bilk an extra $500 a year out of a 10 person start up.

Hikikomori
1 replies
5d22h

Idk about that, it becomes worse at larger scales. We're at 7 figures weekly with obscene amounts of waste. But at that scale it's like hundreds of Aws bills to understand.

okrad
0 replies
2d21h

Is anyone dedicated to working on optimization?

rco8786
12 replies
6d

AWS' entire business model is making the pricing so confusing

My go to line here is that Cloud was a ZIRP. Like the whole entire thing. Took us ~10 years to wind up the cloud, and it will take years to unwind it, but the mass migration away is already happening.

To be clear, I don't mean like AWS is going out of business or anything. Just that companies are a) realizing how insanely expensive it is, b) realizing how wildly volatile the pricing is, and c) starting to reach for services with transparent, fixed pricing

hn_throwaway_99
5 replies
5d23h

My go to line here is that Cloud was a ZIRP. Like the whole entire thing. Took us ~10 years to wind up the cloud, and it will take years to unwind it, but the mass migration away is already happening.

This is Hacker News echo chamber stuff. There is certainly no mass migration away from the cloud. Yes, I personally saw companies in the 2010s say "we're moving everything to the cloud!" without adequate planning or cost analysis and then saying "OK, everyone off the cloud" once they got an insane cloud bill.

But cloud costs can be managed, and for many, many companies the cost of hiring people to manage all this infrastructure and services is usually way more than a well-managed cloud project. Also, the canonical rationale I see on HN for moving away from the cloud is "I can just rent a box for $X/month". If all you're using the cloud for is a dumb, static set of compute, I agree that you can probably do it cheaper on your own. I know hardly any companies (from small startups to large enterprises) who use the cloud that way.

rco8786
2 replies
5d23h

for many, many companies the cost of hiring people to manage all this infrastructure and services is usually way more than a well-managed cloud project.

Yea, this is the line that everyone uses. I've worked at these companies, and the reality just doesn't line up with that. You end up still needing your whole Ops team, they're just building cloud tooling instead of on-prem tooling.

As to whether or not the migration away is happening...major cloud providers are already seeing people leave, and profits starting to contract. It's very early. Like I said in the original post, it took us a long time to wind up the cloud and it's going to take a long time to come back to reality.

Something roughly cloud-shaped will probably always remain..there are some legit use cases especially for companies that have spiky load profiles. I don't mean we're literally going back to running servers out of our IT closets. I just mean that as a whole we're going to be moving back to simpler deployments, simpler architectures, and most importantly, fixed/predictable costs.

hn_throwaway_99
1 replies
5d23h

I just mean that as a whole we're going to be moving back to simpler deployments, simpler architectures, and most importantly, fixed/predictable costs.

That I can definitely agree with, I just believe that's fully possible to do with cloud (for the most part, though there are certainly some head turners like the recent news that AWS was charging for forbidden attempts on private S3 buckets, which is bonkers) and cloud cost management tools.

rco8786
0 replies
5d21h

Yea that’s totally fair. I think we’re using slightly different definitions of “cloud” here. I’m mainly focused on the managed services/abstractions, things that abstract away the server itself and are usage based billed. But firing up a few ec2 instances is still technically “the cloud”, you are right.

hn_throwaway_99
0 replies
5d22h

I would take that with a giant tub of salt:

1. As the saying goes, talk is cheap. It's one thing to ask "What do you plan to do?" vs. what you actually do. Look at revenue graphs for AWS, Azure and GCP over the past 5-10 years, right up until the end of 2023. They are definitely not shrinking.

2. I'd be more than a bit skeptical of the messenger, given that Dell obviously has a vested interest in telling people they need to buy more servers.

3. Even if you take what the surveyed CIOs say at face value, asking "Are you planning to move some workloads back to private cloud/on-prem from public cloud" is totally consistent with what I said. There was rush of "just put everything on the cloud" without thinking through it strategically. But just because you're pulling back on a few ill-thought-out cloud projects doesn't mean that overall industry-wide public cloud investment isn't still skyrocketing.

brigadier132
5 replies
5d23h

I don't think people understand how absurd egress costs are. I was talking with some people in game dev because im making a multiplayer game and i wanted to understand what my future costs would be. Bandwidth was #1 over compute.

bauruine
3 replies
5d23h

Yeah it's more than absurd. I have a rented server that does 230TB egress a month. That's 4710$ in AWS egress fees (0.02$ per GB). I pay 40 Euros or about 0.0002 per GB and that includes an AMD Ryzen 5 3600 with 64GB RAM.

apitman
2 replies
5d23h

May I ask who your provider is?

bauruine
1 replies
5d23h

That's https://mevspace.com/ but hetzner, scaleway, ovh, psychz.net, reliablesite.net etc. have similar pricing. If you increase your budget to 100 there are many, many more in Europe and the US.

apitman
0 replies
5d22h

Thanks!

apitman
0 replies
5d23h

I've done a decent amount of research on the cheapest VPS providers for network-heavy applications. Take a look at Hetzner and OVH if you haven't already. Also here's a useful comparison:

https://getdeploying.com/reference/data-egress

dheera
8 replies
5d23h

We really need laws that allow customers to set a legally-binding max spend per month on any service, after which it's upto the company to suspend services.

This should apply to everything -- cloud compute, healthcare, phone bills, internet services, everything.

balls187
7 replies
5d22h

You sort of have that already in the US for health, its tied to your insurance and its called max out of pocket. Of course it only applies to covered services, and insurance companies and medical providers don't necessarily make it simple to know what is and is not covered (though it IS improving).

If you plan on seeking medical services without going through insurance, the ACA requires providers to provide you with a good faith estimate upfront.

dheera
5 replies
5d20h

its tied to your insurance and its called max out of pocket

F that, if I paid for insurance I shouldn't have to pay anything out of pocket

If you plan on seeking medical services without going through insurance, the ACA requires providers to provide you with a good faith estimate upfront.

F that, if there is any out of pocket payments at all, I should be entitled to estimates even WITH insurance and they should be legally mandated to be within 5% of the actual cost.

balls187
4 replies
5d20h

F that, if I paid for insurance I shouldn't have to pay anything out of pocket

That's not how insurance works, bud. For one, you likely don't pay for insurance, its heavily subsidized by your employer, and they will determine which policies to offer you. For two, you share the risk with everyone else who has a policy with your provider.

I could see an argument for insurance companies being legislated to force them to cover more previously uncovered services, as the ACA did.

F that, if there is any out of pocket payments at all, I should be entitled to estimates even WITH insurance and they should be legally mandated to be within 5% of the actual cost.

Sure, you are welcome to contact your insurance BEFORE you obtain services and find out what they will cover. In fact, its incumbent upon you to do that, and not expect HCP's to do that for you.

ipython
3 replies
5d17h

Sure, you are welcome to contact your insurance BEFORE you obtain services and find out what they will cover. In fact, its incumbent upon you to do that, and not expect HCP's to do that for you.

Had experience with that too. Insurance company will not give you a straight answer as to what’s covered or not- as for example the anesthesiologist may be out of network and you’ll never know until it’s too late. The health care provider won’t give you that info either. So you end up rolling the dice and hoping it all is covered at the end of the day.

relativ575
2 replies
5d16h

as for example the anesthesiologist may be out of network and you’ll never know until it’s too late

Are you sure you don't make this up? Any insurance that I've ever had always let you search if a particular provider is in their network. I just verified on the website of my current insurance. Dozen of anesthesiologists nearby are listed. In fact they often go out of their way to notify you if your doctors leave their network.

ipython
1 replies
5d16h

So let’s say I want to book an outpatient surgical procedure. I know my surgeon is in network and the facility I visit for the procedure is in network. However there are always other services required that you may not be aware of - anesthesiologists are the most common. This is a very common occurrence which I have encountered (thanks for calling me a liar).

See for example https://www.reddit.com/r/personalfinance/comments/ho49d5/ane... - first hit of many for “anesthesiologist out of network”

Especially if you have an emergency surgery it’s not like you’re picking your anesthesiologist from an approved list before entering the OR, even if you took the time to make sure the hospital you visited is in network.

Finally look at the fine print for your insurance company’s website. You’ll see multiple disclaimers noting that the information may not be accurate - for example from blue cross blue shield:

Blue Cross and Blue Shield Licensees have made reasonable efforts to ensure that the list of providers displayed is up to date and accurate. Please call the provider before making an appointment to verify that the provider continues to be part of the network.

Then when you call the provider to confirm, they shunt you back to your insurance provider. It’s hilarious if it didn’t have significant financial consequences.

balls187
0 replies
5d2h

I suppose I just got lucky, using my primary doctor to refer me to services, and doing the research to find which referral docs were covered.

Oddly enough, I had the exact opposite experience recently, doc's billing dept said specific medical appliance would not be covered so I had to pay out of pocket, only to find out later it was reimbursed by insurance and I got refunded for my out of pocket cost.

ipython
0 replies
5d17h

If you plan on seeking medical services without going through insurance, the ACA requires providers to provide you with a good faith estimate upfront.

That is truly funny. I have literally spent an hour across multiple agents to quote a single service out of pocket. I’ve done it multiple times across several providers. Every time it’s the same painful procedure where nobody can ever give you an answer.

I struggled to find the words to describe the way you’re treated when you ask for a quote. This is the best I’ve come up with: It’s like shopping at a luxury store with no price tags- if you dare ask the question “how much does this cost?” perhaps you don’t belong here.

balls187
1 replies
5d22h

It seems like AWS' entire business model is making the pricing so confusing that you don't know what it will cost until after you've used it.

Complicated cost calculations are only a part of the issue. You (or your team) also have fault in that you did not take the time to understand the costs associated with your decisions and not utilize AWS cost management capabilities (that is assuming you did get a surprise bill in lieu of an alert saying you hit a budget threshold).

And that is in part due to the shift from having dedicated ops teams, to having programmers take on more infrastructure tasks. This isn't unique or novel--incorrectly configured buckets, committing access keys, poor IAM setup, etc happen so frequently due to devs who have no real practical experience managing production infrastructure having unchecked access to AWS.

robertlagrant
0 replies
5d21h

Well, all the infra people did this too, just earlier.

thefourthchime
0 replies
5d22h

The other way EFS will bite you is if you don't have enough provisioning and you push changes to prod that make it throttle to a point where it doesn't work and then you're making emergency changes from a hotel room on vacation.

I don't think that's AWS explicit business model, but I think they're perfectly fine with it happening.

inkyoto
0 replies
5d17h

It seems like AWS' entire business model is making the pricing so confusing that you don't know what it will cost until after you've used it.

This is patently not true. AWS is very transparent with the pricing and offers a cost calculator. Specifically for EFS, you could have used the EFS service cost calculator before setting off with your endeavours:

https://calculator.aws/#/createCalculator/EFS

Other services are available at https://calculator.aws/

Any work on any cloud platform has an imperative step: the cost analysis. Once constituents of the solution are finalised (step 1), it is compulsory to proceed to step 2: estimate the charges it is going to cost and decide whether it is affordable or not. Skipping the step 2 and blaming the cloud platform provider business model is not fair and is akin to overspending on a credit card and blaming the credit card issuer for allowing you to do it.

Cloud is not dissimilar from UNIX/Linux – both give one a large variety of elaborate footguns to shoot oneself with, and if they worked as root on a software development project on a UNIX/Linux box and accidentally wiped the root file system out, who is to blame? Since cloud charges are usage based («pay as you go»), the cost estimate is a requisite that can't be avoided unless the budget is unlimited.

freeqaz
29 replies
6d

How are these companies even getting these GPUs? I would imagine NVIDIA would give them all to Microsoft and Google if they are in short supply.

rvba
11 replies
6d

Why couldnt NVIDIA auction some to get best price?

I mean locking customers long term is probably better, but a certain % of product still vould be sold for a spot price? This made me wonder what is the optimum split on that - obviously depends on your market placement, but isnt nvidia basically a monopoly now?

autoexecbat
9 replies
6d

At some point, nvidia does have to worry about amd getting their act together once they've annoyed all the current customers

nothercastle
8 replies
5d23h

Amd will never figure out drivers and software stuff. That’s always been their weakness

margorczynski
2 replies
5d19h

I wonder what Lisa tells the shareholders & board when they ask why billions fly past their noses in valuation. Curious why ATI/AMD never could be on par to Nvidia regarding the software stack (drivers mainly), I would assume bad leadership and goals.

latchkey
1 replies
5d18h

I'd call it a focus on very profitable ventures, like gaming.

Nvidia effectively created this AI world by building tooling and software for something we didn't know existed up until not too long ago. AI was created on the shoulders of the work that was done.

AMD wasn't taking the risk on the unknown, for whatever reason.

Water under the bridge. The ship is certainly busy course correcting now. Give it some time.

nothercastle
0 replies
5d16h

Well they didn’t have any money. They bet big on cpu r&d and they did great but there wasn’t enough money left over for other moon shoots. AMD was at risk of becoming a failed company not too long ago so they recovered quite well.

latchkey
2 replies
5d18h

Amd will never have a CPU with more core's than Intel.

I'm sure people said that too and look at where we are at today.

nothercastle
1 replies
5d16h

I don’t think anyone ever said that. Back when amd was lagging badly the core war wasn’t really a thing

latchkey
0 replies
5d15h

the core war wasn’t really a thing

Core performance has always been a thing.

Palmik
1 replies
5d22h

MI250 and MI300 are competitive for inference and training. The problem is that there's no cloud with simple registration that offers these for reasonable on demand price. This means that basically no one in the open source AI community can use them, so most development and tooling is being tested and built for NVIDIA.

latchkey
0 replies
5d21h

I am working on solving that issue with MI300x.

zer00eyz
0 replies
6d

Why couldnt NVIDIA auction some to get best price?

Have you seen their pricing? They are robbing everyone and selling out quickly. I don't think "auctions" are going to add much to thee bottom line.

jedberg
7 replies
6d

I head up Product at Lambda. We are an NVIDIA preferred partner (and keep winning the preferred partner award). I don't know what the allocation numbers are for other companies, but we get a lot.

It's in NVIDIAs best interest to spread the love around and make sure all the GPUs don't go to the hyperscalers.

tpurves
1 replies
5d22h

I anyone buying AMD MI300? I am curious if anyone is finding AMD, as an alternate GPU vendor, available or useful yet?

latchkey
0 replies
5d21h

My company, Hot Aisle, is.

leetharris
1 replies
6d

I head engineering at Rev and every time we talk to Nvidia about GPUs they tell us to go to you guys. Every time. They really like Lambda!

jedberg
0 replies
5d23h

Very happy to hear this!

Palmik
1 replies
5d22h

Are there any plans to support your on demand cloud in the future? In the past few months there are basically no GPUs available.

My understanding is that your reserved cloud starts at 64 GPUs (8 pods) and with quite a long reservation time frame.

jedberg
0 replies
5d22h

We have new product coming in just a couple weeks to get clusters as small as 16 GPUs per cluster and only a two week minimum reservation, with very short lead times (the first people to reserve will have almost no lead time!).

Also we expect to have a bunch of capacity come online of the on-demand cloud this year. We're getting the GPUs as quickly as we can and racking them as quickly as possible, but we have to wait just like everyone else for the GPUs to roll off the fab. :)

hinkley
0 replies
5d19h

I was at a Christmas party a couple jobs ago and they did a presentation that was supposed to be all good news and instead basically ruined the party for me.

The bit that freaked me out was 2 customers were 60% of revenue.

Those two customers can change your policies in ways you don't want to, because you can't afford to piss them off. And if something happens to one of them you're fucked.

SllX
2 replies
6d

It’s better to have a diversified customer-base rather than being completely beholden to a small number of big customers that might decide they don’t need you down the line.

rjst01
1 replies
5d8h

Indeed, Google and Amazon have already shown an interest in designing their own silicon (TPU and Graviton). Probably Microsoft too, but I'm not aware of an example off the top of my head.

Upstarts are not easily going to be able to pursue that, so NVidia has a strong interest in supporting them.

Workaccount2
1 replies
6d

Google doesn't use nVidia chips, at least for AI. They have their own in house solution.

yardie
0 replies
6d

Nvidia is a shovel maker. If you only have 1-2 shovel buyers, buyers who've started on making their own shovels in-house, then you run the risk of being trapped in a monopsony. You lose 1 shovel buying client and then you might be insolvent before too long.

mym1990
0 replies
6d

I think one of Jensen Huang's big interests is to enable a diverse number of markets that can utilize the technology in a short feedback loop. I don't believe that can be achieved if all of the hardware goes to Microsoft/Google.

m3kw9
0 replies
6d

Nvidia needs these alt cloud so when big co builds their own GPUs they have them to suck up supply. If they are all dead, Nvidia may have no back path

jeffbee
0 replies
5d18h

CoreWeave does not exactly have an arms-length relationship with nvidia.

015a
29 replies
6d

We're absolutely seeing the same thing even in non-GPU traditional servers. Here is an estimated monthly cost breakdown for a new GCP instance type, the n4-standard-2, in us-central1:

    2 vCPU + 8 GB memory: $69.18
    10 GB Hyperdisk Balanced: $0.80
    3060 provisioned IOPS: $15.30
    155 MB/s provisioned throughput: $6.20
    Total: $91.48
Like, you can tear apart that $69/mo for 2vCPU + 8gb of memory, no problem. That's utterly insane. Its Emerald Rapids, so you're paying a premium for new chips, whatever. You can also tear apart the network egress, obviously.

But just look at the SSD pricing. That's a ten gigabyte SSD provisioned for 155MB/s, for $22/month. You can go just outright buy a 256gb NVME at significantly higher bandwidth on Amazon for like $25, flat. The n4 tier instances removed the ability to use their cheaper general-purpose SSDs; you have to use hyperdisks.

I'd be surprised if we don't see the big cloud providers start struggling over the next ten years. I think they engineered planetary-scale systems that are just way too expensive and complex to justify the cost they're charging; ZIRP phenomena.

candiddevmike
13 replies
6d

IMO, the big thing holding back customers (especially smaller ones) from going on-premise these days is networking. Getting a fat pipe similar to what you get with an instance on a cloud provider can be prohibitively expensive (internet service, gear, staff, etc), especially when you want it to be highly available.

tempest_
11 replies
6d

It is really the second one. The minute you want a second site or even HA at a single site the complexity and costs start to explode.

cpill
10 replies
5d23h

k3s is very simple to setup and and nodes to. if the machines are on the same LAN or the internet then it's not such a complex job albeit you need to know the basics of kubernetes.

fwip
8 replies
5d23h

Kubernetes is only the software layer, a lot of the cost is in the hardware/infrastructure, and in the salary for those experts.

fffrantz
6 replies
5d23h

Agreed 100 percent. Software is the easy part. Getting HVAC, power and network up to the levels of cloud providers is difficult to get right and prohibitively expensive.

For instance, the cost for a pair of redundant symmetric gigabit fiber is in the thousands a month and may require tens of thousands of construction costs. These quickly add up, and the upfront costs can quickly reach six figures.

amluto
3 replies
5d21h

There’s a huge middle ground between on-prem and GCP/AWS. You can rent space and connectivity from in very competent datacenter without any of these big fixed costs.

hot_gril
2 replies
5d20h

Can rent the space, but you still have to buy the hardware. Maybe there's money to be made running some low-availability cloud service offering newer hardware.

amluto
1 replies
5d20h

Have you checked the price for a system capable of using two redundant 10Gbps links lately? It’s cheap. You could put gear like this in your closet at home and not feel particularly silly about it, especially if you are willing to buy still-current used enterprise gear.

For that matter, have you checked the price, in qty 1, of a server that will absolutely destroy anything reasonable from a major cloud vendor in terms of IOPS to stick behind that switch or router? Even if you believe the numbers on the website of a major server vendor and forget to ask for a discount, it’s still quite reasonable in comparison to a major cloud.

hot_gril
0 replies
5d19h

Yeah, tends to be. But it's more efficient for multiple customers who don't need the hardware full-time to share it. Someone could set that up without all the expensive HA guarantees and other stuff a regular cloud provides. Maybe was too niche in the past, but now with the AI boom...

saltminer
0 replies
5d22h

I remember seeing a quote for 500/500 metro E from Comcast several years ago. $12k to install, $1.2k/mo. And that only involved laying a few miles of fiber, no redundancy. Dedicated lines are no joke. If you're AWS or GCP, you can be your own ISP and mitigate this to some extent, but that's just the physical connection they save on.

You can always save by going on-prem, assuming you have no uptime requirements. But the moment you sign an SLA, those savings go out the window.

bradstewart
0 replies
5d22h

Not to mention security compliance. If you can afford all of that, seems pretty likely you'll also have SOC2/etc needs. Being able to "ignore" the whole physical security aspect of that stuff is a huge benefit of the cloud.

tempest_
0 replies
5d23h

Yeah, k3s doesn't buy 2 routers, 2 switches, 2 PDUs, 2 firewalls, 2 proxies to sit in front of k3s, 2 internet connections (if those are offered) etc etc the list goes on. Not to mention that HA things like to come in 3s.

Then if you are going to have remember that cloud networking is pretty beefy and if you want k3s to do distributed storage you will need some pretty beefy network hardware.

There are a lot of things hidden in the cloud costs that people forget about.

The one thing running your own stuff does allow you to do is make choices and trade offs. If this switch goes down and we have 6 hours of downtime to replace it what is that worth etc.

hot_gril
0 replies
5d23h

Starting with a basic web backend, you probably have a database that you can't simply run replicas of.

wmf
0 replies
5d21h

Are you talking about colo or an office? Because carrier-neutral colos are pre-wired with plenty of bandwidth that's 10x-100x cheaper than public clouds. Yes, you need routers but the savings elsewhere should pay for them.

ikiris
4 replies
6d

if you don't need hyperdisk, why are you on that type of server?

015a
2 replies
5d23h

Its the only instance class that's on Emerald Rapids. So if you want the best that Intel has to offer, you need to adopt hyperdisks.

But, to be clear: We're not.

acchow
1 replies
5d21h

Curious what the use case is for targeting a specific CPU. These are virtual CPUs anyways, so what benefit does using the latest intel chip offer?

evilduck
0 replies
5d21h

Sometimes the instruction sets change (like the relatively recently added AVX10 extensions) and you have a workload that specifically needs those? I'm just guessing though.

williamstein
0 replies
5d21h

On Google Cloud hyperdisks can be used on h3, c3, c3d, m3, n4 instance types and are required for n4. I.e., you are not allowed to use the n4 instance type without using a hyperdisk.

phh
2 replies
6d

You feel like it's new, I don't. I've evaluated migrating from full blown servers to cloud at various scales for various usages: OTA server for 1M active devices, stat server for 1M active devices, build server for Android (both pure Android that parallelizes and checks out super nearly and vendor Android where they broke all of that). In all those cases the cost of cloud was like an order of magnitude costlier. You're mentioning SSD, the various clouds I tried (I couldn't remember which I tried sorry...) had bad storage performance (it was especially bad when building Android). Also not all those servers were maintained by my employer, when we pay top end 100Gbps server we can pay someone to maintain it for us as well for much lower money than the "cloud tax".

I have no doubt there are great use case for cloud, and that at the proper scale you can negotiate, and I understand that startups might be faster moving with the cloud. But I feel like the highest value cloud provides is 1. replacing capex with opex, and 2. making scaling easier with the direction: cloud is pay first, get questions from direction later. "on-prem" is negotiate with the direction until the service degrades and then scramble to integrate the new server under the pressure.

metadat
1 replies
5d17h

In all those cases the cost of cloud was like an order of magnitude costlier.

Don't discount the value-add of your skills. Giving any of my family members a stack of bare metal servers would serve no end-purpose or web requests. Approximately zero of said servers would even end up plugged in at all, much less operationalized. My family all works in tech or similarly demanding fields. That is, the cost of "the ops team / guy" can be significant for small and medium enterprises.

Despite this, the AWS pricing margins have grown to the point of excess, and are no longer competitive, even against other major cloud players. This is strange to me because the tooling is all interoperable (e.g., Terraform/Vaggi-form), with no fatal friction or lock-in. It is a rare use-case that the latest CPU is worth paying a penny extra for compared to a 3 or 4 generation old chip. The only winner is the cloud provider because it improves their COGS ratio.

TL;DR: the top tier clouds are priced on the order of luxury goods, on par with a Bugatti or G6 jet. That is, unless you ruthlessly track and prune each expense, which costs you time and attention which could have been spent growing the business or at least non-overhead tasks. The eventuality of managing your own DIY fleet of machines is a total headache, as any given computer may work flawlessly for the next ten years or only the next 10 minutes. When it goes south, you're back to being a monkey plugging in cables and scratching your head. This sad activity is only a few people's cup of tea.

Obligatory reference to Warren G's Regulators:

"You've gotta be handy with the steel, if you know what I mean."

https://youtube.com/watch?v=hms5vmekId4

Edit: Sorry for the rambling comment, I suppose it is a more complex topic than I realized prior to crafting the words above.

exceptione
0 replies
5d8h

When it goes south, you're back to being a monkey plugging in cables and scratching your head.

That is if you rent rack space. If you rent bare metal and the bare metal has a failing disk, you file a request to replace said disk, and the data center engineer will be plugging cables, not you.

That means you have to worry about hardware failures, but you don't have the inconvenience of having to be physically present.

wmf
1 replies
5d23h

Hyperdisk is a SAN; it's not comparable to local storage. Unfortunately Google's local SSDs are also overpriced.

015a
0 replies
5d22h

Sure, but you literally cannot use any other kind of SSD with the n4-class instances, and n4 are the only instances they offer on Emerald Rapids; they're advertised as general purpose, flexible, and high performance, basically their workhorses. If you want to use a local SSD you have to use older generation chips.

carbocation
1 replies
6d

Also on GCP, the vCPUs are usually hyperthreads (except for t2d- instance types, and perhaps a few others). So that machine you've described has 1 CPU core.

jiggawatts
0 replies
5d19h

start struggling over the next

They’re already struggling!

In the past, the three big clouds would deploy cutting-edge CPUs at scale ahead of general availability for ordinary rackmount servers.

Now?

The AMD EPYC 9004 series processors were announced over a year ago in March 2023, but are still trickling out as “preview” in selected regions in Azure. Similarly, Intel Xeon fourth-gen CPUs haven’t even been announced by Azure, but Intel is already shipping fifth-generation CPUs!

I suspect that up until a couple of years ago, the usage of public cloud was increasing at such a pace that the providers were buying a truckload of CPUs every six months, so they were keeping up with the latest tech.

They must have had new signups dry up as soon as interest rates went up, and they’re now milking their existing kit instead of expanding with new generation servers.

jeffbee
0 replies
5d18h

The $22/mo hyperdisk is for customers who want effortless durability. The $25 NVMe device is for applications where the data is worthless (not pejorative; there are many worthwhile applications where the data written to storage is of no durable value). It makes sense that there are two different price points.

hot_gril
0 replies
5d23h

Clouds have historically been designed for high-availability workloads, which are very hard to handle yourself. It doesn't always make sense for experimentation or AI training, though they might be trying to optimize more for that now. At past startups, we were fine just buying machines to run on-prem.

zackangelo
9 replies
5d23h

Would like to take a moment to recommend fly.io for GPU workloads.

I've been building a prototype using them for the last couple of weeks and it's been great to use. I didn't have to jump through any hoops or apply for any quota adjustments to get started. And I especially appreciate how easy they make it to automatically scale your GPU instances to zero based on traffic.

indigodaddy
5 replies
5d23h

Wow that is quite decent pricing actually. This makes me want to try to deploy something like llamafile/ollama or similar to fly.io+gpu for my personal on-demand llama/llm whims. (@simonw I’m looking at you— I feel like if you haven’t already done this on fly.io, that you’re probably thinking about it :) ) Seems private enough.. could throw some basic auth on top of it for me and trusted friends/family so I don’t get crazy bills. But privacy-wise with fly.io I think it’s good enough.

Only problem might be everytime it spins up to download the large model might be wasteful as far as getting charged for network/bw usage— wonder if it would be more cost-efficient to have persistent storage or just see how much time and bw it is to download on every cold start…

Palmik
2 replies
5d22h

It's horrible horrible pricing! Their on demand price for A100 is what gets you H100 sxm in other places.

dangrairo
1 replies
5d21h

I am curious what "other places" are you comparing it to.

indigodaddy
0 replies
5d19h

Also we’d have to do apples to apples right? One cannot complain about fly.io gpu pricing and then point to say vast.ai or lambda labs as “evidence” of that. They aren’t at all the same type of service..

indigodaddy
0 replies
5d19h

Oh snap thank you!

nextworddev
1 replies
5d23h

Taking a step back, was it necessary to roll your own LLM (or whatever FM) API as opposed to using an off the shelf API

zackangelo
0 replies
5d18h

Yes we’re running our own inference server because we do nonstandard things in the decoder loop (think logit masking, other types of sampling, etc).

breakingcups
0 replies
5d21h

Hn's opinion of Fly seems to fluctuate a lot. First it was an HN darling, especially with all the high-quality blog posts they were putting out, but on the latest threads there's been a lot of complaining about platform stability and features.

mym1990
8 replies
6d

Isn't the mid/late game of all of these alternative clouds going to be the same as the big cloud players...get customer lock in and then prices slowly creep up until its kind of a wash?

tw04
1 replies
6d

Not every vendor needs Amazon’s eye watering margins to justify their existence. Plenty of people more than happy to make their 12% and call it a day. The difficulty and cost of owning a data center has been grossly exaggerated. It’s just that bean counters would rather lease because it’s easier to massage the numbers than dealing with a depreciating asset.

more_corn
0 replies
6d

Amazon has also spent decades and tons of money building out the ecosystem of AWS: Ebs, alb, rds, etc. You don’t need any of that stuff for training. You just need gpus. If you don’t need it don’t pay for it.

lelanthran
1 replies
6d

Isn't the mid/late game of all of these alternative clouds going to be the same as the big cloud players...get customer lock in and then prices slowly creep up until its kind of a wash?

That might be the plan, but I doubt it's going to work for GPUs like it did for AWS.

There's no moat here, other than the hardware. There's no value-add that the provider can add for free and lock you in.

swores
0 replies
6d

There's no value-add that the provider can add for free and lock you in.

Couldn't you have said that about cloud services like AWS before they existed, that they can rent servers to people but there won't be any lock in?

And equally, just because that's the case with GPU providers right now, is there a reason one or more of them wouldn't be able to develop software which runs on the GPUs to do some of the things people are currently renting GPUs for, or middle-ground software which make it easier to do other stuff on them than just renting plain GPU access, and turning it into optional services (with lock in) on top of the raw hardware rental?

As an example idea: it's good that NVIDIA decided to let as many people use CUDA as possible (and I suspect if they hadn't then they wouldn't have seen nearly as much success), but if they or anyone else releases an equivalent to "CUDA v2" tomorrow, but instead of allowing anyone to download it instead put it behind a billing page with AWS-style pricing that covers both software and GPU, would it not succeed if the software did make thinks easier for people just like AWS does (in some ways)?

edit: I just realised I ignored the "for free" bit of your comment - but it wasn't free for Amazon or Google to build their cloud software either.

TheGRS
0 replies
6d

I would think the late game would be getting bought out by AWS/G Cloud/Azure.

Havoc
0 replies
5d16h

Most are still quite far away from big cloud lock in levels.

Gormo
0 replies
6d

That doesn't seem to be the case for long-tail VPS providers.

It doesn't seem to really be the case for the big cloud providers, either, as most of what they offer is fairly commoditized -- I've been using GCP for years to host Kubernetes clusters, Postgres databases, Redis instances, standard Linux VMs, etc., and while there'd be a cost associated with migrating out of GCP, I don't feel locked into it in any way, as the tools I'm using there are the same tools I'd be using regardless of where they were hosted.

The market for IaaS and for SaaS are very different from each other, and the walled-garden approach doesn't seem to have significant traction in the IaaS world, thankfully.

1123581321
0 replies
6d

Not without some distinctive advantage to the provider's infrastructure beyond raw GPU access. Plus, as more options exist, more SDKs will facilitate switching or distributing workloads to multiple providers as de facto connection standards emerge (like what's happened to S3-compatible storage, as a simple example.)

Gbox4
8 replies
6d

A funny comment from that article:

"On CoreWeave, renting an Nvidia A100 40GB — one popular choice for model training and inferencing — costs $2.39 per hour, which works out to $1,200 per month. On Azure, the same GPU costs $3.40 per hour, or $2,482 per month; on Google Cloud, it’s $3.67 per hour, or $2,682 per month."

Am I missing something? I am sure I'm a bit rusty in math, but I can still handle a calculator. ~720 hours in a month (roughly), and that means CoreWeave would cost $1,720.80 per month, Azure is $2,448 per month, and Google Cloud is $2,642.40 per month.

Why are all of these numbers reported in the article off? Some slightly--Azure and Google Cloud are close, but CoreWeave is off by about 30%. I won't go further into the numbers as to why the author came up with these results, but I'm just wondering if this article was written by AI, which would explain why basic multiplication is incorrect.

fancyfredbot
1 replies
5d23h

The whole thing is nuts. They have the wrong costs multiplied by the wrong time period to get the wrong answers. https://coreweave.com/gpu-cloud-pricing says an A100 40GB NVLink is $2.06 whereas the article says $2.39.

That's $1483.20 a month, whereas the article says $1200 and should say $1720 if they'd got the maths right.

programjames
0 replies
5d23h

Maybe an LLM helped with the math?

Onawa
1 replies
5d23h

I think your guess of AI generation makes sense for the math discrepancy.

Thrymr
0 replies
5d21h

Wonderful that we have evolved large linear algebra models running on expensive computers to the point that they can no longer do basic arithmetic correctly.

umeshunni
0 replies
5d20h

My favorite quote about journalists goes something like "Never trust a journalist's math. If they could do math, they wouldn't have become journalists."

spacebanana7
0 replies
5d23h

Could it be a discount for purchasing an entire month’s worth of capacity? Even if so, such costing plans should be explicit in the article

jmgao
0 replies
5d23h

The CoreWeave number is completely wrong, but Azure and Google Cloud appear to be exactly correct at 730 hours per month, which happens to be the number of hours in 365 / 12 days.

coffeebeqn
0 replies
5d23h

Have you ever asked a LLM to calculate costs for you? This is it exactly what it looks like

maxchehab
5 replies
5d23h

Both Core Weave & Lambda Labs have fairly predatory pricing making it impossible to rent GPUs without a yearly contract.

This doesn't make sense for training models, where a training run is on the scale of days & weeks.

I wished that the techcrunch article mentioned other companies, like sfcompute, which offer hourly compute instead of yearly contracts.

thundergolfer
2 replies
5d22h

Lambda Labs has on-demand GPUs. Just put in a credit card and you're able to launch. I launched an 1x H100 server just 10 minutes ago on Lambda Labs.

The price is also $2.49/hr which does not seem predatory at all.

matroid
1 replies
5d22h

I have never seen 1x H100 available on Lambda Labs. Don't know why though.

williamstein
0 replies
5d20h

I've been checking about twice a week for the last 6 months, and they are very rare, but it does happen. I caught one on video 2 weeks ago! https://youtu.be/NkNx6tx3nu0?t=744

ec109685
0 replies
5d21h

That would make the TechCrunch article highly misleading since AWS and the other clouds offer big savings for reserved instances.

Palmik
4 replies
5d22h

I really don't get this. Most of these low cost providers actually re-rent GPUs from Azure, AWS or GCP (*), yet they offer much better on-demand pricing (as low as $3.8/hr for H100 sxm and $2.5/hr for PCIe).

And it's a fact that you can get even much better on-demand (not to mention reserved) pricing from the big clouds if you're a decent startup with connections.

If one of these clouds offered fair pricing to SMBs, it could be a great bottoms up growth strategy.

(*) Not LambdaLabs afaik, but they rarely have on demand capacity anyway, and you can only get reasonable price with 3 year reservation (which is, surprise surprise, more than the hardware cost).

thundergolfer
2 replies
5d22h

It's the opposite of what you say. Slim to none of the alternative cloud providers re-rent their GPUs.

Coreweave, Fluidstack, Lambda Labs, Paperspace, Cudo Compute, Hydra, Datacrunch.io, Vultr, Crusoe Cloud, SF Compute.

As far as I can tell none of these providers give you a GPU originating in Azure, AWS, or GCP.

Palmik
1 replies
5d8h

RunPod is one of the providers that has some of the best on demand pricing (there are places where you can get H100 PCIe for less, but they typically don't have any capacity) without contracts and doesn't own hardware (or didn't as of few months ago).

gg2ez
0 replies
5d2h

We have a decent amount of h100 sxm availability on demand on tensordock right now at far less than runpods h100 pcie price

claytonjy
0 replies
5d22h

The middlemen you're talking about do two things: buy lots of reserved compute on the hyperscalers, and then pit the hyperscalers against eachother to get better pricing.

If you're reserving thousands of GPUs from the same hyperscaler, even if they're the only cloud you run on, you're not paying the price shown in the calculator. If you have other suppliers, you'll get an even better deal. Then you resell that reserved compute as on-demand compute, somewhere between your costs and what your customers would pay a hyperscaler directly.

rootedbox
2 replies
5d23h

@ Core-weave an A100 40GB NVLink is $2.06 ... the only way they are doing this is by burning investor money. At that price it running it 24/7 it will take way over a year to recoup hardware + electricity cost.

So my suggestion.. dump all your work @ Core-weave.. It's cheaper than buying the hardware yourself let alone the cost of managing it.

latchkey
0 replies
5d22h

the only way they are doing this is by burning investor money... it will take way over a year to recoup hardware

They've been running for years already. They can also offer these lower end gpus at that price cause the higher end ones offset things.

jerrygenser
0 replies
5d22h

Does this assume retail price? Maybe they are getting a discount for buying in bulk

mgdev
2 replies
6d

This will be a boon for Oracle.

Sarkie
1 replies
5d23h

Aren't they killing it in this space?

Nvidia love OCI I thought I read

htrp
0 replies
5d22h

OCI loves Nvidia.... the feeling is less than mutual

la64710
2 replies
5d22h

How are people using these GPU clouds with their data residing in one of the big cloud providers like AWS or Azure? Are they paying for egress to get data onto the GPU clouds?

htrp
1 replies
5d22h

Yes.... you eat the egress cost.

Pretty soon, these specialists will build object storage and all of the other "costs" that the legacy hyperscalers already incur.

claytonjy
2 replies
5d22h

What are folk's experiences with alternative cloud GPUs for inference?

If you're doing a lot of model training, buying GPUs or long-term reservations of GPUs is a no-brainer. But when it comes to inference, latency matters and it gets trickier talking between e.g. your AWS infra and your GPUs somewhere else.

It seems lots of providers can give you enough to get by doing inference in a company's earliest stages. But what if I need hundreds or thousands of A100s during peak usage? Is anyone doing this successfully with a non-hyperscaler?

htrp
1 replies
5d22h

Just go with an inference provider like fireworks/together/modal/baseten

claytonjy
0 replies
5d20h

An issue we've had when looking into some of these is that, they provide a layer of software abstraction we're not looking for. I don't want to use some providers bespoke library to wrap my model code; I just want to use NVIDIA Triton, either by providing an image or by providing a model repo. I only want the inference provider to handle the hardware.

I understand that's exactly what those provider _don't_ want, because it means they can't lock us in. But particularly when comparing an inference provider to GCP, where we already run everything in Triton on GKE, I don't want to rewrite my code just to see what their hardware layer is like.

Another complication is we often run multiple tightly-integrated models for a single application, where having them on the same GPU is critical. This is tricky or impossible in some inference-provider-frameworks.

There's too many options for running the latest LLM, and far fewer for running a bespoke set of fine-tuned models on GPUs.

DrNosferatu
2 replies
5d23h

Besides Alibaba Cloud and AWS, are there other FPGA cloud services?

latchkey
0 replies
5d15h

I've run one in the past, we might do it again if there is enough demand. Let me know!

paxys
1 replies
5d23h

It's funny that "We managed to get our hands on a handful of H100s/A100s" is an actual (and profitable) tech business model right now.

latchkey
0 replies
5d15h

"Selling shovels in a gold rush" has always been a profitable business model.

nextworddev
1 replies
5d23h

I have good reasons to believe that all this capex into GPUs will backfire. Metrics aren’t looking good for AI adoption, and growth rate is slowing down while inference and training costs are plummeting.

latchkey
0 replies
5d15h

The need for more powerful compute is never going to go away and it will continue to grow. Especially around compute that is difficult to obtain and deploy.

lowlevelprog
1 replies
6d

I have switched to buying GPUs and I already see saved costs as compared to cloud renting. Also, networking is fun.

latchkey
0 replies
5d22h

Also, networking is fun.

Let me know how your 400G deployment goes with your vendors VRF implementation.

edgoode
1 replies
5d22h

This is a trend we noticed early last year, so we started building a single console for all these clouds at https://shadeform.ai.

It has been amazing to watch this industry explode, and we believe it is great for consumers. The same instances on Amazon versus these alternative providers are 3x more expensive.

NVIDIA and many hardware providers are leaning into this trend. As clouds become more and more vertically integrated, AMD, NVIDIA, and others will benefit from spreading their hardware to more clouds.

Knowing that these models will not be running in 3 easily controlled clouds may also benefit us in the long run as each provider will have different levels of comfort with models of varying capabilities.

breakingcups
0 replies
5d21h

How does Shadeform make money?

bzmrgonz
1 replies
5d22h

saw one touting itself to be the airbnb for gpu's

latchkey
0 replies
5d22h

That was just a marketplace to resell compute that others are running. A super common practice in the industry.

ReptileMan
1 replies
5d23h

To have cheaper GPUs we need at least four companies to design them and four companies that are able to produce them. Before that you always have cartel like behavior.

latchkey
0 replies
5d22h

Maybe on the consumer low end, but on the enterprise level... it'll always be expensive.

user3939382
0 replies
5d21h

I’ve been using Wasabi for S3-interface cloud storage. Way cheaper and works great.

tehlike
0 replies
6d

Switching to hetzner was one of the good things I did last year!

ptero
0 replies
5d23h

It seems to be that a large part of the AWS business model is the revenue received as a fledgling startup tries to grow (and ideally succeeds) into a large company.

As a small tech startup, an easy button for compute needs is perfectly sensible, as it allows focusing on the product. If the startup grows switching becomes more expensive, so AWS gets its money as long as the amounts are not seen as the main cost driver. That stage, I think (with no hard data), is the AWS sweetspot. The company is paying a lot for AWS, but does not yet want to do a full analysis, hire dedicated cloud cost optimization staff and deal with friction of switching.

If the startup grows stable and profitable it will likely do a proper cost analysis and make AWS bills saner, maybe with a mix of on-prem, AWS and non-AWS cloud services. But that requires a stable period, both in time and in functionality, which is not something that an unprofitable startup has.

I think with the end of ZIRP and tighter access to VC funds the number of startups that can afford losing a lot of money to go through an explosive growth period will shrink, and so will the AWS profits.

nektro
0 replies
3d21h

this is gonna crash big time and idk what's gonna happen on the other side

iotapi322
0 replies
6d

Crypto has this covered with 4090's available on clore and several other platforms.

ilaksh
0 replies
5d22h

If you want a _really_ cheap alternative cloud, look at vast.ai. Not sure you can beat their prices, and they have 4090s and 3090s if you can use those. Something like RunPod might be second place for pricing.

dmyriel
0 replies
5d23h

Hardware is a vendor lock-in. I can’t imagine sinking millions into a massive project and getting stuck with a huge bill.

Then, to migrate away you need all sorts of devops folks and the ability to deal with incompatibilities.

Uncertainty about pricing and the hardware bottleneck is a real problem for our users.

I just raised this point in our blog today.

https://qdrant.tech/blog/are-you-vendor-locked/

devops000
0 replies
5d5h

They are booming in VC money, they are not talking about user usages.

d416
0 replies
5d18h

Skypilot is worth a mention here:

https://github.com/skypilot-org/skypilot

Open source CLI to deploy multiple gpu vm’s on all major cloud providers, with an option to use spot pricing with 1 cheap vm used as a controller to always make sure you have the most inexpensive deployment available with failover and load balancing.

It’s like beating the cloud providers at their own game I wouldn’t be surprised if they banned it.

ctocoder
0 replies
5d23h

https://console.crusoecloud.com/request

They are providing H100s, A100s, L40s for very cheap. They also do not charge for the network usage. I Highly recommend them as the price per flop is unbeatable anyplace, and they have over 4000 gpus to use at a time.

benced
0 replies
5d17h

The San Francisco Compute Company has been plastering their name at the 4th and King Caltrain station with a similar thesis: https://sfcompute.com/

apitman
0 replies
5d23h

Given hyperscaler dominance of the overall public cloud market, which demands vast investments in infrastructure and range of services that make little or no revenue, challengers like CoreWeave have an opportunity to succeed with a focus on premium AI services without the burden of hypercaler-level investments overall

Interesting. GPU-only providers targeting the AI market only need to implement a fraction of the services that AWS does. They don't even need to be geographically distributed. What does it matter if your GPU cluster is on the other side of the planet?

Tollen
0 replies
6d

I wish there were more of these for artists who specifically need GPU rendering for their own animations and heavy duty production. It seems like all of this new infrastructure isn't going towards "classic" GPU use cases.