“Meta spent almost as much as the Manhattan Project on GPUs in today's dollars”

I suspect it's because inflation has been underestimated for the past ~hundred years. Over time, that really compounds.

A fairer way to look at it is how many years of the average citizens salary is it.

The manhattan project was $2B in 1944, or 121,000 median-family-years of work using salary figures from the 1940 census.

Apples stock buyback was $110B in 2024, or 122,910 median-family-years of work using figures from the 2022 census.

So, the inflation figures under represent the cost by a factor of 3.5 over those 80 years.

I think this comic says it all: https://pbs.twimg.com/media/DuJMkjIXcAcRru9?format=jpg

The US was much poorer back then. Much lower standard of living. So wages were also much lower even inflation-corrected. Also there was a war going in and people were drafted and had no choice but to work for a minimal salary. I think that explains a lot of it.

They really weren't cheap. The costs were just pushed to the future. It's estimated that it will take between $16.8 billion and $550 billion to clean up the Hanford site. The Santa Susana facility where Apollo rockets were tested is going to cost billions to clean up (there are other things contributing to the cost there).

It is noteworthy that Wikipedia gives as cost for the Artemis program a total sum of US$93+ billion (2012–2025).

It is a bit difficult to compare, because NASA shuffles around cost, and the plan is that Artemis will do way more (moon base) than Apollo. But Apollo seems to have been not only a technological marvel, but exceptionally well managed?

Regarding Manhattan project it has this mythical quality (the name alone is cool), but the Russians had the a-bomb only 4 years later and smaller countries (South Africa) are able to achieve it too.

I think the Manhattan project had a lot of consideration towards making it economical. (and huge government programs weren't quite so common back then) Plus, I think things that were returned/recycled weren't counted against the total. e.g. the $600 million worth of silver they borrowed to make calutrons, because there was a copper shortage.

Everything about the official inflation calculation is almost totally borked in terms of generating useful information. Like, information that would be useful for someone doing something real, like future historians or Zombie FDR.

There's probably dozens of mechanisms at work to explain "why??!!", but the root cause is - like so many other damn things in America - financialization, from its birth post-1972 with vehicles like EUROBOND, to its "pedal to the metal" moment with all the liquidated commie national assets, to the craptacular commodity runup crisis courtesy of Bush II wrecking the Levant and West Asia. Oh, right, and the financial system ending in 2008. When we seamlessly blended up - legally - private with public risk. Oh yeah. Sixteen years ago.

Anyway, financialization. We don't account for money supply accurately, they're dynamic with price indices now, blabbity blabbity blah blah, 1962 dollars were probably worth waaaayyyyyyyyy more than we think they were.

All this should tell you is that inflation is vastly under-counted.

it's almost as if we've been mislead about that big wasteful central planning government

When you’re under the gun, bang for buck is crucial. When you have a relatively fat, wealthy and lazy society (compared to WWII and even the post-war 60’s) all the costs go up. Eg you could build roads or railways or ships back then. Now there are 100’s of rules, lobbyists everywhere and everyone wants their cut. Financial incentives matter far more now, since survival has been (temporarily) taken care of.

They weren't cheap, money just still had value back then.

What's bizarre is how few people understand how currency has become devalued since its separation from the gold standard, and even more so in the last 5 years. These are the same fools who put their money into "high yield" savings accounts giving them 3% when official inflation is >5%.

Manhattan is technically just the R&D component, the full Manhattan is the trillion+ US has spend on building out the nuclear force and 1T+ more it will spend to modernize in next 30 years.

It’s the difference between doing something transcendental a small number of times and doing something amazing billions of times.

I guess that means we need more government funded Moonshots.

So... A single company has just spent about 0.1% of the US GDP in a pet project, and it's not even the leading one on the market?

That's just incredible, and not a good thing.

- Meta is paying for these GPUs over multiple years, not a single year. Meta capex is only $30B a year, not all of that went to GPUs.

- There's no way Meta is paying retail.

Exceeded by Apple's stock buyback then, $110B.

I feel like this move's the goalposts a little bit. In the recent MFTF thread people compared the entire 70 years of fusion research funding (about 80 Bn USD) to the Apollo program. This is in the context that the research is heavily underfunded given its applications and potential viability. If we agree that society got an Apollo Program's worth of value out of the Apollo Program, what value has society gained from Facebook's resource usage? A more effective manipulation engine?

Probably would be better to look at is as compared to federal government spending. https://fred.stlouisfed.org/series/FYONGDA188S

That’s production, not inflation

If you add together the expenditure (or revenue) of all companies the sum will greatly exceed GDP.

less than a third

That's still an insane amount of money.

And also for perspective: if ad spend globally hits $1T in 2027 then Facebook has put down about 3% today to perhaps own 30%.

But those projects were purely American funded projects. Meta profits from the entire globe. So I don't think a comparison to the US GDP is completely relevant.

There were supposedly 400,000 people behind Apollo 11. I bet you could make some pretty high quality, well crafted stuff today as well, if you had nearly half a million people devoting their lives to building one thing.

   Nvidia: 26,000 people
   TSMC: 73,000
   Intel: 124,000
   AMD: 25,000
   Qualcomm: 50,000
   ASML: 42,000

It's not just the quality and craft of works. Worker's pay didn't increase but productivity did, the wealthy have become ever wealthier, the bottom 90% of earners stopped doing better off, etc. etc. It's kind of crazy how so many measures have gotten steadily worse since then[1].

[1] https://wtfhappenedin1971.com/

Professional management class. Various major reforms toward “more efficient” management, first in manufacturing and such, then applied to knowledge work. Reduced specialization when it comes specifically to bureaucratic tasks in white collar work (everyone’s their own secretary, among other things, now—even doctors)

[edit] anecdotally, as related by people who watched the transition happen, one of the things this new management culture screwed up was management itself, especially lower and middle management. There used to be a gradient of how much time managers spent in meetings with peers and higher-ups, with the lowest ones doing very little of that and mostly focusing on the needs of their subordinates and keeping things running well—after the shift, all levels of management got sucked into a culture of meetings, meetings, meetings, most of them very low-value. Posturing and politics soaked down into lower levels of organizations so-afflicted to a much more extreme degree than before.

* Postwar surplus of machine tools and experience with metal manufacturing processes.

* Lack of injection-molded thermoplastic suitable for much more than toys.

* Relatively high labor costs may have reduced incentives to save money on materials.

* Consumers accustomed to paying higher prices for manufactured goods with the expectation that they would be repairable and last a long time.

* Tube-based electronics tended to require high voltages and heavy transformers or batteries. This dictated the form of the enclosure (metal or wood) and its usage (non-portable). For example, radios made for consumers would typically be incorporated into furniture.

I'm always amazed by the quality and craft of works produced in the periods before roughly the 1960s.

Yeesh. This statement is not defensible.

The fifties-era cookie cutter houses were pretty bad (but at least they moved past using coal and gaslight and knob and tube wiring and so on). The quality of passenger cars has only ever trended upward, with a dip during the combined energy crunch and inflation of the seventies that was remedied pretty quickly. The sixties ushered in the era of solid-state consumer electronics.

My theory is that capitalism wasn't quite as sophisticated and globalized yet, so an important project could collect literally the top 100 engineers in a field together. It's also back in a time when working for the government was a prestigious job that paid top of field.

Additionally the U.S. had probably better education and a stronger work culture. When only 5% of your country goes to college the classes can move really fast.

Survivorship bias. Plenty of failures, incompetence, corruption, and stupidity before.

But it fades into history.

always amazed by the quality and craft of works produced in the periods before roughly the 1960s

Survivorship bias. I have a 19th century Eerie Company cast iron pan. It’s beautiful. It is not representative of most cast iron of the era. We can cast far better iron today, we just don’t devote those production processes to making pans.

I have my theories, but it's probably best that I keep them to myself, and do my best on my projects.

Cheaper labor, less opportunity to have negative impact, and you’re also probably biased and not looking for the projects that failed

Perhaps it's worth mentioning some lesser known big achievements of the Manhattan Project. It's not the scientists. It's Hanford and Oak Ridge. Hanford was not only nuclear reactors but a city of more than 43 000 people out of literally nothing. Oak Ridge had the largest building in the world, K-25. It was only surpassed by the Boeing Everett Factory more than twenty years later. These two sites were also the major cost centers of the project. Science is cheap. Enriching uranium and manufacturing plutonium -- especially when no one did it before -- is not.

10% is overwhelming complexity and 90% is Nvidia's profit margin.

Not a ML expert but I think “can’t” is a strong word. There was a podcast with Mark Zuckerberg where I interpret his remarks as, Meta doesn’t want to directly complete with OpenAI, their existing AI services like recommendation already consume a lot of resources. Ads has and will continue to be a cash cow and LLMs has not generated large revenue yet. OpenAI generated about 2B revenue in 2023 (I’m sure it’ll grow significantly in 2024). Meta has made 40B net income in 2023 so you can see financially, there’s no reason to compete (yet).

Have you tried meta.ai which they offer for free? It’s already very competitive in terms of its abilities, they just don’t care about it or advertise it as much yet.

Remember that Llama400B is still in training and could end up crushing a lot of the use cases for GPT-4

OpenAI is impressive, but IMO Meta is more likely to find product fit.

LLAMA 3 is pretty sweet and some of their vision stuff like segment anything is cutting edge. Facebook just don’t need to publish or make demos since they have a robust path to revenue.

It's not unlikely that the Llama 3 405B that's still training will outperform the best OpenAI model of today.

The Llama 3 70B model in English already beats all models from Google, from Anthropic and it's only bested by the last GPT4 version. (Based on the Open LLM Arena results)

Is this all speculation? Got a source for some of these big claims? Admittedly, it's a fun read.

your handle is also appropriate given the epic amounts of electricity needed to power all of that gear. so many articles on the energy consumption of cryptocurrency, yet so few for model training.

This weapons comparison is really not going anywhere

They used the best 1940's equivalent: IBM card tabulators. This is discussed at some length in Surely You're Joking, Mr. Feynman.

My favorite part:

"When I went back to work on the calculation program, I found it in a mess: There were white cards, there were blue cards, there were yellow cards, and I started to say, "You're not supposed to do more than one problem --­­ only one problem!" They said, "Get out, get out, get out. Wait ­­-- and we'll explain everything."

"So I waited, and what happened was this. As the cards went through, sometimes the machine made a mistake, or they put a wrong number in. What we used to have to do when that happened was to go back and do it over again. But they noticed that a mistake made at some point in one cycle only affects the nearby numbers, the next cycle affects the nearby numbers, and so on. It works its way through the pack of cards. If you have fifty cards and you make a mistake at card number thirty­-nine, it affects thirty-­seven, thirty-­eight, and thirty-­nine. The next, card thirty-­six, thirty­-seven, thirty-­eight, thirty­-nine, and forty. The next time it spreads like a disease.

"So they found an error back a way, and they got an idea. They would only compute a small deck of ten cards around the error. And because ten cards could be put through the machine faster than the deck of fifty cards, they would go rapidly through with this other deck while they continued with the fifty cards with the disease spreading. But the other thing was computing faster, and they would seal it all up and correct it. Very clever. That was the way those guys worked to get speed. There was no other way. If they had to stop to try to fix it, we'd have lost time. We couldn't have got it. That was what they were doing. Of course, you know what happened while they were doing that. They found an error in the blue deck. And so they had a yellow deck with a little fewer cards; it was going around faster than the blue deck. Just when they are going crazy ­­ because after they get this straightened out, they have to fix the white deck ­­ the boss comes walking in. "Leave us alone," they say. I left them alone and everything came out. We solved the problem in time and that's the way it was."

Of course; how else were they going to make the bomb without a bunch of Genius Physics Utilizers?

I've just checked and surprise, right now Nvidia is worth two times more than Meta.

Metaverse was an obvious failure, but Quest is a huge success in the VR/AR market. They managed to create high quality products without it becoming overpriced novelty items (eg Valve Index, Vive, Apple Vision), while being an open ecosystem that supports OpenXR and sideloading.

They made about 40B net income in 2023 (that’s right, net) despite spending billions on the metaverse. They have more than enough money in their war chest to do moonshot ideas.

There is a lot of money to be made selling stuff to Meta for their latest project apparently. Figure out where they want to go and $$$.

...So you need three plates? Machinists have known this for over a century.

If current society collapse sometimes I wonder how many centuries will be needed to recreate 3nm from scratch. Even 1000 years doesn't seem enough.

More like it gets hard to achieve anything else because all other heavy duty sectors of the economy are already monopolized.

When Google entered the telco space you would think they would easily take down the ancient parasites that are AT&T/Verizon/Comcast, but what happened?

Same story with Apple trying to get into auto/banking (credit cards)/content.

So the money is only going to be used in this way. They can't touch Exxon or Monsanto or Pfizer or Boeing. They can't do a thing about housing or education or health or banks.

So burning cash on novelty becomes their default path of least resistance. "Useful areas" have all been walled up.

Meta does not sell personal data, they just use it for ads

The silly AI chatbots are what have been built with their existing compute. This spending is for bigger and even more powerful models.

we are a while into spending energy to shuffle bits around

Facebook’s value is in the social network, rather than IP/data.

Even if the code was all open sourced, the data was portable (e.g like open banking), and the messaging standards were open (like XMPP); the supermajority of users would keep using FB.

See the example of gmail.

For GPUs it could work as you say (except that the incentive for development of expensive and risky technologies would be gone), but I think that wouldn't help that much with things like websites. Most of the value of Facebook is in the network effect. Messenger has more than 1B users and I don't think it's a good app, Telegram is much better IMO (and is also used by almost 1B people in some countries - network effect there), and all Telegram apps are open source - well, backend isn't, but at least you can have modified clones (communicating with Telegram servers) with different features etc. And there is a small open source server implementation made by someone.

Enriched uranium was exploded in a gun bomb for the Manhattan Project, not by implosion. The lenses in Fat Man used two different explosives, in each of which TNT was a minority component.

TSMC has had aot of help. America helping ASML get started to power TSMC the end of a long chain of events here, is enormous amounts of science done by many, over time.

Who really knows, but I tend to think the nuclear physics were pretty involved. I enjoyed the What John Von Neumann Did At Los Alamos, from the evr excellent 3quarksdaily crew, which talks about how his brilliant mind combined with his explosives expertise was key to steering the Manhattan Project towards success. The article enhanced my appreciation for the challenge. https://3quarksdaily.com/3quarksdaily/2020/10/what-john-von-...

The @MetaAI as a group member in group chats has been great. Riff on images together, write stories, have it impersonate authors or write chapters of a (bad) book. Revitalized a few old group chats that went days without messages to constant messing around with the AI.

Zing!

That's actually less than the price they fetch during this shortage:

https://www.cnbc.com/2023/04/14/nvidias-h100-ai-chips-sellin...

https://www.tomshardware.com/tech-industry/artificial-intell...

However, they are likely vastly cheaper in bulk at MSRP, that's true. And Meta apparently got them before the AI hype cycle.

Probably money gets wasted and GPUs go unused.