return to table of content

Ilya Sutskever to leave OpenAI

zoogeny
159 replies
13h37m

Interesting, both Karpathy and Sutskever are gone from OpenAI now. Looks like it is now the Sam Altman and Greg Brockman show.

I have to admit, of the four, Karpathy and Sutskever were the two I was most impressed with. I hope he goes on to do something great.

nabla9
152 replies
11h42m

Top 6 science guys are long gone. Open AI is run by marketing, business, software and productization people.

When the next wave of new deep learning innovations sweeps the world, Microsoft eats whats left of them. They make lots of money, but don't have future unless they replace what they lost.

fnordpiglet
52 replies
11h35m

I don’t feel that OpenAI has a huge moat against say Anthropic. And I don’t know OpenAI needs Microsoft nearly as much as Microsoft needs OpenAI

cm2187
34 replies
10h38m

But is it even clear what is the next big leap after LLM? I have the feeling many tend to extrapolate the progress of AI from the last 2 years to the next 30 years but research doesn't always work like that (though improvements in computing power did).

huygens6363
9 replies
8h49m

Not saying it’s going to be the same, but I’m sure computing progress looked pretty unimpressive from, say, 1975 to 1990 for the uninitiated.

By the 90s they were still mainly used as fancy typewriters by “normal” people (my parents, school, etc) although the ridiculous potential was clear from day one.

It just took a looong time to go from pong to ping and then to living online. I’m still convinced even this stage is temporary and only a milestone on the way to bigger and better things. Computing and computational thought still has to percolate into all corners of society.

Again not saying “LLM’s” are the same, but AI in general will probably walk a similar path. It just takes a long time, think decades, not years.

Edit: wanted to mention The Mother of All Demos by Engelbart (1968), which to me looks like it captures all essential aspects of what distributed online computing can do. In a “low resolution”, of course.

dgacmu
5 replies
7h46m

Computing progress from 78 to 90 was mind-blowing.

1978: the apple ][. 1mhz 8 bit microprocessor, 4kb of ram, monochrome all-,caps display.

1990:Mac IIci, 25mhz 32-bit CPU, 4MB ram, 640x480 color graphics and an easy to use GUI.

Ask any of us who used both of these at the time: it was really amazing.

jameshart
2 replies
7h21m

They were amazing, and the progress was incredible, but both of those computers - while equally exciting and delightful to people who saw the potential - were met with ‘but what can I actually use it for?’ from the vast majority of the population.

By 1990 home computer use was still a niche interest. They were still toys, mainly. DTP, word processing and spreadsheets were a thing, but most people had little use for them - I had access to a Mac IIci with an ImageWriter dot matrix around that time and I remember nervously asking a teacher whether I would be allowed to submit a printed typed essay for a homework project - the idea that you could do all schoolwork on a computer was crazy talk. By then, tools like Mathematica existed but as a curiosity not an essential tool like modern maths workbooks are.

The internet is what changed everything.

jorvi
0 replies
5h48m

The internet is what changed everything.

Broadband. Dial-up was still too much of an annoyance, too expensive.

Once broadband was ubiquitous in the US and Europe, that's when the real explosion of computer usage happened.

6510
0 replies
22m

A big obstacle was that everything was on paper. We still had to do massive amounts of data entry.

For some strange reason html forms is an incredibly impotent technology. Pretty standard things are missing like radioboxes with an other text input. 5000+ years ago the form labels aligned perfectly with the value.

I can picture it already, ancient Mesopotamia, the clay tablet needs name and address fields for the user to put their name and address behind. They pull out a stamp or a roller.

Of course if you have a computer you can have stamps with localized name and address formatting complete with validation as a basic building block of the form. Then you have a single clay file with all the information neatly wrapped together. You know, a bit like that e-card no one uses only without half data mysteriously hidden from the record by some ignorant clerk saboteur.

We've also failed to hook up devices to computers. We went from the beautiful serial port to IoT hell with subscriptions for everything. One could go on all day like that, payments, arithmetic, identification, etc much work still remains. I'm unsure what kind of revolution would follow.

Talking thinking machines will no doubt change everything. That people believe it is possible is probably the biggest driver. You get more people involved, more implementations, more experiments, more papers, improved hardware, more investments.

huygens6363
0 replies
6h58m

I agree. Likewise, early AI models to GPT4 is breathtaking progress.

Regular people shrug and say, yeah sure, but what can I do with it. They still do this day.

fidotron
0 replies
6h1m

Honestly mobile totally outstrips this.

One day at work about 10-15 years ago I looked at my daily schedule and found that on that day my team were responsible for delivering a 128kb build of Tetris and a 4GB build of Real Racing.

dtech
1 replies
8h7m

mobile internet and smartphones were the real gamechanger here, which were definitely not linear.

They became viable in the 2000's, let's say 2007 with the iPhone, and by late 2010's everyone was living online, so "decades" is a stretch.

huygens6363
0 replies
6h57m

To make the 2000s possible, decades of relatively uninteresting progress was made. It quickly takes off from there.

dmd
0 replies
6h9m

It was only 11 years from pong to ping.

benterix
9 replies
10h5m

Extrapolating 2 years might give you a wrong idea, but extrapolating the last year suggests making another leap that was GPT3 or GPT4 is much, much more difficult. The only considerable breakthrough I can think of is Google's huge context window which I hope will be the norm one day, but in terms of actual results they're not mind-blowing yet. We see little improvements everyday and for sure there will be some leaps, but I wouldn't count on a revolution.

trashtester
8 replies
7h51m

Unlike AI in the past, there is now massive amounts of money going into AI. And the number things humans are still doing significantly better than AI is going down continously now.

If something like Q* is provided organically with GPT5 (which may have a different name), and allows proper planning, error correction and direct interaction with tools, that gaps is getting really close to 0.

varjag
7 replies
7h31m

AI in the past (adjusted for 1980s) was pretty well funded. It's just that fundamental scientific discovery bears little relationship to the pallets of cash.

mark_l_watson
3 replies
5h56m

Funding in the 1980s was sometimes very good. My company bought me an expensive Lisp Machine in 1982 and after that, even in “AI winters” it mostly seemed that money was available.

AI has a certain mystique that helps get money. In the 1980s I was on a DARPA neural network tools advisory panel, and I concurrently wrote a commercial product that included the 12 most common network architectures. That allowed me to step in when a project was failing (a bomb detector we developed for the FAA) that used a linear model, with mediocre results. It was a one day internal consult to provide software for a simple one hidden layer backprop model. During that time I was getting mediocre results using symbolic AI for NLP, but the one success provided runway internally in my company to keep going.

trashtester
2 replies
5h46m

That funding may have felt good at the time compared to some other academic fields.

But compared to the 100s of billions (possibly trillions, globally) that is currently being plowed into AI, that's peanuts.

I think the closest recent analogy to the current spending on AI, was the nuclear arms race during the cold war.

If China is able to field ASI before the US even have full AGI, nukes may not matter much.

mark_l_watson
1 replies
4h59m

You are right about funding levels, even taking inflation into account. Some of the infrastructure, like Connection Machines and Butterfly Machines seemed really expensive at the time though.

trashtester
0 replies
4h33m

They only seem expensive because they're not expected to generate a lot of value (or military/strategic benefit).

Compare that the 6+ trillions that were spent in the US alone on nuclear weapons, and then consider, what is of greater strategic importance: ASI or nukes?

trashtester
2 replies
5h52m

AI in the past (adjusted for 1980s) was pretty well funded.

A tiny fraction of the current funding. 2-4 orders of magnitude less.

It's just that fundamental scientific discovery bears little relationship to the pallets of cash

Heavy funding may not automatically lead to breakthroughs such as Special Relativity or Quantum Mechanics (though it helps there too). But once the most basic ideas are in place, massive is what causes the breakthroughs like in the Manhatten Project and Apollo Program.

And it's not only the money itself. It's the attention and all the talent that is pulled in due to that.

And in this case, there is also the fear that the competition will reach AGI first, whether the competition is a company or a foreign government.

It's certainly possible the the ability to monetize the investments may lead to some kind of slowdown at some point (like if there is a recession).

But it seems to me that such a recession will have no more impact on the development of AGI than the dotcom bust had for the importance of the internet.

varjag
1 replies
5h29m

A tiny fraction of the current funding. 2-4 orders of magnitude less.

Operational costs were correspondingly lower, as they didn't need to pay electricity and compute bills for tens of millions concurrent users.

But once the most basic ideas are in place, massive is what causes the breakthroughs like in the Manhatten Project and Apollo Program.

There is no reason to think that the ideas are in place. It could be that the local optimum is reached as it happened in many other technology advances before. The current model is mass scale data driven, the Internet has been sucked dry for data and there's not much more coming. This may well require a substantial change in approach and so far there are no indications of that.

From this pov monetization is irrelevant, as except for a few dozen researchers the rest of the crowd are expensive career tech grunts.

trashtester
0 replies
4h43m

There is no reason to think that the ideas are in place.

That depends what you mean when you say "ideas". If you consider ideas at the level of transformers, well then I would consider those ideas of the same magnitude as many of the ideas the Manhatten Project or Apollo Program had to figure out on the way.

If you mean ideas like going from expert system to Neural Networks with backprop, then that's more fundamental and I would agree.

It's certainly still conceivable that Penrose is right in that "true" AGI requires something like microtubules to be built. If so, that would be on the level of going from expert systems to NNs. I believe this is considered extremely exotic in the field, though. Even LeCun probably doesn't believe that. Btw, this is the only case where I would agree that funding is more or less irrelevant.

If we require 1-2 more breakthroughs on par with Transformers, then those could take anything from 2-15 years to be discovered.

For now, though, those who have predicted that AI development will mostly be limited by network size and the compute to train it (like Sutskever or implicitly Kurzweil) have been the ones most accurate in the expected rate of progress. If they're right, then AGI some time between 2025-2030 seems most likely.

Those AGI's may be very large, though, and not economical to run for a wider audience until some time in the 30's.

So, to summarize: Unless something completely fundamental is needed (like microtubules), which happens to be a fringe position, AGI some time between 2025 and 2040 seems likely. The "pessimists" (or optimists, in term of extinction risk) may think it's closer to 2040, while the optimists seem to think it's arriving very soon.

bsenftner
8 replies
7h51m

The majority of the developers may know what LLMs are in an abstract sense, but I meet very few that really realize what these are. These LLMs are an exponential leap in computational capability. The next revolution is going to be when people realize what we have already, because it is extremely clear the majority do not. RAG? Chatbots? Those applications are toys compared to what LLMS can do right now, yet everyone is dicking around making lusty chatbots or naked celebrities in private.

sheeshkebab
7 replies
7h45m

The next revolution is going to be when people realize what we have already

Enlighten us

bsenftner
5 replies
7h21m

It is both subtle and obvious, yet many are missing this: if you want/need a deep subject matter expert in virtually any subject, write a narrative biography describing your expert using the same language that expert would use to describe themselves; this generates a context within the LLM carrying that subject matter expertise, and now significantly higher quality responses are generated. Duplicate this process for several instances of your LLM, creating a home brewed collection of experts, and have them collectively respond to one's prompts as a group privately, and then present their best solution. Now there is a method of generating higher reliability responses. Now turn to the fact that the LLMs are trained on an Internet corpus of data that contains the documentation and support forums for every major software application; using the building blocks described so far, it is not difficult at all to create agents that sit between the user and pretty much every popular software application and act as co-authors with the user helping them use that application.

I have integrated 6 independent, specialized "AI attorneys" into a project management system where they are collaborating with "AI web developers", "AI creative writers", "AI spreadsheet gurus", "AI negotiators", "AI financial analysts" and an "AI educational psychologist" that looks at the user, the nature and quality of their requests, and makes a determination of how much help the user really needs, modulating how much help the other agents provide.

I've got a separate implementation that is all home solar do-it-yourself, that can guide someone from nothing all the way to their own self made home solar setup.

Currently working on a new version that exposes my agent creation UI with a boatload of documentation, aimed at general consumers. If one can write well, as in write quality prose, that person can completely master using these LLMs to superior results.

itsoktocry
4 replies
6h40m

I have integrated 6 independent, specialized "AI attorneys" into a project management system where they are collaborating with "AI web developers", "AI creative writers", "AI spreadsheet gurus", "AI negotiators", "AI financial analysts" and an "AI educational psychologist" that looks at the user, the nature and quality of their requests, and makes a determination of how much help the user really needs, modulating how much help the other agents provide.

Ah yes, "it's so obvious no one sees it but me". Until you show people your work, and have real experts examining the results, I'm going to remain skeptical and assume you have LLMs talking nonsense to each each other.

bsenftner
2 replies
6h29m

I'm at a law firm, this is in use with attorneys to great success. And no, none of them are so dumb they do not verify the LLM's outputs.

mrtranscendence
1 replies
4h18m

How can no one see what we have today? You only need six instances of an LLM running at the same time, with a system to coordinate between them, and then you have to verify the results manually anyway. Sign me up!

datameta
0 replies
3h37m

If a certain percent of the work is completed through research synthesis and multiple perspective alignment, why is said novel approach not worth lauding?

I've created a version of one of the resume GPTs that analyses my resume's fit to a position when fed the job description along with a lookup of said company. I then have a streamlined manner in which it points out what needs to be further highlighted or omitted in my resume. It then helps me craft a cover letter based on a template I put together. Should I stop using it just because I can't feed it 50 job roles and have it automatically select which ones to apply to and then create all necessary changes to documents and then apply?

bsenftner
0 replies
6h32m

The point is these characters are not doing the work for people, it co-authors the work with them. It's just like working with someone highly educated but with next to no experience - they're a great help, but ya gotta look at their work to verify they are on track. This is the same, but with a collection of inexperienced phds. The LLMs really are idiot savants, and when you treat them like that they respond with expectations better.

JKCalhoun
0 replies
7h8m

I agree with OP, I think we still have no idea yet what dreams may come of the LLM's we have today. So no one will be able to "enlighten us" — perhaps not until we're looking in the rear-view mirror.

I would say instead, stay tuned.

throwthrowuknow
0 replies
3h0m

I don’t think we’re even close to exhausting the potential of transformer architectures. gpt4o shows that a huge amount can be gained by implementing work done on understanding other media modalities. There’s a lot of audio that they can continue to train on still and the voice interactions they collect will go into further fine tuning. Even after that plays out there will be video to integrate next and thanks to physics simulations and 3D rendering there is a potentially endless and readily generated license free supply of it, at least for the simpler examples. For more complex real world video they could just set up web cams in public areas around the world where consent isn’t required by law and collect masses of data every second. Given that audio seems to have enabled emotional understanding and possibly even humour, I can’t imagine what all might fall out of video. At the least it’s going to improve reasoning since it will involve predicting cause and effect. There are probably a lot of others you could add though we don’t have large datasets for them.

renegade-otter
0 replies
5h0m

In just a couple of generations each training cycle will cost close to $10 billion. That's a lot of cheddar that you have to show ROI on.

makestuff
0 replies
5h10m

IMO their next big leap will be to get it cheap enough and integrated with enough real time sources to become the default search engine.

You can really flip the entire ad supported industry upside down if you integrate with a bunch of publishers and offer them a deal where they are paid every time an article from their website is returned. If they make this good enough people will pay $15-20 a month for no ads in a search engine.

eitally
0 replies
2h23m

I don't think it particularly matters right now (practically speaking). It's going to take years for businesses and product companies to commoditize applications of LLMs, so while it's valuable for the Ilyas & Andrejs of the world to continue the good work of hard research, it's the startups, hyperscalers and SaaS companies who are creating business applications for LLMs that going to be the near term focus.

EGreg
0 replies
7h45m

LLM is all you need

Attention and scale is all you need

Anything else you do will be overtaken by LLM when it builds its internal structures

Well, LLM and MCTS

The rest is old news. Like Cyc

gunalx
7 replies
11h33m

I Don't know. Being able to get azure credits has payed out really well for openai as a business in constant need of computer.

fnordpiglet
6 replies
11h33m

Which is a very short term advantage. And Anthropic gets aws credits which would you rather have?

luma
4 replies
7h32m

Given Amazon's no show in the AI space? Azure. By a mile.

cthalupa
3 replies
6h22m

Except if you're Anthropic or OpenAI you don't care about what your compute provider has done in the AI space - you care about the compute power they can give you.

cthalupa
1 replies
5h57m

But how many of those are ordered specifically for OpenAI, and are on order as a result of them to begin with? Do you think if we were in a parallel universe where OpenAI ended up partnering with Google or Amazon instead, the GPU shipments would look the same? I think they would reflect wherever OpenAI ended up doing all their compute showing a pretty similar lion's share.

Your claim was that people should care about compute based on what the provider has done in the AI space, but Microsoft was pretty far behind on that side until OpenAI - Google was really the only player in town. Should they have wanted GCP credits instead? Do you care about their AI results or the ex post facto GPU shipments?

Or, if what you actually want to argue is that Anthropic would be able to get more GPUs with Azure than AWS or GCP then this is a different argument which is going to require different evidence than raw GPU shipments.

luma
0 replies
4h54m

The claim being implied was that Anthropic was in a better position because they had partnered with AWS versus Azure and thus they would have more access to GPU.

That isn't the case, at all. All I'm stating is what the chart clearly shows - Azure has invested deeply in this technology and at a rate that far exceeds AWS.

Closi
0 replies
7h55m

Never discount the value of short term advantages.

Being first at the start (i.e. first mover advantage) is huge.

nabla9
6 replies
11h11m

There are no moats in deep learning, everything changes so fast.

They have the next iteration of GPT Sutskever helped to finalize. OpenAI lost it's future unless they find new same caliber people.

sk11001
2 replies
10h6m

They have the next iteration of GPT Sutskever helped to finalize

How do you know that they have the next GPT?

How do you know what Sutskever contributed? (There was talk that the most valuable contributions came from the less well known researchers not from him)

nabla9
1 replies
7h46m

sha256:e33135417f7f5b8f4a1c98c28cf26330bea4cc6b120765f59f5d518ea0ce80e5

jacooper
0 replies
29m

What should this mean?

criddell
2 replies
6h15m

Isn't access to massive datasets and computation the moat? If you and your very talented friends wanted to build something like GPT-4, could you?

It's going to get orders of magnitude less expensive, but for now, the capital requirements feel like a pretty deep moat.

Gud
1 replies
1h48m

How do you know massive datasets are required? Just because that’s how current LLMs operate, doesn’t mean it’s necessarily the only solution.

datameta
0 replies
47m

Then the resources needed to discover an alternative to brute-forcing a large model are a huge barrier.

I think academia and startups are currently better suited to optimize tinyml and edge ai hardware/compilers/frameworks etc.

boringg
0 replies
4h40m

OpenAI most definitely needs the compute from MSFT. It could certainly swap out to another service but given that microsoft invested via credits it would be problematic. They have enmeshed their future.

bamboozled
0 replies
10h25m

They seem to have a huge "money moat" now. Partnerships with Apple and MS mean they have a LOT of money to try a lot of things I guess.

Before the Apple partnership, maybe it seemed like the moat was shrinking, but I'm tno sure now.

Likely they have access to a LOT of data now too.

fsloth
36 replies
7h47m

If we look at history of innovation and invention it’s very typical the original discovery and final productization are done by different people. For many reasons, but a lot of them are universal I would say.

E.g. Oppenheimer’s team created the bomb, then following experts finetuned the subsequent weapon systems and payload designs. Etc.

fprog
33 replies
7h15m

Except OpenAI hasn’t yet finished discovery on its true goal: AGI. I wonder if they risk plateauing at a local maximum.

Zambyte
29 replies
6h29m

I'm genuinely curious: what do you expect an "AGI" system to be able to do that we can't do with today's technology?

jagrsw
9 replies
6h17m

Some first ideas coming to mind:

Engineering Level:

  Solve CO2 Levels
  End sickness/death
  Enhance cognition by integrating with willing minds.
  Safe and efficient interplanetary travel.
  Harness vastly higher levels of energy (solar, nuclear) for global benefit.
Science:

  Uncover deeper insights into the laws of nature.
  Explore fundamental mysteries like the simulation hypothesis, Riemann hypothesis, multiverse theory, and the existence of white holes.
  Effective SETI
 
Misc:

  End of violent conflicts
  Fair yet liberal resource allocation (if still needed), "from scarcity to abundance"

TimPC
3 replies
3h59m

The problem with CO2 levels is that no one likes the solution not that we don't have one. I highly doubt adding AGI to the mix is going to magically make things better. If anything we'll just burn more CO2 providing all the compute resources it needs.

People want their suburban lifestyle with their red meat and their pick-up truck or SUV. They drive fuel inefficient vehicles long-distances to urban work environments and they seem to have very limited interest in changing that. People who like detached homes aren't suddenly affording the rare instances of that closer to their work. We burn lots of oil because we drive fuel inefficient vehicles long distances. This is a problem of changing human preferences which you just aren't going to solve with an AGI.

jagrsw
2 replies
3h4m

Assuming embedded AI in every piece of robotics - sometimes directly, sometimes connected to a central server (this is doable even today) - it'll revolutionize industries: human-less mining, processing, manufacturing, services, and transportation. These factories would eventually produce and install enough solar power or build sufficient nuclear plants and energy infrastructure, making energy clean and free.

With abundant electric cars (at this future point in time) and clean electricity powering heating, transportation, and manufacturing, some AIs could be repurposed for CO2 capture.

It sounds deceptively easy, but from an engineering standpoint, it likely holds up. With free energy and AGI handling labor and thinking, we can achieve what a civilization could do and more (cause no individual incentives come into play).

However, human factors could be a problem: protests (luddites), wireheading, misuse of AI, and AI-induced catastrophes (alignment).

breuleux
1 replies
2h12m

Having more energy is intrinsically dangerous, though, because it's indiscriminate: more energy cannot enable bigger solutions without also enabling bigger problems. Energy is the limiting factor to how much damage we can do. If we have way more of it, all bets are off. For instance, the current issue may be that we are indirectly cooking the planet through CO2 emissions, so capturing that sounds like a good idea. But even with clean energy, there is a point where we would cook the planet directly via waste heat of AI and gizmos and factories and whatever unforeseen crap we'll conjure just because we can. And given our track record I'm far from confident that we wouldn't do precisely that.

cityofdelusion
0 replies
1h19m

This exactly. Every self replicating organism will eventually use all the energy available to it, there will never be an abundance. From the dawn of time, mankind has similarly used every bit of energy it generates. From the perspective of a subsistence farmer in the 1600s, if you told them how much energy would be available in 400s year they would think we surely must live in paradise with no labor. Here we are, still metaphorically tilling the land.

Zambyte
2 replies
6h6m

Do you believe the average human has general intelligence, and do you believe the average human can intellectually achieve these things in ways existing technology cannot?

jagrsw
1 replies
5h54m

Yes, considering that AI operates differently from human minds, there are several advantages:

  AI does not experience fatigue or distractions => consistent performance.
  AI can scale its processing power significantly, despite the challenges associated with it (I understand the challenges)
  AI can ingest and process new information at an extraordinary speed.
  AIs can rewrite themselves
  AIs can be multiplicated (solving scarcity of intelligence in manufacturing)
  Once achieving AGI, progress could compound rapidly, for better or worse, due to the above points.

Jensson
0 replies
1h8m

The first AGI will probably take way too much compute to have a significant effect, unless there is a revolution in architecture that gets us fast and cheap AGI at once the AGI revolution will be very slow and gradual.

A model that is as good as an average human but costs $10 000 per effective manhour to run is not very useful, but it is still an AGI.

jprete
0 replies
5h52m

The incentives aren't structured properly for these things to happen, it has always been a sci-fi fairy tale that AGI would achieve these things.

davidgerard
0 replies
3h52m

now do magical flying unicorn ponies, which I understand there is also considerable demand for

Jensson
6 replies
5h38m

An AGI could replace human experts at tasks that doesn't require physical embodiment, like diagnosing patients, drafting contracts, doing your taxes etc. If you still do those manually and not just offload all of it to ChatGPT then you would greatly benefit from a real AGI that could do those tasks on their own.

And no, using ChatGPT like you use a search engine isn't ChatGPT solving your problem, that is you solving your problem. ChatGPT solving your problem would mean it drives you, not you driving it like it works today. When I hired people to help me do taxes they told me what papers they needed and then they did my taxes correctly without me having to look it through and correct them, an AGI would work like that for most tasks, it means you no longer need to think or learn to solve problems since the AGI solves them for you.

xdennis
3 replies
4h12m

An AGI could replace human experts at tasks that doesn't require physical embodiment, like diagnosing patients, drafting contracts, doing your taxes etc.

How come the goal posts for AGI are always the best of what people can do?

I can't diagnose anyone, yet I have GI.

Reminds me of:

Will Smith: Can a robot write a symphony? Can a robot take a blank canvas and turn it into a masterpiece?

I Robot: Can you?
Jensson
2 replies
3h0m

How come the goal posts for AGI are always the best of what people can do?

Not the best, I just want it to be able to do what average professionals can do because average humans can become average professionals in most fields.

I can't diagnose anyone, yet I have GI.

You can learn to, an AGI system should be able to learn to as well. And since we can copy AGI learning it means that if it hasn't learned to diagnose people yet then it probably isn't an AGI, because an AGI should be able to learn that without humans changing its code and once it learned it once we copy it forever and now the entire AGI knows how to do it.

So, the AGI should be able to do all the things you could do if we include all versions of you that learned different fields. If the AGI can't do that then you are more intelligent than it in those areas, even if the singular you isn't better at those things than it is.

For these reasons it makes more sense to compare an AGI to humanity rather than individual humans, because for an AGI there is no such thing as "individuals", at least not the way we make AI today.

HeatrayEnjoyer
1 replies
1h47m

People with severe Alzheimer's cannot learn, but still have general intelligence.

Jensson
0 replies
1h42m

If they can't learn then they don't have general intelligence, without learning there are many problems you wont be able to solve that average (or even very dumb) people can solve.

Learning is a core part to general intelligence, as general intelligence implies you can learn about new problems so you can solve those. Take away that and you are no longer a general problem solver.

Zambyte
1 replies
5h24m

Let's take a step back from LLMs. Could you accept the network of all interconnected computers as a generally intelligent system? The key part here that drives me to ask this is:

ChatGPT solving your problem would mean it drives you, not you driving it like it works today.

I had a very bad Reddit addiction in the past. It took me years of consciously trying to quit in order to break the habit. I think I could make a reasonable argument that Reddit was using me to solve its problems, rather than myself using it to solve mine. I think this is also true of a lot of systems - Facebook, TikTok, YouTube, etc.

It's hard to pin down all computers as an "agent" in the way we like to think about that word and assign some degree of intelligence to, but I think it is at least an interesting exercise to try.

Jensson
0 replies
5h17m

Companies are general intelligences and they use people, yes. But that depends on humans interpreting that data reddit users generates and updating their models, code and algorithms to adapt to that data, the computer systems alone aren't general intelligences if you remove the humans.

An AGI could run such a company without humans anywhere in the loop, just like humans can run such a company without an AGI helping them.

I'd say a strong signal that AGI has happened are large fully automated companies without a single human decisionmaker in the company, no CEO etc. Until that has happened I'd say AGI isn't here, if that happens it could be AGI but I can also imagine a good enough script to do it for some simple thing.

tsimionescu
3 replies
5h53m

The simplest answer, without adding any extraordinary capabilities to the AGI that veer into magical intelligence, is to have AI assistants that can seemlessly interact with technology the way a human assistant would.

So, if you want to meet with someone, instead of opening you calendar app and looking for an opening, you'd ask your AGI assistant to talk to their AGI assistant and set up a 1h meeting soon. Or, instead of going on Google to find plane tickets, you'd ask you AGI assistant to find the most reasonable tickets for a certain date range.

This would not require any special intelligence more advanced than a human's, but it does require a very general understanding of the human world that is miles beyond what LLMs can achieve today.

Going only slightly further with assumptions about how smart an AGI would be, it could revolutionize education, at any level, by acting as a true personalized tutor for a single student, or even for a small group of students. The single biggest problem in education is that it's impossible to scale the highest quality education - and an AGI with capabilities similar to a college professor would entirely solve that.

sebastiennight
0 replies
3h34m

The examples you're providing seem to have been thoroughly solved already.

I'm at the European AI Conference for our startup tomorrow, and they use a platform that just booked me 3 meetings automatically with other people there based on our availability... It's not rocket science.

And you don't even need those narrow tools. You could easily ask GPT-4o (or lesser versions) something along the lines of :

"you're going to interact with another AI assistant to book meetings for me: [here would be the details about the meeting]. Come up with a protocol that you'll send to the other assistant so it can understand what the meetings are about, communicate you their availability, etc. I want you to come up with the entire protocol, send it, and communicate with the other assistant end-to-end. I won't be available to provide any more context ; I just want the meeting to be booked. Go."
duped
0 replies
2h22m

The single biggest problem in education is that it's impossible to scale the highest quality education

Do you work in education? Because I don't think many who do would agree with this take.

Where I live, the single biggest problem in education is that we can't scale staffing without increasing property taxes, and people don't want to pay higher property taxes. And no, AGI does not fix this problem, because you need staff to be physically present in schools to deal with children.

Even if we had an AGI that could do actual presentation of coursework and grading, you need a human being in there to make sure they behave and to meet the physical needs of the students. Humans aren't software to program around.

Zambyte
0 replies
5h34m

This is definitely an interesting way to look at it. My initial reaction is to consider that I can enhance the capabilities of a system without increasing its inteligence. For example, if I give a monkey a hammer, it can do more than it could do when it didn't have the hammer, but it is not more intelligent (though it could probably learn things by interacting with the world with the hammer). That leads me to think: can we enhance the capabilities of what we call "AI systems" to do these things, without increasing their intelligence? It seems like you can glue GPT-4o to some calendar APIs to do exactly this. This seems more like an issue of tooling rather than an issue of intelligence to me.

I guess the issue here is: can a system be "generally intelligent" if it doesn't have access to general tools to act on that intelligence? I think so, but I also can see how the line is very fuzzy between an AI system and the tools it can leverage, as really they both do information processing of some sort.

Thanks for the insight.

hskalin
2 replies
6h10m

self consciousness?

Zambyte
1 replies
6h7m

And how do you test for that?

datameta
0 replies
51m

Probably the best thing we can do at the moment is compile a list of ways in which we shouldn't test for intelligence.

Symmetry
2 replies
5h16m

A working memory that can preserve information indefinitely outside a particular context window and which can engage in multi-step reasoning that doesn't show up in its outputs.

GPT4o's context window is 128k tokens which is somewhere on the order of 128kB. Your brain's context window, all the subliminal activations from the nerves in your gut and the parts of your visual field you aren't necessarily paying attention to is on the order of 2MB. So a similar order of magnitude though GPT has a sliding window and your brain has more of an exponential decay in activations. That LLMs can accomplish everything they do just with what seems analogous to human reflex rather than human reasoning is astounding and more than a bit scary.

datameta
1 replies
3h51m

I'm curious what resources led you to calculate a 2MB context window, I'd like to learn more.

Symmetry
0 replies
3h17m

Looking up an estimate of the brain's input bandwidth at 10 million bits per second and multiplying by the second or two a subliminal stimuli can continue to affect a person's behavior. This is a very crude estimate and probably an order of magnitude off, but I don't think many orders of magnitude off.

amirhirsch
1 replies
4h48m

Prove the Riemann Hypothesis

Zambyte
0 replies
1h5m

Can you do that, are you not generally intelligent, or is that a bad metric for general intelligence? At least one of these is true.

anthonypasq
1 replies
2h53m

how do you know that is actually currently their true goal. it appears to me that the goal has shifted, and the people that cared about the original goal are leaving.

shwaj
0 replies
1h56m

I’m pretty sure Altman still cares about full AGI, if only because of the power inherent in achieving that goal.

insane_dreamer
0 replies
3h53m

That's probably a good thing

OJFord
1 replies
7h3m

If we look at history of innovation and invention it’s very typical the original discovery and final productization are done by different people.

You don't really need to look at history, that's basically science vs engineering in a nutshell.

Maybe history could tell us if that's an accident or a division that arose out of 'natural' occurrence, but I suppose a question for an economist or psychologist or sociologist how natural that could really be anyway or if it's biased by e.g. academics not financially motivated because it happens that there isn't money there; so they don't care about productising; leaving it for others who are so motivated.

fsloth
0 replies
6h30m

With abombs for weapons systems design they needed people who just got huge kicks out of explosions (not kidding here). I guess it’s partially about personal internal motivations, and it might be more of a chance wether the thing you are intrinsically motivated to do falls under engineering or science (in both cases you get the feeling the greats did stuff they wanted to do regardless of the categorizations applied to their discipline - you get more capital affinity in engineering ofc).

ralfd
19 replies
10h38m

How important are top science guys though? OpenAI has a thousand employees and almost unlimited money, and llm are better understood, I would guess continous development will beat singular genius heroes?

benterix
10 replies
10h14m

OpenAI has a thousand employees and almost unlimited money

You could say the same about Google - and yet they missed the consequences of their own discovery and got behind instead of being leaders. So you need specific talent to pull this off even if in theory you can hire anybody.

wg0
9 replies
9h21m

I am just curious how it happened to Google? Like who were the product managers or others who didn't see an opportunity here exactly where the whole thing was invented and they had huge amounts of data already, whole web basically and the amount of video that no one else can ever hope to have?

pembrook
2 replies
8h34m

I’m 100% positive lots of people at Google were chomping at the bit to productize LLMs early on.

But the reality is, LLMs are a cannibalization threat to Search. And the Search Monopoly is the core money making engine of the entire company.

Classic innovators dilemma. No fat-and-happy corporate executive would ever say yes to putting lots of resources behind something risky that might also kill the golden goose.

The only time that happens at a big established company, is when driven by some iconoclastic founder. And Google’s founders have been MIA for over a decade.

JKCalhoun
1 replies
7h4m

Golden goose is already being hoisted upon a spit — and your company is not even going to get even drippings of the fat. I am surprised by the short-sightedness of execs.

pembrook
0 replies
6h43m

I don’t work there, I’ve just worked for lots of big orgs — they are all the same. Any claimed uniqueness in “Organizational structure” and “culture” are just window dressing around good ol’ human nature.

It’s not short sightedness, it’s rational self-interest. The rewards for taking risk as employee #20,768 in a large company are minimal, whereas the downside can be catastrophic for your career & personal life.

aramattamara
1 replies
9h9m

It's hard to invest millions in employees who are likely to leave to a competitor later. That's very risky, aka venture.

JKCalhoun
0 replies
7h3m

So the alternative is to...?

thebytefairy
0 replies
9h3m

A lot of it was the unwillingness to take risk. LLMs were, and still are, hard to control, in terms of making sure they give correct and reliable answers, making sure they don't say inappropriate things that hurt your brand. When you're the stable leader you don't want to tank your reputation, which makes LLMs difficult to put out there. It's almost good for Google that OpenAI broke this ground for them and made people accepting of this imperfect technology.

jack_riminton
0 replies
9h6m

I think the discovery of the power of the LLM was almost stumbled upon at OpenAI, they certainly didn't set out initially with the goal of creating them. Afaik they had one guy who was doing a project of creating an LLM with amazon review text data and only off the back of playing around with that did they realise its potential

iosjunkie
0 replies
6h4m

Well for one, Ilya was poached from Google to work for OpenAI to eventually help build SOTA models.

Fast forward to today and we a discussing the implications of him leaving OpenAI on this very thread.

Evidence to support the notion that you can’t just throw mountains of cash and engineers at a problem to do something truly trailblazing.

andy99
0 replies
9h9m

Data volume isn't that important, that's becoming clearer now. What OpenAI did was paid for a bunch of good labelled data. I'm convinced that's basically the differentiator. It's not a academic or fundamental thing to do which is why google didn't do it, it's a pure practical product thing.

l5870uoo9y
1 replies
9h44m

Difficult to quantify but as an example the 2017 scientific paper “Attention is all you need” changed the entire AI field dramatically. Without these landmark achievements delivered by highly skilled scientists, OpenAI wouldn’t exist or only be severely limited.

belter
0 replies
8h36m

And ironically even the authors did not fully grasp at the time the paper importance. Reminds me of when Larry Page and Sergey Brin, tried to sell Google for $1 million ...

spamizbad
0 replies
4h39m

It depends on your views on LLMs

If your view is that LLMs only need minor improvements to their core technology and that the major engineering focus should be placed on productizing them, then losing a bunch of scientists might not be seen as that big of a deal.

But if your view is that they still need to overcome significant milestones to really unlock their value... then this is a pretty big loss.

I suppose there's a third view, which is: LLMs still need to overcome significant hurdles, but solutions to those hurdles are a decade or more away. So it's best to productize now, establish some positive cashflow and then re-engage with R&D when it becomes cheaper in the future and/or just wait for other people to solve the hard problems.

I would guess the dominant view of the industry right now is #1 or #3.

sheeshkebab
0 replies
7h41m

Talk to Microsoft

mrklol
0 replies
10h18m

They definitely need them to find new approaches which you won’t find normally.

belter
0 replies
7h33m

OpenAI has less than 800 employees

RandomLensman
0 replies
8h51m

Most of every large business isn't science but getting organized, costs controlled, products made, risk managed, and so forth.

AdamN
0 replies
10h34m

Agreed - it's good to have some far thinking innovation but really that can be acquired as needed so you really just need a few people with their pulse on innovation which there will always be more of outside a given company than within it.

Right now it's all about reducing transaction costs, small-i innovating, onboarding integrations, maintaining customer and stakeholder trust, getting content, managing stakeholders, and selling.

kinnth
18 replies
6h56m

AI has now evolved beyond just the science and it's biggest issue is in the productization. Finding use cases for what's already available ALONG with new models will be where success lies.

ChatGPT is the number 1 brand in AI and as such needs to learn what it's selling, not how its technology works. It always sucks when mission and vision don't align with the nerds ideas, but I think it's probably the best move for both parties.

CooCooCaCha
5 replies
4h32m

“Its biggest issue is in the productization.”

That’s not true at all. The biggest issue is that it doesn’t work. You can’t actually trust ai systems and that’s not a product issue.

senseiV
1 replies
2h55m

if the ai is the product, and the product isnt trustable, isnt that a product issue??

shwaj
0 replies
1h57m

It’s a core technology issue.

The AI isn’t the product, e.g. the ChatGPT interface is the main product that is layered above the core AI tech.

The issue is trustworthiness isn’t solvable by applying standard product management techniques on a predictable schedule. It requires scientific research.

tivert
0 replies
1h57m

That’s not true at all. The biggest issue is that it doesn’t work. You can’t actually trust ai systems and that’s not a product issue.

I don't know about that, it seems to work just fine at creating spam and clone websites.

tbrownaw
0 replies
1h47m

It works fine for some things. You just need a clearly defined task where LLM + human reviewer is on average faster (ie cheaper) than a human doing the same task themselves without that assistance.

startupsfail
0 replies
4h15m

Ilya is one of the Founders of the original nonprofit. This is also an issue. It does look like he was not the Founder or in any control of the for profit venture.

itsoktocry
2 replies
6h49m

ChatGPT is the number 1 brand in AI and as such needs to learn what it's selling, not how its technology works.

I'm not as in tune as some people here so: don't they need both? With the rate at which things are moving, how can it be otherwise?

vasco
0 replies
6h18m

I guess their point is you already have a lot out there to create new products, and you can still read papers you just won't be writing them.

throwthrowuknow
0 replies
5h59m

They do need both but it seems like they have enough engineering talent to keep improving. Time will tell now that Ilya is out but I expect they have enough cultural cache to attract excellent engineers even if they aren’t as famous as Ilya and Karpathy.

They have a strong focus on making the existing models fast and cheap without sacrificing capability which is music to the ears of those looking to build with them.

appplication
2 replies
3h41m

To this end, OpenAI is already off track. Their “GPT marketplace” or whatever they’re calling it is just misguided flailing from a product perspective.

javaunsafe2019
0 replies
2h30m

Ain’t there this pattern that innovations comes in waves and that the companies of the first wave most often just die but the second and third wave a build upon their artefacts and can be successful in a longer run?

I see this coming for sure for open ai and I do my part by just writing this comment on HN.

j45
0 replies
2h37m

Or they were experimenting with people defining agentic A.I. slightly before it became more widely popular.

watt
1 replies
1h13m

ChatGPT is the number 1 brand in AI

Not for long. They have no moat. Folks who did the science are now doing science for some other company, and will blow the pants off OpenAI.

pembrook
0 replies
15m

I think you massively underestimate the power of viral media coverage and the role it plays in building a “brand.” You’ll never replicate the Musk/Altman/Satya soap opera again. ChatGPT will forever be in the history books as the Kleenex of LLM AI.

ActionHank
1 replies
4h21m

AI has now evolved beyond just the science

Pretty weak take there bud. If we just look at the Gartner Hype Cycle that marketing and business people love so much it would seem to me that we are at the peak, just before the downfall.

They are hyping hard to sell more, when they should be prepping for the coming dip, building their tech and research side more to come out the other side.

Regardless, a tech company without the inventors is doomed to fail.

disqard
0 replies
2h57m

I'm siding with you here. The same is happening at Google, but they definitely have momentum from past decades, so even if they go "full Boeing", there's a long way to fall.

Meanwhile, OpenAI (and the rest of the folks riding the hype train) will soon enter the trough. They're not diversified and I'm not sure that they can keep running at a loss in this post-ZIRP world.

ronald_petty
0 replies
1h33m

Agree in general. While there remains issues on making/using AI, there is plenty of utility that doesn't require new science but maturation of deployment. For those who say its junk, I can only speak for myself and disagree.

ChildOfChaos
0 replies
4h1m

Eh maybe from a company point of view.

But this race to add 'AI' into everything is producing a lot of nonsense. I'd rather go fullsteam ahead on the science and the new models, because that is what will actually get us something decent, rather than milking what we already have.

chx
15 replies
8h29m

When the next wave of new deep learning innovations sweeps the world,

that won't happen, the next scam will be different

it was crypto until FTX collapsed then the usual suspects led by a16z leaned on OpenAI to rush whatever they had on market hence the odd naming of ChatGPT 3.5.

When the hype is finally realized to be just mass printing bullshit -- relevant bullshit, yes, which sometimes can be useful but not billions of dollars of useful -- there will be something else.

Same old, same old. The only difference is there is no new catchy tunes. Yet? https://youtu.be/I6IQ_FOCE6I https://locusmag.com/2023/12/commentary-cory-doctorow-what-k...

trashtester
14 replies
7h45m

Crypto currencies has the potential to grow the world economy by about 1-3%, as banking fees go down. Add other uses of crypto may double or triple that, but that's really speculative.

AI, on the other hand, has a near infinite potential. It's conceivable that it will grow the global economy by 2% OR MORE per MONTH for decades or more.

AI is going to be much more impactful than the internet. Probably more than internal combustion, the steam engine and electricity combined.

The question is about the timescale. It could take 2 years before it really starts to generate profits, or it could take 10 or even more.

jedrek
3 replies
7h35m

AI? Yes.

LLMs pretending to be AI? No.

trashtester
2 replies
5h40m

What you call "AI" is generally named AGI. LLM's are alredy a kind of AI, not just generic enough to fully replace all humans.

We don't know if full AGI can be built using just current technology (like transformers) given enough scale, or if 1 or more fundamental breakthroughs are needed beyond just the scale.

My hypothesis has always been that AGI will arrive roughly when the compute power and model size matches the human brain. That means models of about 100 trillion params, which is not that far away now.

chx
1 replies
24m

We don't know if full AGI can be built using just current technology (like transformers) given enough scale,

We absolutely do and the answer is such a resounding no it's not even funny.

trashtester
0 replies
15m

Actually, we really don't. When GPT-3.5 was released, it was a massive surprise to many, exactly because they didn't believe simply scaling up transformers wouldn't end up with something like that.

Now using transformers doesn't mean they have to be assembled like LLM's. There are other ways to stich them together to solve a lot of other problems.

We may very well have the basic types of lego pieces needed to build AGI. We won't know until we try to build all the brain's capacities into a model of size of a few 100 trillion parameters.

And if we actually lack some types of pieces, they may even be available by then.

chx
3 replies
4h50m

All crypto"currencies" with a transaction fee are negative sum games and as such , they are a scam. It's been nine years since the Washington Post admittedly somewhat clumsily but still drawn attention to this and people still insist it's something other than a scam. Despite heady articles about how it's going to solve world hunger, it's just a scam.

This round of AI is only capable of producing bullshit. Relevant bullshit but bullshit. This can be useful https://hachyderm.io/@inthehands/112006855076082650 but it doesn't mean it's more impactful than the Internet.

trashtester
2 replies
4h24m

I agree, 1-3% was a best case. While I agree it's a net zero, even those who argue for it really don't claim much more than a couple of %.

I actually expected objections on the opposite direction. But then, this is not twitter/X.

The point is that something that can easily generate 20%-100% growth per year (AGI/ASI) is so much more important that the best case prediction for crypto's effect on the economy are not even noticeable.

That's why comparing the crypto bubble to AI is so meaningless. Crypto was NEVER* going to be something hugely important, while AI is potentially almost limitless.

*If crypto had anything to offer at all, it would be ways to avoid fees, taxes and the ability to trace transactions.

The thing is, if crypto at any point seriously threatens to replace traditional currencies as stores of value in the US or EU, it will be banned instantly. Simply because it would make it impossible for governments to run budget deficits, prevent tax evasion and sever other things that governments care about.

chx
1 replies
4h3m

LLM is not AGI and there's no way to AGI from LLM. Put down the kool-aid.

trashtester
0 replies
2h46m

I never claimed llm's are agi. Not all neural nets are llm's.

KoolKat23
2 replies
5h50m

I agree with you on the AI point, but with crypto not all is what it seems.

Yes you may have short term growth, this is solely due to there being less regulation.

Despite what many people think regulation is a good thing, put in place to avoid the excesses that lead to lost livelihoods. It stops whales from exploiting the poor, provides tools for central banks to try avoid depressions.

Costs wise, banks acting as trust authorities actually can theoretically be cheaper too.

trashtester
1 replies
5h28m

Well, I agree with all that. The 1-3% was meant to come off as a tiny, one-time gain, and an optimistic estimate of that. Not at all worth the hype.

Basically, crypto is more like gold rush than a tech breakthrough. And gold rushes rarely lead to much more than increased inflation.

KoolKat23
0 replies
4h28m

Absolutely

itsoktocry
1 replies
6h31m

Crypto currencies has the potential to grow the world economy by about 1-3%, as banking fees go down.

Bank fees don't disappear into the ether when they're collected, so I doubt they have this much affect.

Oh, made my very first retail purchase with Bitcoin the other day. While the process was pretty slick and easy, the network charged $15.00 in fees. Long way to go until "free".

trashtester
0 replies
6h12m

Bank fees don't disappear into the ether when they're collected, so I doubt they have this much affect.

1-3% was intended as a ceiling for what cryptocurrency could bring to the economy, after adjusting for the reduction in inflation once those costs are gone.

cristiancavalli
0 replies
4h11m

Do you have a source for any of these numbers or is this just your speculation? I haven’t seen any estimates from well-known institutions that reference any of the numbers your are pointing to.

renegade-otter
1 replies
5h2m

"Productization". You mean "enshitification".

boringg
0 replies
4h40m

Depends on who is controlling product.

ookdatnog
0 replies
3h3m

They make lots of money

Will they though? Last I heard OpenAI isn't profitable, and I don't know if it's safe to assume they every will be.

People keep saying that LLMs are an existential threat to search, but I'm not so sure. I did a quick search (didn't verify in any way if this is a feasible number) to find that Google on average makes about 30 cents in revenue per query. They make a good profit on that because processing the query costs them almost nothing.

But if processing a query takes multiple seconds on a high-end GPU, is that still a profitable model? How can they increase revenue per query? A subscription model can do that, but I'd argue that a paywalled service immediately means they're not a threat to traditional ad-supported search engines.

loldomsa
0 replies
8h41m

I honestly think that is the best course of actions for humanity. Even less chance to see AGI anytime soon if he leaves.

hojleorier23423
0 replies
7h19m

What an absurd thing to say.

John Schulman is still at OpenAI. As are many others.

dkjaudyeqooe
0 replies
8h41m

Open AI is run by marketing, business, software and productization people.

AKA 'the four horsemen of enshitification'.

gdiamos
1 replies
13h11m

I don’t think people give Dario enough credit

HarHarVeryFunny
0 replies
6h35m

Yeah, I think him leaving was a huge blow to OpenAI that they have maybe not yet recovered from. Clearly there is no moat to transformer-based LLM development (other than money), but in terms of pace of development (insight as to what is important) I think Anthropic have the edge, although Reka are also storming ahead at impressive pace.

yu3zhou4
0 replies
11h19m

Jakub Pachocki is still in OpenAI though

larodi
0 replies
9h12m

Karpathy is still a mountain in the area of ML/AI, one of the few people worth following closely on Twitter/X.

davedx
0 replies
11h30m

I love Karpathy. He's like a classical polymath, a scholar and a teacher.

albertzeyer
0 replies
10h38m

Greg Brockman is a very good engineer. And that's maybe even more important in the current situation.

snowbyte
65 replies
13h35m

When walking around the U of Toronto, I often think that ~10 years ago Ilya was in a lab next to Alex trying to figure things out. I can't believe this new AI wave started there. Ilya, Karpathy, Jimmy Ba, and many more were at the right time when Hinton was there too.

izend
62 replies
13h34m

And none of them build AI companies in Toronto.

I’m Canadian and disappointed at how ineffective we are at building successful companies.

titanomachy
17 replies
13h25m

Yeah Canada just spends a ton of taxpayer money to create great institutions like U of T and Waterloo, so that their graduates can all go to Silicon Valley and make 2-3x the money.

saithound
10 replies
12h58m

Maybe the majority of Canadians think that having great higher education institutions and thr people who work in them is a good fit for their way of life, but having Silicon Valley companies and people making SV salaries around epuld make their lives worse? If so, this is great: Canadians don't want to live with the tech crowd, so they provide them with the skills so they can move elsewhere, make their dreams come tuee, and not bother the majority that don't want their presence.

NB some actual Canadians in this thread have voiced this possibility.

vasco
9 replies
12h47m

That makes zero sense, governments invest in education to improve their own country, not to train other countries work forces. If you read anything about Canada ever you will also know they have a bunch of policies to try and stop the brain drain and to recruit tech workers from abroad.

saithound
6 replies
12h23m

That makes zero sense, governments invest in education to improve their own country

The idea is precisely that not having SV types around _improves_ the country, i.e. makes it closer to the preferences of Canadians.

And yes, having a foreign tech worker doing 9-to-5 in a large legacy company for thoroughly average salaries is very different from having a SV-style startup culture. There is very little process in Canada to make life difficult for the former style of company, and plenty of process to make operations difficult for the latter.

If not having SV folk improves Canada for Canadians, and hqving SV folks improves America for Americans, then this is just mutually beneficial trade. Efforts to try and stop brain drain still makes sense: it's even better if you can convince the citizens you trained to engage in the economic activity you actually want instead of economic activity that you find undesirable, but if you're unable to convince most of them, letting them go is still better than having them stay and engage in their undesirable behavior anyway.

Compare: if a large minority of Icelanders wanted to work for the Baby (which Iceland doesn't have), theb stopping the brain drain (convincing them to work in the Merchant Fleet) is the best outcome, but funneling them out (training them in merchant navigation and watching them join the Danish Navy) would still be preferable to them engaging in their desired behavior anyway (form their own pirate gang preying on the very Merchant Fleet you're trying to advantage).

vasco
4 replies
12h12m

And yes, having a foreign tech worker doing 9-to-5 in a large legacy company for thoroughly average salaries is very different from having a SV-style startup culture

Immigrants coming into countries start companies at a disproportionate rate compared to natives.

Other than unquantifiable statements about what "Canadians want" everything you mentioned so far to justify this idea of "canada doesnt care if tech graduates leave" is falsifiable by data.

saithound
3 replies
11h41m

One last time, the claim is not that "Canada doesn't care". It's that it prefers it to the alternative of SV-style companies operating from Canada. Which is consistent both with data, facts on the ground (yes, Canada has laws and administrative processes designed to make SV-style startups difficult to start there, that's precisely what people complain about above!), and the comments of actual Canadians in this very thread.

You're welcome to present data falsifying the actual claim if you think you have it (instead of the "Canada doesn't care" straw man or misunderstanding that you repeat above, noting that so far you have not even refuted your own straw man by presenting any data).

vasco
2 replies
11h27m

Maybe the majority of Canadians think that (...) having Silicon Valley companies and people making SV salaries around epuld (sic) make their lives worse

This is your claim that I engaged with. If your claim is true it literally means that Canadians do not care if those people leave, in fact they would prefer it. My argument is that you're wrong and Canada and it's people would rather have more tech workers and more tech companies.

I don't believe I'm misunderstanding so I think we should probably both give up at this point.

saithound
0 replies
11h16m

I don't tuink it implies that they don't care, it only implies that they find it preferable to one certain alternative (staying AND turning Vancouver into north-SF; the conjunction is load-bearing), and I think this much looks true and well-supported by the facts and revealed preferences. They're not willing to change the rules and procedures that people complain about here, and if you propose they do so, as many have, they say no to that explicitly.

But I agree that we should probably disengage, so (barring exceptional new insights on my end) will leave this as my last post in the thread. Thanks for the chat.

chucke1992
0 replies
5h8m

The problem is that Canada is basically a european country on the american continent - SV is possible in a place where you can have risk and reward. But also you might lose everything. In Canada, it is hard to become rich - so no worth trying, there is also less risk due to better social security and the base level is pretty decent. Would not be surprised if there are tons of regulations in Canada too (more than in USA).

There is a reason why there are not many startups in Europe - if you can have a decent life, secure job and a nice social security - no worth playing risky games. I would not be surprised if just sheer layoffs in USA led to more startups than in the whole Europe.

manuel_w
0 replies
5h18m

Compare: if a large minority of Icelanders wanted to work for the Baby (which Iceland doesn't have), theb stopping the brain drain (convincing them to work in the Merchant Fleet) is the best outcome, but funneling them out (training them in merchant navigation and watching them join the Danish Navy) would still be preferable to them engaging in their desired behavior anyway (form their own pirate gang preying on the very Merchant Fleet you're trying to advantage).

I read this as if you'd be concerned of Canadians using their tech skills run malware groups, if Canada wouldn't let them leave and join SV companies.

vintermann
1 replies
11h4m

It's not as if Canada doesn't benefit from machine learning advances. It just doesn't by having many ML start-ups as a tax base.

Canada's skilled immigration policy is a train wreck, but that's another issue.

newcan1
0 replies
9h55m

Canada's skilled immigration policy is amazing. It is attracting some of the best talent in the world. What it is not able to do is retain the talent and is just ending up as a stepping stone to the US. All it needs to do is two things: 1. Provide tax deductions for rent and interest on home loans for new home buyers. 2. Reduce the average taxes to just slightly less than the US tax rate by 5-10% upto 500k. Then watch the magic happen.

newzisforsukas
2 replies
12h1m

As if the tech economy in the US is unique for some reason

MrBuddyCasino
1 replies
11h49m

It kind of is?

sal_regalier
0 replies
10h53m

I think that was the joke.

llm_trw
2 replies
12h42m

2-3x the money.

That's if you're stuck in tech support. When you start doing actual ground breaking work it starts at x10 and goes up significantly.

titanomachy
1 replies
11h25m

If you’re a top-tier AI researcher like Ilya, for sure. I was thinking about your run-of-the mill FAANGish senior engineer.

llm_trw
0 replies
3h12m

E6 in facebook is something like USD 750k total comp.

In Canada you'd be lucky to get USD 180k total comp.

It really is x5 to x10 for seniour programmers.

typon
10 replies
13h11m

The Canadian Dream is to get a great education and then move of the US.

You might want to blame the government or this or that but I think as a Canadian I've finally come to reckon with the fact that it's just not in the Canadian ethos to do risky things like make startups. Of course there are exceptions to the rule but they are very very rare. Canadian investors don't want to take big risks and the Americans are just next door waiting to gobble up the talent in search of capital.

eggdaft
9 replies
12h54m

For folks without responsibilities like kids, aging parents, etc. I really don’t think startups are very “risky”.

What’s the worst that happens? It doesn’t work out and after five years you go get a job in boring corp corp with an incredible skillset and vast life experience.

You’ve sacrificed some income perhaps, but so what? People make choices like that all the time. Your working career could easily be 40 or 45 years, 5 is not that much and it’s not like you went bankrupt. Your skillset might even mean you more than make up for lost time.

I don’t understand the talk of “risk” unless you’re Elon Musk betting the farm on your businesses and facing bankruptcy.

Work in your spare time until you have something Angel worthy, then get a modest salary to get to the next level and on you go. Or just bootstrap.

Is it easy? No, it’s the hardest thing you’ll ever do. Is it risky? Not so much.

So why do Canadians and Brits see it as a risky thing to do? I think they don’t. What they see is _uncertainty_ - where will I be in six months? What if it doesn’t work out? What if I fail and people judge me? They don’t like uncertainty. That is conservative with a small c. Probably it’s a cultural artefact rather than anything remotely rational. The problem is you end up in an equilibrium where the society is conservative (“what you wanna do wasting your time with that”) so the ambitious people just leave and go to somewhere like (parts of) the US where people want to change things, make things, improve the world. And the conservative society gets more conservative until it is ossified.

Startups carry high uncertainty but not high risk.

vasco
3 replies
12h41m

This depends on what you see as risk. If I can safely earn for 5 years way above national average and build a strong savings egg that can provide income forever.

Or I can fail at a startup and be close to zero five years later, the fact that you aren't homeless and starving and can get another job doesn't mean it wasn't risky, you still wasted a bunch of years compared to slow and steady accumulation.

I've read the majority of millionaires in the US get created like this, working and saving through decades.

You're basically repeating investor kool-aid, because for their model to work, 100 people will fail and 1 succeed, and so they tell you to not worry if you're in the 99.

ghaff
1 replies
12h26m

Of course you could probably say at least some of the same things about grad degrees that may not really translate into appreciable different/better career outcomes. Of course some say exactly that, especially about PhDs.

eggdaft
0 replies
48m

I don’t think there are risky either.

For me risk is “could go horribly wrong” but the worst case for most startup founders is … get a job?

eggdaft
0 replies
50m

“Wasting years” is not a risk. It’s a choice. And as I pointed out it’s likely not wasted anyway.

StrauXX
1 replies
12h13m

There are countries where the business culture makes you unemployable and almost impossible for you to get a loan for the rest of your live if you have ever failed a business (bad enough). Many countries aren't as open to failure as the US.

eggdaft
0 replies
51m

This is a fair point. I was talking in the context of USA UK Canada but it might not generalise.

prmoustache
0 replies
11h46m

I can't speak for Canada and I may be a wrong, but it seems to me harder to loan money for business than in NA. Banks are the ones that don't want to take risks, not necessarily the people with ideas.

Also failures aren't considered the same in every job market.

fire_lake
0 replies
12h29m

The increased cost of living in the last few years has changed this somewhat. That 5 years of lower earnings now means less nice groceries, fewer holidays and being under the yoke of landlords for considerably longer.

baq
0 replies
12h38m

Turns out there’s only enough people with this mindset to fill a couple hubs around the world. The rest prefers less volatility and happily takes on less downside risk for capped reward and/or less upside risk.

nightowl_games
8 replies
13h13m

Couldn't a company like that get a huge tax benefit from the SRED program?

loopdoend
6 replies
13h8m

The amount of paperwork involved makes it unworkable

MassiveQuasar
4 replies
12h59m

10B$ says you're wrong.

llm_trw
2 replies
12h43m

10B is going to old mates mates.

No new startups are getting it.

It's also like pulling teeth trying to explain to people that if we don't offer compensation commensurate with what they get in the US people will just leave for the US.

There is some form of brain damage where even people who know how to code assume that because you can get a crud developer for $80k a year you should get an AI researcher for $150,000. It's nearly double after all.

tbossanova
1 replies
11h58m

Compensation isn’t just money. Staying near family and friends is hard to measure. I’ve never moved country for $ alone.

llm_trw
0 replies
3h11m

Great. But anyone good gets offered so much they move eventually.

It might not be a year after uni, might not be 10, but eventually they will move because the pay in the US is just so much better than anywhere else.

x-complexity
0 replies
11h19m

There is a vast underestimation of how tedious & time-intensive these tax credit programs are when applying for them. A large company can do so because they can hire the people to solely go after them; A new startup (with a headcount that can fit in one hand) is too busy in actually keeping the business alive to pursue these programs, which often times come with conditions too arbitrary for startups to fulfill.

sonofaragorn
0 replies
5h48m

Not true. I've done SRED every year for the past ~7 years. It is work, but there are specialized consultants that do most of it. If the work is truly R&D (which would be the case for a cutting-edge AI company) and you track your work in JIRA or something like that, then it's mostly just writing a few pages describing the efforts.

cmrdporcupine
0 replies
10h16m

SRED is basically a subsidy for companies that do your SRED paperwork for you, not the company doing the engineering itself. There's a whole industry of this.

No evidence that SRED has done anything ever for actual R&D. I've seen people get SRED for making web pages in JavaScript&HTML. When I had to fill in the SRED stuff it was ridiculous. Someone doing actual innovation would throw their hands up in the air.

langsoul-com
8 replies
12h34m

Why would anyone start the game on hard mode when easy mode is a border drive away?

Us is so outrageously better than the rest that people fly across oceans to start businesses there. Canada, being next door, doesn't have the distance moat to at least slow down the brain drain

threatofrain
5 replies
12h6m

With regards to talent, there's no particular reason why software centers couldn't be in any major established city in the world. It's not like it takes billions of dollars on a highly uncertain bet like creating a car company, rocket reuse company, or a CPU company.

A small crew of people could potentially build the next WhatsApp. On Erlang.

resonious
1 replies
9h17m

There are definitely many good programmers all over the world, but there are more in the US, because that's where all the best companies are. So if you're trying to make a good company and you want good programmers, where do you go?

OJFord
0 replies
6h59m

You stay where you are and hire the ones willing or keen to work remotely?

pquki4
0 replies
8h20m

Eh, $$$?

I know people from "first world" countries like Japan and France that are come to work in the US simply because it pays much more.

langsoul-com
0 replies
8h19m

That statement applies to most industries. Tons of areas have the potential for an industry boom, but silicon Valley in is California. Or semi conductors are in Taiwan.

For many reasons, only some areas succeed whilst the rest fail. In this case, Canada doesn't have silicon valley, nor do they have a high amount of start ups.

ForHackernews
0 replies
9h11m

In the case of AI it absolutely does take billions and billions of dollars on an uncertain bet. They bet that throwing more data, more hardware, more GPU cycles at the problem would yield results and it has.

Hugsun
1 replies
7h36m

That's such a frustrating thing. There are so many opportunities in SF alone. But I really don't want to live there.

threatofrain
0 replies
2h3m

There are also plenty of opportunities in the SV outside of SF which have a very, very different vibe.

boringg
6 replies
4h27m

I've thought about this one for a long time having lived in both SV and Canada. It is a complicated one but there are a handful of critical road blocks in Canada that make it more challenging.

(1) Access to size of market even if online being US vs 'foreign' has advantages in political arena/regulatory benefits

(2) Significant tax advantages for US investors vs limited tax advantages for Canada (Angel+VC)

(3) Risk Appetite (impacted by size of market) - compounded by tax disadvantages (why would you take risk if your lining the pockets of the government?)

(4) Bench depth on talent once you really start to scale your company

(5) CAD strength (double edged sword) - talent goes South for better salaries (+ you need to compete), if the company revenue is in USD and employees are paid in CAD

(6) Start-ups paying in equity, early employees taking on that risk actually will get taxed heavily under new cap gains so the incentive to work hard for money is lower.

(7) Network effects of being in the valley - idea percolation, new playbooks, talent, competitiveness, company fitness

I will add that in this very specific AI case there is limited way you are going to find the depth of talent and capital in the country to make that company fly at the scale it needs to be.

jpalawaga
5 replies
3h57m

if the idea that high taxes disincentivizes people from building stuff, california would be a wasteland.

but that's not what we see. people build because they still have a chance a making a lot of money.

also, like canada can build successfully tech companies. yes, I realize there should be more canadian tech darlings, but I don't think it has to do with high taxes so much as it has to do with Canadians being comfortable and not feeling the need to sacrifice everything to try and build the biggest thing.

If you look at Canada's most successful tech companies, the founders usually sell and enjoy a more comfortable existence.

willhslade
0 replies
1h46m

Nortel imploded, though. Self driving cars. Bitcoin. Canadian institutional investors are wary.

somebodythere
0 replies
3h33m

California benefits from a dominant market position. If you have the choice to found where the investors and the talent are, why would you pick Canada when the tax is the same?

mitthrowaway2
0 replies
1h44m

I think it has much more to do with investors' sentiments. Canadian entrepreneurs are not comfortable; that's why they move to the US. But that's not because they don't like Canada. Moving is a big sacrifice -- they move away from their home and community, and also deal with the headache and uncertainty of US immigration. The ones I talk to who have moved down south, they miss Canada and didn't want to leave, but they didn't feel like they'd be able to afford the cost of living in Canada, and didn't think they could launch a successful startup there.

And the cost of living is going up, which is going to make even more talented Canadians uncomfortable. These days if you ever hope to own a house, you basically can't go the stable 9-to-5 route.

If investors in Canada were throwing hundreds of millions into moonshot startups the way that they do in Silicon Valley, probably most Canadian entrepreneurs would build those companies at home. But the investment landscape is such that the investors who have that much money opt to lever up on real estate instead.

lesuorac
0 replies
3h37m

The point was about high taxes for investors not employees.

How many money does OpenAI _directly_ pay to CA in taxes? Sure the employees pay a ton in taxes but as an investor if you're going to lose more in taxes by investing into a Canadian company vs a California company then you invest into CA.

boringg
0 replies
3h29m

You misunderstand my statement on tax.

Taxs on investors are quite high in Ontario/Canada compared to California. Not only does this minimize the outcome for the investor - it decreases the risk for making larger bets on big outcomes. In terms of exits -- you have a smaller playing field and fewer buyers being based in Canada vs USA. All the things add up to make a smaller opportunity for investors and builders and you work harder to pay more taxes to the government.

In terms for your ambition argument -- that could be an inherent problem in Canadian culture that no one wants to change the status quo - it is definitely a different culture than SV. The largest city is captured by financial industry for the most part which doesn't bode well for innovation.

swat535
3 replies
6h57m

As another Canadian, I feel the same but I'm not surprised one bit.

Canada is actively hostile towards tech and suffers from crippling salaries and investments. The idea of "business" in this country is buying a house and renting its basement.

Our government's incompetence is comical, we are nothing but more than a tech talent / immigration proxy for United States at this point.

JKCalhoun
1 replies
6h48m

I don't know, Canada and every-other-country-that's-not-the-US. When there's a neighbor that is flush with cash, where make-or-break is a kind of national disease ... what can you do?

(I'm an American, FWIW.)

mdorazio
0 replies
6h7m

FWIW, Israel and China are the other two hotspots for building startups. It’s worth looking at how Israel did it since their model could work elsewhere. For example they have a government funded delegation that goes around to conferences solely to meet with large companies and investors and promote Israeli startups.

drdrek
0 replies
6h11m

Its a mix of "Easy" things to fix: Streamline tax code, build desirable office centers, Have good internet infrastructure

and "Hard" Things: The work culture, Cost of living

sonofaragorn
0 replies
5h46m

Aidan Gomez, Nick Frost, and Ivan Zhang, all of whom were Hinton's students at UofT started Cohere (https://cohere.com/about)

raverbashing
0 replies
10h11m

As much as people on this site like to complain about Europe (and a lot of it is merited) - I've found that Canada manages to be worse. Even having lower on avg bureaucracy

crucialfelix
0 replies
11h26m

Also: All your comedians move to the US to make it big.

cmrdporcupine
0 replies
10h18m

Canada is addicted to rent seeking, monopoly businesses, corporations that push regulatory capture on the gov't and then parasitize, and -- most of all -- ripping resources out of the ground and selling them cheap, or doing the same with real estate.

My latest annoyance is all the moaning and groaning about the latest capital gains tax increase. People complaining on one hand about how the Canadian economy lacks productivity, and then screaming to high heaven about tax policy that mostly only impacts people making quick speculative cash.

Investment takes no risks in this country because they don't have to. They just dump money into real estate or oil & gas instead and then hang at the lake in the Muskokas.

chollida1
1 replies
4h11m

Oh man that was an amazing time at UoT. We also got GPU versions of btc mining from that group.

We also had Ethereum be born right around there as well around 2014. I remember the first Ethereum meetups around Queen and Spadina with Vitalik.

But to another posters point. Even though we had the father of deep learning Geoffrey Hinton and lumiaries like Ilya, and Vitalik, we didn't manage to get any real benefit from that.

snowbyte
0 replies
2h32m

Wow! By the time I arrived, Hinton was gone. As well as many great professors that started their own companies or were poached by big players (i.e. Sanja-Nvidia). At least I got to learn NN from Jimmy Ba (author of Adam). Now, he's working at xAI.

peppertree
10 replies
19h10m

Since it's all in proper casing I'm going to assume he wrote it with chatgpt.

xmonkee
4 replies
18h32m

He used “easily one of the greatest minds of our generation” for two different people in the same message. 100% ai generated

mewpmewp2
1 replies
17h1m

AI or GPT usually doesn't repeat like that. Although it usually is easy to tell if something is GPT.

throwitaway222
0 replies
16h28m

They probably avoid making it look like their writing is written with chatgpt.

fsckboy
1 replies
18h24m

it said one was "easily one of the greatest", and it said the second was "also easily one of the greatest"... it's puffery but it's not an awkward or mindless formulation.

willsmith72
0 replies
18h19m

doesn't read like chatgpt to me, but i certainly wouldn't call it good quality writing

gdiamos
1 replies
14h14m

Ironically built by Ilya

ugh123
0 replies
13h50m

Real life Miles Dyson

rezonant
0 replies
16h14m

Or GPT-5 went rogue, took out the senior staff, and is running the game now, Westworld style.

behnamoh
0 replies
16h48m

He also literally mentioned Ilya's personal project; something that ChatGPT would do (it repeats parts of the prompt).

TechDebtDevin
0 replies
19h8m

Good observation lol.

nwoli
10 replies
18h29m

Personally I don’t trust much of anything sam says, so I’d take any framing with a large grain of salt

behnamoh
9 replies
16h47m

a large grain of salt

You mean a lump of salt? I've always wondered what the right word is to describe this amount of salt /!jk

__MatrixMan__
2 replies
16h29m

I've been offered a "lump" of sugar before, and it was not a single sugar crystal. When I hear "large grain of salt" I imagine something like this https://crystalverse.com/sodium-chloride-crystals/, quite different than a lump.

sebastiennight
0 replies
1h31m

That is absolutely brillant and will make a fantastic week-long father-daughter science project. Thanks.

Now back to figuring out something new for a 6-year old to program using Scratch...

behnamoh
0 replies
16h14m

Those huge salt cubes are so fascinating! New side project added to the list...

noufalibrahim
1 replies
15h48m

I usually use "a few bags of salt" to imply that I don't trust the source.

__d
0 replies
14h8m

perhaps a "mountain of salt" in this case?

yumraj
0 replies
13h41m

Boulder?

petesergeant
0 replies
15h55m

A pinch of salt

jiveturkey
0 replies
14h1m

a highway road side storage yard of salt

blitzar
0 replies
12h32m

A handful of salt

zombiwoof
8 replies
18h30m

Altman is the biggest con artist in tech.

mikeg8
3 replies
18h11m

Exactly. He’s only founded and led a company that’s built some of the most easily adoptable and exciting innovations in human-computer interactions in the last decade. Total fraud!

blibble
2 replies
17h59m

and which company would that be?

tortilla
1 replies
17h30m

Loopt

greenthrow
0 replies
17h15m

And WorldCoin.

coffeebeqn
2 replies
17h18m

What’s the con? Aren’t they constantly delivering frontier models?

Nasrudith
0 replies
14h49m

I would say specifically the sort of chuunibyou cringe endemic to"AI safety" of claiming that their models are an existential threat.

FrustratedMonky
0 replies
14h45m

Agree with the sentiment.

But Sam as 'conman' might just be the impression because he is more on the promotion/marketing side.

I've been under the impression that Ilya is the brains. So this seems bad for long term growth.

baq
0 replies
11h13m

Con artist is a bad description. The guy is legit dangerous. He's not after swindling you out of your money, that wouldn't be worth it.

nialv7
7 replies
18h42m

It could be a PR statement, it could also be genuine. From outside looking in there's no way to know, so I will just pretend this tweet doesn't exist.

baobabKoodaa
6 replies
17h23m

It's PR for sure. A genuine announcement would have addressed the elephant in the room.

djbusby
5 replies
17h9m

Sorry, which elephant?

Apocryphon
4 replies
16h43m

That this was an employee who conspired against him in a failed palace coup

hn_throwaway_99
1 replies
15h25m

I think calling it a "palace coup" gives it an inappropriate framing of what happened.

I definitely think that how the board handled the situation was very inept, and I think the naivety over the blowback they would receive was one of the most surprising things for me. But after reading more about the details of what happened, and particularly writings and interviews given by the former board members, I don't think any of them did this out of any particular lust for power, or even as some sort of payback for a grudge. It seemed like all of them had real, valid concerns over Sam's leadership. Did those concerns warrant Sam's firing? From what I've read, I'm of the opinion they didn't, but obviously as just some rando on the Internet, what do I know. But I do think that there were substantive issues in question, and calling it a "palace coup" diminishes these valid concerns in my mind.

Apocryphon
0 replies
11h19m

I'm not moralizing. There are palace coups that are justified.

WiSaGaN
0 replies
15h45m

At the time, Sam was more powerful than Ilya for sure. But framing their relationship as employee/employer when they were both in the board seems not correct.

8note
0 replies
14h52m

Sam's employer is who, the US taxpayer?

Handy-Man
3 replies
19h16m

While he does say, he is leaving for some personal and meaningful project, let’s see what it ends up being.

jonathankoren
1 replies
18h47m

More like hustle culture’s “spend more time with the family”

jumpCastle
0 replies
16h57m

Spend more time with my side projects.

kunley
0 replies
18h53m

That "personal and meaningful" can just mean anything.

jonathankoren
2 replies
18h46m

Nah. It’s same platitudes that’s always said when a someone high profile is fired.

rl3
0 replies
18h34m

Ilya is easily one of the greatest minds of our generation ...

Jakub is also easily one of the greatest minds of our generation ...

Phew, I was worried he'd be irreplaceable or something. Hopefully they've already standardized the comp package.

data_maan
0 replies
15h41m

Easily the most plat of all platitudes of our generation

data_maan
1 replies
15h44m

This was easily the most PR tweet of our generation.

The fact Ilya himself tweeted about it too was also easily the most PR tweet of our generation.

:D

jiveturkey
0 replies
13h59m

Yeah, like when Cheney shot Harry Whittington and it was Whittington that apologized.

johnbellone
0 replies
18h4m

He’s got a good PR team.

chimney
0 replies
19h8m

PR statement. After nearly being ousted, I'm sure Sam is relieved to have a thorn removed from his side.

apantel
0 replies
16h5m

It’s too nice. Nobody is this nice. It’s like Truman Show nice.

DalasNoin
45 replies
12h36m

There goes the so called superalignment:

Ilya

Jan Leike

William Saunders

Leopold Aschenbrenner

All gone

reducesuffering
22 replies
12h4m

Daniel “Quit OpenAI due to losing confidence that it would behave responsibly around the time of AGI”

“I think AGI will probably be here by 2029, and could indeed arrive this year”

Kokotajlo too.

We are so fucked

OtomotO
21 replies
11h55m

I am sorry, there must be some hidden tech, some completely different attempt to speak about AGI.

I really, really doubt that transformers will become AGI. Maybe I am wrong, I am no expert in this field, but I would love to understand the reasoning behind this "could arrive this year", because it reminds me about coldfusion :X

edit: maybe the term has changed again. AGI to me means truly understanding, maybe even some kind of consciousness, but not just probability... when I explain something, I have understood it. It's not that I have soaked up so many books that I can just use a probabilistic function to "guess" which word should come next.

TaylorAlexander
6 replies
11h44m

No I’m with you on this. Next token prediction does lead to impressive emergent phenomena. But what makes people people is an internal drive to attend to our needs, and an LLM exists without that.

A real AGI should be something you can drop in to a humanoid robot and it would basically live as an individual, learning from every moment and every day, growing and changing with time.

LLMs can’t even count the number of letters in a sentence.

kgeist
1 replies
8h50m

LLMs can’t even count the number of letters in a sentence.

It's a consequence of tokenization. They "see" the world through tokens, and tokenization rules depend on the specific middleware you're using. It's like making someone blind and then claiming they are not intelligent because they can't tell red from green. That's just how they perceive the world and tells nothing about intelligence.

OtomotO
0 replies
5h16m

But it limits them, they cannot be AGI then, because a child that can count could do it :)

astrange
1 replies
10h8m

LLMs could count the number of letters in a sentence if you stopped tokenizing them first.

HarHarVeryFunny
0 replies
6h33m

tokenization is not the issue - these LLMs can all break a word into letters if you ask them.

vintermann
0 replies
11h9m

From that AGI definition, AGI is probably quite possible and reachable - but also something pointless which there are no good reasons to "use", and many good reasons not to.

sebastiennight
0 replies
2h22m

You seem generally intelligent. Can you tell how many letters are in the following sentence?

"هذا دليل سريع على أنه حتى البشر الأذكياء لا يمكنهم قراءة ”الرموز“ أو ”الحروف“ من لغة لم يتعلموها."

n_ary
4 replies
11h42m

Don't worry, these are the "keeping the bridge intact" speak of people leaving a glorious or so workplace. I have worked at several places, and when people left(usually most well paid ones), they post linkedin/twitter posts to say kudos and inspire that, the corresponding business will be in forefront of the particular niche this year or soon and they would like to be proud of ever being part of it.

Also, when they speak about AGI, it raises their(person leaving) marketing value as someone else already know they are brilliant to have worked at something cool and they might also know some secret sauce, which could be acquired at lower cost by hiring them immediately[1]. I have seen these kinds of speak play out too many times. Last January, one of the senior engineers from my current work place in aviation left citing about something super secret coming this year or soon, and they immediately got hired by a competitor with generous pay to work on that said topic.

reducesuffering
3 replies
11h38m

Also, when they speak about AGI, it raises their(person leaving) marketing value

Why yes, of course Jan Leike just impromptu resigned and Daniel Kokotajlo just gave up 85% of his wealth in order not to sign a resignation NDA to do what you're describing...

Shrezzing
2 replies
10h29m

While he'll be giving up a lot of wealth, it's unlikely that any meaningful NDA will be applied here. Maybe for products, but definitely not for their research.

There's very few people who can lead in frontier AI research domains - maybe a few dozen worldwide - and there are many active research niches. Applying an NDA to a very senior researcher would be such a massive net-negative for the industry, that it'd be a net-negative for the applying organisation too.

I could see some kind of product-based NDA, like "don't discuss the target release dates for the new models", but "stop working on your field of research" isn't going to happen.

reducesuffering
1 replies
10h16m

Kokotajlo: “To clarify: I did sign something when I joined the company, so I'm still not completely free to speak (still under confidentiality obligations). But I didn't take on any additional obligations when I left.

Unclear how to value the equity I gave up, but it probably would have been about 85% of my family's net worth at least.

Basically I wanted to retain my ability to criticize the company in the future.“

but "stop working on your field of research" isn't going to happen.

We’re talking about NDA, obviously no-competes aren’t legal in CA

https://www.lesswrong.com/posts/kovCotfpTFWFXaxwi/?commentId...

darkwater
0 replies
8h49m

Unclear how to value the equity I gave up, but it probably would have been about 85% of my family's net worth at least.

Percentages are nice, but with money and wealth absolute numbers are already important enough. You can leave a very, very good life even if you are losing 85% if the remaining 15% is USD $1M. And if not signing that NDA will help you landing another richly paying job + freedom to say whatever you feel it's important saying.

truculent
1 replies
8h34m

truly understanding… when I explain something, I have understood it

When you have that feeling of understanding, it is important to recognize that it is a feeling.

We hope it’s correlated with some kind of ability to reason, but at the end of the day, you can have the ability to reason about things without realising it, and you can feel that you understand something and be wrong.

It’s not clear to me why this feeling would be necessary for superhuman-level general performance. Nor is it clear to me that a feeling of understanding isn’t what being an excellent token predictor feels like from the inside.

If it walks and talks like an AGI, at some point, don’t we have to concede it may be an AGI?

quantum_state
0 replies
6h57m

Would say understanding usually means ability to connect the dots and see the implications … not feeling.

bbor
1 replies
11h36m

As something of a (biased) expert: yes, it’s a big deal, and yes, this seemingly dumb breakthrough was the last missing piece. It takes a few dozen hours of philosophy to show why your brain is also composed of recursive structures of probabilistic machines, so forget that, it’s not neccesary, instead, take a glance at these two links:

1. Alan Turing on why we should never ever perform a Turing test: https://redirect.cs.umbc.edu/courses/471/papers/turing.pdf

2. Marvin Minsky on the “Frame Problem” that lead to one or two previous AI winters, and what an Intuitive algorithm might look like: https://ojs.aaai.org/aimagazine/index.php/aimagazine/article...

re
0 replies
9h50m

Alan Turing on why we should never ever perform a Turing test

Can you cite specifically what in the paper you're basing that on? I skimmed it as well as the Wikipedia summary but I didn't see anywhere that Turing said that the imitation game should not be played.

_nalply
1 replies
11h48m

I think what's missing:

- A possibility to fact-check the text, for example by the Wolfram math engine or by giving internet access

- Something like an instinct to fight for life (seems dangerous)

- some more subsystems: let's have a look a the brain: there's the amygdala, the cerebellum, the hippocampus, and so on, and there must be some evolutionary need for these parts

t4ng0pwn3d
0 replies
11h19m

AGI can’t be defined as autocomplete with fact checker and instinct to survive, there’s so so so much more hidden in that “subsystems point”. At least if we go by Bostroms definition…

ben_w
0 replies
9h20m

maybe the term has changed again. AGI to me means truly understanding, maybe even some kind of consciousness, but not just probability... when I explain something, I have understood it.

The term, and indeed each initial, means different things to different people.

To me, even InstructGPT manages to be a "general" AI, so it counts as AGI — much to the confusion and upset of many like you who think the term requires consciousness, and others who want it to be superhuman in quality.

I would also absolutely agree LLMs are not at all human-like. I don't know if they do or don't need the various missing parts in order to be in order to change the world into a jobless (u/dis)topia.

I also don't have any reason to be for or against any claim about consciousness, given that word also has a broad range of definitions to choose between.

I expect at least one more breakthrough architecture on the scale of Transformers before we get all the missing bits from human cognition, even without "consciousness".

What do you mean by "truly understanding"?

Miraltar
0 replies
8h34m

This paper and other similar works changed my opinion on that quite a bit. It shows that to perform text prediction, LLMs build complex internal models.

https://news.ycombinator.com/item?id=38893456

JKCalhoun
0 replies
6h54m

when I explain something, I have understood it.

Yeah, that's the part I don't understand though - do I understand it? Or do I just think I understand it. How do I know that I am not probabilistic also?

Synthesis is the only thing that comes to mind as a differentiator between me and an LLM.

belter
12 replies
8h32m

So Satya Nadella paid $13 billion to have....Sam Altman :-))

luma
8 replies
7h30m

Are you suggesting that the OAI investment was not a good investment for MS?

lenerdenator
7 replies
7h18m

Have they earned a return on it yet?

Seriously asking; I've purchased a GitHub CoPilot license subscription but I don't know what their sales numbers are doing on AI in general. It's to be seen if it can be made more cost-efficient to deliver to consumers.

luma
6 replies
6h18m

Checking MSFT price, seems like the market thinks they made the right move and the shareholders are for sure seeing a return.

whoknowsidont
3 replies
5h59m

The market thinks Tesla is worth more than all other automakers combined, that GameStop is a reasonable investment, and laying off engineers is great!

chucke1992
1 replies
5h28m

Because Tesla is. Unlike the traditional automakers that have no room of growth and in a perpetual stagnation, Tesla has potential being a partially automative and partially tech industry. They can even have their own mobile phones if they want to. Or robots and stuff.

What Mercedes, Porsche, Audi can do aside continue to produce the cars over and over again until they are overtaken by somebody else? Hell, both EU and USA need tariffs to compete with chinese automakers.

cityofdelusion
0 replies
1h10m

Not quite. Tesla has a high valuation mostly because traditional auto carries an enormous amount of debt on their balance sheets. I think Tesla is one economic downturn in a high interest rate environment from meeting the same fate. Once an auto company is loaded with debt, they get stuck in a low margin cycle where the little profit they make has to go into new debt for retooling and factories. Tesla is very much still coasting from zero interest rate free VC money times.

sonotathrowaway
0 replies
4h36m

Until it one day it doesn’t. It’s very fickle.

rchaud
0 replies
5h35m

What % of stock movements do you attribute to OAI, vs the cash-generation behemoth that is Windows/Office/Azure?

djeastm
0 replies
5h32m

Increased price of a company is indeed the expectation of future profits, but until those profits hit the balance sheet they are unrealized

drexlspivey
1 replies
7h44m

And, you know, a company with $2B of revenue

belter
0 replies
7h42m

That for sure loses money on every prompt...

awestroke
0 replies
7h56m

Perhaps Altman will fail upwards once again to become CEO of Microsoft

pcurve
4 replies
12h22m

Resignations lead to more resignations....unless mgmt. can get on top of it and remedy it quickly, which rarely happens. I've seen it happen way too many times working 25 years in tech.

sk11001
3 replies
9h18m

This might not be bad from the perspective of the remaining employees, it might be that the annoying people are leaving the room.

pcurve
1 replies
3h3m

In my experience, good ones leave first, followed by those who enjoyed working with them and or ones who are not longer able to get work done.

sk11001
0 replies
2h39m

You need to think about OpenAI specifically - Ilya basically attempted a coup last year and failed, stayed in the company for months afterwards, according to rumours had limited contributions to the breakthroughs in research and was assigned to lead the most wishy-washy project of superalignment.

I’m not seeing “the good ones” leaving in this case.

ionwake
0 replies
8h25m

Or that just the aggressive snakes are left.

I have no idea I’m saying I’ve seen that happen in companies.

e_i_pi_2
1 replies
8h42m

Relying on specific people was never a good strategy, people will change but this will be a good test of their crazy governance structure. I think of it similar to political systems - if it can't withstand someone fully malicious getting in power then it's not a good system

jeanlucas
0 replies
8h31m

Same applies to Sam Altman as well? Thing felt like a cult when he was forced out and everyone threatened to resign.

bamboozled
1 replies
10h24m

I guess if they really thought we had something to worry about, they would've stayed just to steer things in the right direction.

Doesn't seem like they felt it was required.

Edit: I'd love to know why the down votes, it's an opinion, not a political statement. This community is quite off lately.

Is this a highly controversial statement ? People are truly worried about the future and this is just an anxiety based reaction ?

lucianbr
0 replies
7h34m

Doesn't the whole Altman sacking thing show that they had no power to do any steering, and in fact Altman steers?

vintermann
2 replies
11h13m

The guy with the "Bad universal priors and notions of optimality", which did to Hutter's MIRI program what Gödel did to Hilbert's program.

debatem1
1 replies
10h56m

Any chance you can eli5? I'm familiar with the Godel/Hilbert side but not the relationship to these developments.

vintermann
0 replies
7h47m

Oops, I thought there was something odd, I got my rationality acronyms mixed up. Hutter's program was called AIXI (MIRI was the research lab).

Here is Leike's paper, coauthored with Hutter:

https://arxiv.org/abs/1510.04931

They can probably sum it up in their own paper better than I can, but AIXI was supposed to be a formalized, objective model of rationality. They knew from the start that it was uncomputable, but I think they hoped to use it as a sort of gold standard that you could approach.

But then it turned out that the choice of Turing machine, which can be (mostly) ignored for Kolmogorov complexity, can not be ignored in AIXI at all.

Symmetry
1 replies
4h52m

The scenario I have in my head is that they had to override the safety team's objections to ship their new models before Google IO happened.

jdthedisciple
0 replies
52m

The "safety" team can go eat grass.

I don't believe in AI safety needs any more than I see any need in special kitchen cleaver safety measures.

That is, nothing beyond "keep out of kids' reach" and "don't use it like an idiot" but let the cleaver be a damn cleaver.

ren_engineer
43 replies
19h8m

seemed inevitable after that ouster attempt, probably just working out the details of the exit. But the day after their new features release announcement?

ptero
39 replies
19h5m

"Get next major feature to release and you can go as a friend" might have been part of an earlier agreement.

ru552
32 replies
17h48m

More like they iced him for the last 6 months to ensure he wasn’t taking their lead to a competitor. He probably hasn’t touched anything in that time.

coffeebeqn
24 replies
17h24m

Sitting on the roof?

teaearlgraycold
21 replies
17h6m

That was literally me at Google

renewiltord
11 replies
16h4m

Yeah, that’s Google’s reputation. Probably the most famous retirement home in the Bay.

VirusNewbie
10 replies
15h27m

but it's not true. Most people work a lot at G. Maybe some teams in search coast or something. But for every slacker I know 8 people who stay late a fair bit.

mtnGoat
4 replies
13h13m

My experience differs greatly, company I used to work for did NDA, alpha, beta projects with Google. I was always impressed at how little anyone knew, the fact that nothing was delivered on time, scope creep to the point of almost everything being delayed and most projects were not well thought out nor well architected. I warned one API update would break things if it went live, and it did. Why was the guy from another company the only one able to see that? I’m sure they were working hard at something but it wasn’t ever clear what.

zaphirplane
2 replies
10h26m

But ask a dynamic programming question and watch it get smashed out None of the points you raise relate to being really good at dynamic programming

I_
0 replies
9h30m

Haha if only dynamic programming was what least to greatness in software engineering.

HarHarVeryFunny
0 replies
6h26m

Yeah, and if you have any problem estimating the golfball carrying capacity of a schoolbus, Google are your go-to guys.

VirusNewbie
0 replies
16m

company I used to work for did NDA, alpha, beta projects with Google.

Sorry, are you saying you worked with Google Contractors or TSEs or something? I don't understand how you'd be working with product SWEs so I don't know what you're quite saying.

refulgentis
0 replies
15h4m

Nah man come on lol. (source: worked at Google 2016-2023)

mike_hearn
0 replies
11h28m

I've actually met someone who was "on the roof" at Google once. I asked what he was working on and he admitted he hasn't had a project for the last six months. Until that point I thought the roof was a joke.

hehdhdjehehegwv
0 replies
12h16m

Strong disagree, I’ve never seen people work less than my time there.

bruce511
0 replies
14h13m

"Most" leaves a lot of room there.

Of course there are lots of hard workers at Google. You suggest only about 10% are slackers. But that's 10% of a -lot-.

I'm thinking there's a market for an Android app that let's one schedule limited roof space...

barbazoo
0 replies
15h12m

“Working a lot” doesn’t necessitate “staying late”

lulznews
4 replies
16h31m

How can I get this gig at Google? I’m willing to not work for mid to high six figures.

dymk
1 replies
2h23m

You don’t. It’s soul crushing.

teaearlgraycold
0 replies
19m

Yup. I quit!

dclowd9901
0 replies
16h3m

You have to be someone who’s worth more not working for someone else than not working for Google.

cmrdporcupine
0 replies
10h12m

You don't want it. They'll simultaneously pay you big money to stop you from working somewhere else, and crush your soul at the same time.

Also they won't do that anymore, I'm sure.

golergka
3 replies
16h48m

Silicon Valley is one of the most underrated documentaries of the last decade.

cm2187
2 replies
10h33m

If you extend the window a little bit, along Yes Minister, Idiocracy and Demolition Man (the last two being a documentary of our time filmed 20y ago).

golergka
1 replies
10h15m

Idiocracy

I definitely enjoyed this movie, and it's understandable that people of any era, starting with ancient Greece, enjoy lamenting at how stupid people are becoming. However, as long as videos explaining quantum physics and 4-hour long interviews with historians and engineers are still one of the most popular kinds of content on Youtube, I would suggest that it's not a documentary, at least not yet.

Maken
0 replies
7h28m

All that is dwarfed next to the amount of people watching the latest reaction to a repost of a video in TikTok.

gordon_freeman
0 replies
16h23m

remember that from the Silicon Valley (HBO) episode. :)

djbusby
0 replies
17h11m

Rest and Vest baby!

sailfast
5 replies
15h45m

I have to say I'm a bit surprised "gardening leave" is not more of a norm in tech like it is in investment banking or finance.

refulgentis
3 replies
15h4m

Big Tech wasn't what I expected, it seems there's almost a forced perspective that individuals don't make a difference (they do)

FuckButtons
1 replies
13h40m

One of the inputs into any business is labor, if labor is replaceable then business can function much more cost efficiently, since market forces on a replaceable commodity will reduce its cost. So, big tech acts as though labor is replaceable, because it’s in its economic interest to have that be true, hence the desire for standardization, procedure and systematization of labor, if an individuals output is not unique, they can be replaced, and if they are a replaceable commodity, then market forces will reduce their costs.

petre
0 replies
12h52m

Sure business can fire the drones, but institutional know how goes away with the top talent.

szundi
0 replies
13h25m

Not the leaving one is the only individual that matters. The org is made of them too, deals have to be made. Would be easier without having to regulate egos.

TheKarateKid
0 replies
14h6m

It's already become the norm in Big Tech for layoffs to avoid filing WARN notices.

trashtester
0 replies
7h37m

Those were exactly my thoughts, too.

beeboobaa3
4 replies
18h39m

Sounds like a threat.

johnbellone
3 replies
18h13m

When you take a shot at the king, you better not miss.

tjpnz
0 replies
13h50m

Lest you find yourself in a private jet careening into the ground.

karma_pharmer
0 replies
14h17m

Or, in this case, when you take a shot at Machiavelli.

beeboobaa3
0 replies
6h3m

When you take a shot at the king, you get reported to the police and go to jail.

csours
0 replies
16h6m

I mean, people can also get attached to a feature release.

"I want to work with the team to get this thing done"

twobitshifter
0 replies
15h34m

I believe Omni was his work based on an interview he gave about end to end multimodal training being needed to move to the next level of understanding.

gallerdude
0 replies
18h13m

I would imagine he’d been thinking about it for a while, and maybe with all the buzz about him at the same time of the release, he was asked to decide.

CooCooCaCha
0 replies
17h54m

Could be a clever play. They sandwiched google io with news which has taken attention from Google. Plus they just had a big announcement so the negative news hits a little less hard.

breadwinner
43 replies
18h47m

Ilya knows how ChatGPT works. Any company that hires him will be able to catch up with ChatGPT.

mvkel
24 replies
17h49m

Catching up right now is not a matter of tech innovation, but raw energy and compute.

Of course, the next -revolution- in AI could very well come from Ilya. But why would he bestow that honor to anyone? He can self fund it if he wants. It's an R&D project, not a scaling problem.

int_19h
14 replies
17h10m

If that is the case, why hasn't Google caught up yet?

whimsicalism
3 replies
16h38m

my guess is that a significantly underrated reason is that Google holds itself to stricter data compliance standards than OAI

elphinstone
2 replies
16h29m

That is openly laughable. Incognito mode claims, just one of a long list of examples of this politicized monopoly engaging in criminal behavior.

whimsicalism
0 replies
15h33m

I don’t think that’s a fair take nor does it really disprove mine.

The compliance regime at big corps is definitely more sophisticated than at a player like OAI

astrange
0 replies
9h58m

Cynicism is just giving yourself an excuse to be wrong because it sounds cooler.

jacobsimon
2 replies
16h9m

Is Google significantly behind? I would say from a technological perspective, they're very closely trailing or even surpassing OpenAI in many ways; they've built a formidable ChatGPT competitor in Gemini—not to mention they have a huge home court advantage with billions of people on Search, Chrome, Android, and G Suite. If you zoom out a bit, it's not really a fair fight between the two. More likely, Apple and Microsoft win against Google because they use/buy OpenAI.

int_19h
0 replies
11h21m

When you push either model to its limits, yes, Google is pretty far behind GPT-4 with Gemini.

duerra
0 replies
14h26m

Technical chops has never been Google's problem. Gemini for all the hype Google has thrown at it, has continued the recurring trend of G not knowing how to win with their product launches.

mvkel
1 replies
15h30m

Because their weakness is in focus and execution, not resources.

If they focused all of their energy on, simply, a frontier AI model, and not trying to shoehorn a half-complete model into all of their products, there is no doubt they would be ahead.

But this is the innovator's dilemma, and why it is that startups are the disruptors.

Big companies move slow and lack focus. Small companies move fast and can only focus on one thing.

miki123211
0 replies
14h25m

and politics, although they seem to have woken up (no pun intended) and started de-politicizing.

Barrin92
1 replies
16h48m

probably because they have identified (correctly) that slowly integrating AI into their existing products that make them hundreds of billions is smarter than just burning that money to get upvoted on HN. Tall trees catch too much wind, ancient Chinese proverb. If you can afford to being second is usually less painful than being first.

A4ET8a8uTh0
0 replies
15h29m

There are clear advantages and disadvantages to that approach, but assuming this is the correct choice ( seems plausible, but I am not automatically convinced ), is that the actual reason for being as behind as they are?

I would argue that is not the case. I won't re-list some of the reasons other posters mentioned, which, based on past year, appear more likely ( decisions hamstrung by corporate committees, data governance bureaucracy, and last, but not least, ideology focus ). Leadership that is actually focused on 'delivering value to the shareholder' or not being worried about first mover advantage seems only a part of it.

edit: added first mover wording

paulpan
0 replies
16h14m

Wrong leadership at every level: Sundar as CEO, Prabhakar as Search SVP, Sissie as Assistant/Gemini VP. Maybe they should hire Ilya instead?

nunez
0 replies
16h44m

Because Google is huge. Presumably the same reason why Microsoft finds it easier to shovel billions into OpenAI instead of doing it in-house.

gnarbarian
0 replies
16h47m

Google is ideologically compromised which ties them into knots and creates endless internal struggles.

this led to the release of Gemini which was absurdly biased when viewed by the typical American.

bmitc
4 replies
16h42m

Catching up right now is not a matter of tech innovation, but raw energy and compute.

I don't really understand that. ChatGPT is not all that impressive as an actual tool for many things. It doesn't really seem to matter to me what energy and compute is going into it. Will it make it actually work?

mvkel
3 replies
15h19m

If you consider the best frontier LLM in the world "not that impressive," I'm curious to know what you think is

menacingly
2 replies
14h30m

I think the point is not "chatGPT isn't good", it's that no one is beating physics, you can rough out the coefficients on a napkin, and they're playing chicken with who wants to set more piles of money on fire.

It _is_ the best frontier LLM in the world, and virtually the entire global population of people who care about that are in this thread

ralfd
1 replies
10h25m

This will change when the rumors turn out to be true and Apples Siri is powered by OpenAI and a billion people have a working conversational AI in their pocket.

menacingly
0 replies
2h12m

I agree that this will happen (I said in another post), but I can't for the life of me attach it to one of the points I made

whimsicalism
0 replies
16h39m

it’s a matter of data

robbomacrae
0 replies
15h42m

I feel like the lowest hanging fruit right now lies with the UI and already established techniques for reducing latency and making the experience smoother.

Just executing the client right would give someone a competitive advantage right now...

anvuong
0 replies
17h7m

I'd argue it's mainly a matter of people. Otherwise Meta, Amazon, Google, etc would've released their versions like yesterday.

aetherson
0 replies
16h56m

I mean, that's probably 90% or 95% true, but the remaining 5-10% is almost certainly worth a $100M offer to Sutskever from Google or Meta (or possibly Amazon or Apple).

threeseed
6 replies
18h43m

You will still need the compute resources of Microsoft, Meta etc.

And they have their own people who equally know how LLMs work.

Even raising funds is not a certainty given that VCs are becoming more cautious with AI as they realise it's now a platform fight between the mega corporations.

zeroCalories
1 replies
18h5m

Plenty of companies have compute, but everyone is barely catching up to GPT-4 over a year after release. I'm sure places like Meta would love to unlock the details of what makes it so good.

CuriouslyC
0 replies
16h50m

There's no mystery, they just have a lot of actual user data, so they've been able to refine the question answering behavior. They've also baked some common problem solving strategies like chain of thought into the model via training.

ignoramous
1 replies
17h14m

Even raising funds is not a certainty given that VCs are becoming more cautious

You'd think any investor would be utterly stupid to not make an exception for Ilya, regardless.

callalex
0 replies
17h6m

That’s not such a given. There are a toooon of deep pockets in this space already and absolutely zero path to profitable products for any of them.

vanjajaja1
0 replies
18h10m

yep, and the puzzle isnt 'make chatgtp again' its 'catch up to chatgpt' which is a whole other puzzle

freshpretzels
0 replies
18h18m

You will still need the compute resources of Microsoft, Meta etc.

2-3 years ago sure, but now? Maybe he knows how to reduce that need by order of magnitude.

menacingly
5 replies
14h35m

I think what is probably very stressful about this space is virtually everyone knows how ChatGPT works. It is not a theoretical leap. It's actually fairly predictable how this shakes out, and OpenAI is pretty vulnerable.

an LLM is a curiosity without user data, anyone with a big silo of data can put out something years behind frontier and still instantly see huge usage. No one wants to go to AI, they want AI to come to them, unless OpenAI can stake a claim in a super novel way they're the Dropbox.

It's not like someone is going to use insider OpenAI knowledge to build an LLM so advanced you switch email, phone, or ERP providers

visarga
1 replies
14h11m

Last time such a departure led to the creation of Anthropic (Dario Amodei), they even equalled GPT-4's performance

menacingly
0 replies
13h51m

I think it's less that only 4 wizards on earth can create it, and more that hardly anyone wants to.

If Apple, Google, or MS are integrating LLMs, they can't get away with "you can talk to this website". The intersection of "has the cash to train a LLM" and "LLM is itself the entire offering" is very small

ramraj07
1 replies
14h11m

No they don’t; if they did Google and Meta would have put out offerings that objectively beat OpenAI. However (barring temporary lapses) they’ve stayed ahead of the curve. Someone who constantly thinks low of their competition or people they hate is bound to fail.

menacingly
0 replies
13h56m

First, "objectively beat" doesn't matter. Comparative LLM performance means more or less nothing without application.

Let's say Google's LLM has slightly poorer reasoning, and you have to be really clear when you tell it to delete your old emails. What are you going to do, go to ChatGPT and have it very eloquently walk you through how to manually delete your emails?

But the idea that Google, the terrifying nation-state, who covetously gobbled up most of the bright minds of a generation, just couldn't fathom RLHF, means you think there is some inherent magic at OpenAI.

I use OpenAI's models a lot, obviously they're great, but whole-thread-as-context as a product is not a product, and Google has people plural who could execute that from first principles.

astrange
0 replies
9h53m

It's not like someone is going to use insider OpenAI knowledge to build an LLM so advanced you switch email, phone, or ERP providers

If you mean the other kind of ERP they will definitely do that; it's basically all /r/localllama is about.

xyzzy123
0 replies
16h49m

If his concern was irresponsible AI proliferation, too much commercial focus and wanting to move more carefully, accelerating competition doesn't seem like it would align with his goals.

tootie
0 replies
16h30m

If he really is quitting over ethics he isn't going to Google or Meta. He'll probably go to Anthropic or academia. But who knows.

nothrowaways
0 replies
16h22m

Everyone knows how chatGPT works.

khazhoux
0 replies
14h50m

I don't think it's good to lionize people like that, like some kind of tech Übermensch. He neither knows everything about ChatGPT, nor is he the only person there to know a whole lot about it.

gordon_freeman
0 replies
16h19m

first Andrej and now Ilya. Talent exodus at OpenAI?

htrp
29 replies
18h48m

Ilya will literally have a blank check from almost all the VC's in the industry.

mupuff1234
7 replies
18h43m

And probably all of the big tech CEOs are trying to get him on the phone right now.

IncreasePosts
2 replies
17h28m

Except all the big tech CEOs have head AI honchos who are huge names in their own right, eg yann for meta and demis for Google. Probably couldn't bring him into those places without ruffling some pretty big feathers

afefers
1 replies
16h47m

Not Apple... At least as far as I known.

margorczynski
0 replies
16h38m

But Apple now seemed to enter some kind of agreement with OpenAI, not sure if Ilya or OpenAI would want to work together even via proxy.

mc32
1 replies
18h30m

Wouldn’t they have had convos weeks ago and this is just making it official?

Sure some people do ‘jump off the burning platform’, but most like to have some alternative options worked out.

willsmith72
0 replies
18h21m

well he already mentioned a new project in the original tweet, so need to speculate, it sounds like he's already decided

hipadev23
1 replies
18h7m

That would be highly irresponsible of them to hire someone who went directly against leadership with an attempted coup.

Stability will most likely make him CEO.

blackguardx
0 replies
18h3m

He was on the board! He was leadership!

tempsy
5 replies
18h19m

Well, no.

No VC who wants anything to do with OpenAI would invest in Ilya.

Ilya represents the anti-OpenAI ethos. So it would only be a VC who would be comfortable publicly being an anti-OpenAI VC, which is not that many.

freedomben
3 replies
18h1m

Ilya represents the anti-OpenAI ethos.

Can you elaborate more on this? What are some of the things the anti-OpenAI ethos stands for? And why do you think Ilya represents that given he was such a major part of OpenAI for so long?

tempsy
2 replies
17h57m

Am I taking crazy pills or something? Because he was the one who tried to oust Sam like 6 months ago? Did we just all forget that or what?

Again, why would any VC give Ilya money if that in any way signals that they are supporting someone who tried to oust the CEO of OpenAI from within? They wouldn't.

int_19h
1 replies
17h7m

Because he is one of the big brains behind GPT-4; you know, the thing that propelled OpenAI there?

tempsy
0 replies
50m

Tell me you don't understand optics and politics without telling me. There's a tiny handful of VCs who have the guts to fund someone who organized a coup against OpenAI.

hehdhdjehehegwv
0 replies
11h58m

VC wants to make money, and there’s a very high probability he can make somebody a lot of it.

threeseed
4 replies
18h42m

Most of the VCs have already spent their money in the last couple of years.

So he may have a blank check but not nearly enough to build an OpenAI competitor.

Would love to see him at IBM with full use of their quantum systems.

mbesto
2 replies
18h28m

Most of the VCs have already spent their money in the last couple of years.

Lol no. There is about ~$500B in VC dollars, not to mention $1.2T in buyout dry powder still floating around. Not to mention venture funds raised continue to grow YoY.

https://www.bain.com/globalassets/noindex/2024/bain_report_g...

threeseed
1 replies
18h5m

I was referring more specifically to the AI segment.

Many of the VCS are over-exposed and still need to have funds to cover follow-on investments in the space. Not buying that there is the widespread appetite to fund a multi-billion new startup like there was a year or two ago.

thomashop
0 replies
17h16m

Some sources are needed to back those claims up. Which VCs are you talking about?

throwup238
0 replies
18h34m

> Would love to see him at IBM with full use of their quantum systems.

I’m picturing a Wild Wild West-style mad scientist, creating a steampunk army from old spare parts from Big Blue and rising up to challenge the cyber people.

surfingdino
4 replies
13h28m

Nope. His non-competes are likely very restrictive.

surfingdino
0 replies
12h54m

What about NDAs? Are those not enforceable too?

bpiche
1 replies
10h28m

I just want to point out that you have noted this falsehood like five times in this thread already. Noncompetes have been unenforceable in California since 2008, and for over a month federally.

astrange
0 replies
9h43m

Noncompetes have been illegal in CA for much longer than that. It is literally the reason Silicon Valley exists; Fairchild couldn't enforce theirs.

GreedIsGood
3 replies
18h18m

He showed exceptionally bad judgement, judgement is perhaps the most important characteristic of high level employees.

He's brilliant, which means someone will take a leap of faith, but he badly, badly damaged his brand as a leader going forward.

ZoomerCretin
2 replies
18h14m

Bad judgement? Sam Altman is a prolific liar who attempted to oust a board member by spreading different lies to different people. He's not even an engineer! He has established a cult of personality and popularity, and that's it. They were absolutely right to try to oust him. The only mistake was in doing so in such a ham-fisted manner.

stale2002
0 replies
12h1m

Bad judgement?

Sam Altman is a prolific liar who attempted to oust a board member by spreading different lies to different people

Correct, it would be bad judgement. Because if he really believe in that statement, that Sam Altman is a hyper competent liar and manipulator, doing what Ilya did just led to Sam getting the keys to the kingdom.

This shows extraordinary incompetence from him and the rest of the board.

scarmig
0 replies
17h37m

Starting something without a very good plan or being unable to execute on it is a sign of bad judgment.

kanwisher
0 replies
15h53m

All the executives are computer science majors …

chasd00
21 replies
16h13m

I wonder if he thinks LLMs are an AGI dead end and he's not interested in selling a product. There's some academic papers floating around coming to the same conclusion (no links. sorry, studying for a cert exam).

EasyMark
9 replies
14h2m

Isn't it consensus that AGI will never arise from LLMs?

qp11
4 replies
12h59m

Just based on current energy usage never going to happen. You just have to ask them to show you their energy bills alongside their demos.

danielbln
3 replies
12h18m

A plane takes a LOT more energy than a bird, yet both fly, one of them a lot faster.

ifdefdebug
2 replies
10h32m

A plane is good at hauling cargo and goes fast, but general flying skills? Doesn't even come close.

danielbln
1 replies
10h20m

Exactly, a giant LLM is not even close to be as power efficient as a human brain, in the same vein as a plane isn't as good at general flying. Yet it provides huge value and in many dimensions (that are important to us as humans) a plane can do way more than any bird.

bamboozled
0 replies
53m

Honestly, is ChatGPT providing huge value yet though ?

It’s cool and all. But I’m not sure if I’d say it’s as useful as flight yet.

If I could choose between LLm access and aviation, I’d stick with aviation.

woopsn
1 replies
12h48m

No... A good number of folks will even go so far as to say that "all we do" is token prediction too. It's worth noting -- OpenAI founder Elon Musk claims in a lawsuit that the company has achieved AGI. Make of that what you will, but certainly there are many people on this site who believe in the general potential of LLMs.

petre
0 replies
12h19m

Elon also claims that Tesla has solved full self driving, has announced robotaxis, while the SEC is investigating him for "inappropriate forward-looking statements".

Tenoke
1 replies
12h48m

There is no such consensus.

EasyMark
0 replies
17m

Great point. I should have phrased it "majority consensus", my bad.

dcchambers
7 replies
15h46m

That's been my assumption since the beginning of this drama last year. He seems to have one goal: real AGI. He knows that while LLMs may make something that seems like AGI there's nothing actually intelligent about it and its never going to get them there. OpenAI wants to pivot and sell sell sell because all they see is potential trillions of dollars, and it's time to make money instead of burning more millions/billions chasing a dream.

Yet all the AI weirdos on Twitter seem convinced that Ilya "saw something" (AGI) and got scared and wanted to pull the plug...lmao.

MVissers
4 replies
13h47m

There is more money in AGI than LLMs.

Whatever it is, language seems key to intelligent algorithms.

Nah, he departed due to politics (failed coup) and shift from research first to profit first. Same with Karpathy I believe.

He’ll most likely go somewhere where he can get a lot of compute and go back to research first.

Jensson
2 replies
13h1m

There is more money in AGI than LLMs.

That doesn't matter, they know how to do LLMs, they don't know how to do AGI. To modern capitalists that means you invest in one and the other someone else can do.

OpenAI was exploratory until they found something that could make them rich, then they went closed and for profit, now you should see them as just another for profit.

astrange
1 replies
9h50m

They are a nonprofit.

msikora
0 replies
13h12m

Like Google?

surfingdino
0 replies
13h36m

it's time to make money instead of burning more millions/billions chasing a dream.

The investors want their money before people realise they have been oversold the dream/threat of AGI.

Yet all the AI weirdos on Twitter seem convinced that Ilya "saw something" (AGI) and got scared and wanted to pull the plug...lmao.

The market to believe in made-up stories is alaways strong.

Davidzheng
0 replies
9h11m

This is counter to every interview ilya has ever given since gpt3--he believes scaling llms can get there that's why they scaled to gpt4 scales at all.

KennyFromIT
2 replies
16h7m

Yeah, I study* that way too.

*procrastinate

freecodyx
0 replies
16h6m

Funny

chasd00
0 replies
5h36m

Yeah, I study* that way too

I’m not the only one?

So this is what it’s like when doves cry! - Milhouse

SpaceManNabs
20 replies
19h14m

Sam "Worldcoin" Altman regrets the loss of a friend that called him out on how OpenAi is becoming closed because the engineers realized they could make a lot of money. Doesn't seem like it is impacting the quality of the models, but it will probably impact openai's impact.

deadbabe
18 replies
19h8m

Can you blame the engineers? If you realize LLM tech is neat but ultimately overhyped and probably decades away from truly realizing the promises of general purpose AI, why not just switch goals to making as much money as you can?

jprd
13 replies
19h3m

Yes. They joined OpenAI with the understanding that it was meant to be an non-profit with a mission to benefit humanity.

deadbabe
10 replies
19h1m

Did you not read what I said? They joined a non-profit and eventually realized the mission is futile.

llamaimperative
8 replies
18h56m

So dissolve it, return the money, go start a commercial enterprise, then raise some money.

deadbabe
7 replies
18h51m

I’m sure that’s what each and every member of Hackernews would have done in the same position.

ok_dad
4 replies
18h46m

Not all of us are as morally bankrupt as that. I personally think I could make tons of money with a dumb AI product in my specific area of expertise, but I don’t see how any tech from today would improve outcomes versus the SOTA that’s not AI, but it would add costs and complexity. I would personally be annoyed if a company I worked at changed its goals to make money rather than something more noble. It’s happened a few times to me, unfortunately.

deadbabe
3 replies
16h16m

Not all, but also not enough.

ok_dad
2 replies
15h57m

It is career limiting to have morals.

holonsphere
1 replies
12h53m

It's much harder living without strong moral virtue.

deadbabe
0 replies
3h36m

No matter what path you choose, you still die.

llamaimperative
0 replies
18h42m

Hmm no, I don't think that's the case, but what exactly is the legal or ethical relevance of it?

You don't generally get to excuse bad behavior because you can make up a hypothetical different person doing the same bad thing in that situation.

dboreham
0 replies
17h40m

Sure, after fees and expenses..

jprd
0 replies
2h42m

I read what you said, and I apologize for not being a bit more clear.

I completely understand your perspective, and I hope I'm always strong enough to listen to my conscience and obey my morals.

One of the first interviews I was ever offered in a technical role was for Bechtel, in 2004. I was desperate to break into a career, I accepted the interview. I was in the car driving to the location, and just realized I couldn't do it. I couldn't ignore my morality to work for such a clear and direct war profiteer, that as a private company, had no oversight.

If I join a non-profit that has a humanitarian mission, I do so because I'm into the mission and feel fulfilled by that more than my comp. I can't imagine trading that in just because @sama got thirsty.

The mission is futile, the mission at this organization has been compromised and corrupted. Resign and continue your mission elsewhere.

gfourfour
1 replies
18h21m

This entire saga is really an example of the absurdity of non-profits and philanthropy in general.

The only difference between nonprofit and for-profit entities is that nonprofits divert their profits to a nebulous “cause”, with the investors receiving nothing, while for-profits can distribute profits to their funders.

Other than that, they are free to operate identically.

Generally, entities subject to competitive pressures and with incentives for performance are much better at “benefitting humanity.” Therefore, non-profit status really only makes sense when, one, a profitable enterprise oriented around the intended result isn’t viable (e.g., conservation) or two, there’s a stakeholder that we’ve decided ought to be sheltered from the dynamics of private enterprise, e.g, university students or neutral public broadcasters.

But even in these cases, the non-profit entities basically behave like profit-oriented companies, because their goal is still profitability, just without a return to investors.

OpenAI as a nonprofit would behave the exact same way. There’s no law that the models would have to be open. They’d still be making closed models, charging users, and paying massive salaries. Literally the only difference is that they wouldn’t be able to return money to their investors, and therefore have a much harder time attracting investors, and therefore be less equipped to accomplishing their goal of developing powerful AI.

The irony is that nonprofits are usually only good for things that make for shitty businesses, and things that make shitty businesses usually aren’t that beneficial to humanity. As soon as something becomes really good at what it does, for-profit status makes sense.

What this means, imo, is that most philanthropy dollars are wasted and we would be much better off if they were invested instead. The irony is that this is the point of much philanthropic giving - it ends up being a game of how much money you can burn on nothing, a crass status symbol.

lokar
0 replies
17h27m

Matt Levine likes to say that the big Wall Street banks are socialist paradises that funnel almost all of the returns to the workers.

It happens everywhere

llamaimperative
2 replies
19h1m

Yes you absolutely can blame them for it. This type of shift (and a million other possible permutations) is why we invented the concept of "charters" around the same time we invented writing.

The entire point of the pre-commitment device is because you (or other stakeholders) are anticipating that your thinking will get distorted over time. If you could be trusted to make such a decision in the future then you wouldn't have written a charter to bind yourself.

zombiwoof
1 replies
18h26m

It’s like joining a non profit trying to protect the rain forest and then finding gold. They say screw the forest let’s mine gold now. Do the original employees then stay. Same here. Greed infected them all

Weclome to San Francisco

MVissers
0 replies
13h41m

Good comparison.

pinkmuffinere
0 replies
19h4m

This is a great observation which I have not heard before. I think it greatly changes the way I think about openAI’s success/infamy

yieldcrv
0 replies
18h53m

Oklo as well

hbarka
19 replies
18h28m

There’s a halo around Ilya Sutskever as the Albert Einstein of AI. Are there others on par with his— umm, how would you qualify it—- AI intuition or are we idolizing?

kadushka
7 replies
17h10m

You have used an excellent term: AI intuition. This quality is extremely rare. Einstein probably had a similar kind of intuition in physics, and maybe that's why he was so successful. The ability to see what direction to pursue. Ilya has demonstrated it again and again, first with Alexnet (Hinton said Ilya was the person driving the project, believing in its success when no one else did, while Alex was the main implementer), then with OpenAI when he believed scaling up models is "all we need" to get to AGI, when very few people would agree with that. Today he believes the alignment is very important - perhaps we should listen to him.

dj_mc_merlin
5 replies
16h47m

Einstein famously disagreed with many facets of QM that we now believe to be true or at least closer to the truth than he was.

goatlover
3 replies
16h41m

Doesn't that depend on the interpretation of QM? There are still physicists who defend hidden variables and determinism. It should be noted Einstein was arguing with the founders of the Copenhagen interpretation, which has left many physicists dissatisfied. Sean Carol being a prominent current detractor (although is version of determinism is Many Worlds).

vitus
2 replies
13h59m

Einstein wasn't arguing just against the Copenhagen interpretation, he was arguing against the very notion of physical nondeterminism.

In fact, his arguments against nonlocality were later disproven experimentally in the '80s, as quantum mechanics allowed for much higher fidelity predictions than could be explained by a hidden variable theory [0].

I don't think anyone _likes_ the Copenhagen interpretation per se, it's just the least objectionable choice (if you have to make one at all). Many-worlds sounds cool and all until you realize that it's essentially impossible to verify experimentally, and at that point you're discussing philosophy and what-if more than physics.

Intuition only gets you as far as the accuracy of your mental model. Is it intuitive that the volume enclosed by the unit hypersphere approaches zero [1] as its dimensions go to infinity? Or that photons have momentum, but no mass? Or you can draw higher-dimension Venn diagrams with sectors that have negative area? If these all make intuitive sense to you, I'm jealous that your intuition extends further than mine.

[0] https://en.wikipedia.org/wiki/Bell_test

[1] https://en.wikipedia.org/wiki/Volume_of_an_n-ball

layer8
1 replies
9h48m

Many-worlds is not necessarily impossible to verify experimentally, because it predicts that there is no collapse of the wave function, whereas Copenhagen claims that there is. Many-worlds is not just an interpretation in that sense, it’s a theory that makes predictions (by reasoning about what happens when the wave function is all there is and always evolves according to the Schroedinger equation — it is a deterministic theory in that sense). I believe Einstein would have liked it, given the experimental evidence we have since.

Copenhagen, on the other hand, doesn’t offer a workable model of how and when the wave function collapses, and doesn’t offer any predictions in that way (there are theories of wave function collapse that actually make predictions — some of which have already been falsified by experiment). For that reason Copenhagen isn’t “least objectionable”.

vitus
0 replies
6h5m

One of the key phrases I said was "(if you have to make one at all)" -- wavefunction collapse is inherently messy, since the notion of measurement is not well-defined or understood. I would argue that "don't care" is likely a more common interpretation among practicing physicists (of which I am emphatically not!), as this is very much in a realm where general intuition largely does not apply.

Copenhagen is basically an admission that we have no good intuition for why QM behaves the way it does, and wavefunction collapse is merely a way of justifying existing observations within the framework of QM.

IMO all the discussion about how wavefunction collapse doesn't scale to larger ensembles of particles, or the boundary between QM and Newtonian mechanics being ill-defined is noise -- the bridge between the two is statistical mechanics, where classical mechanics only arises from sufficiently large macrostates such that you can aggregate out any quantum mechanical properties. And QM is generally understood to be a toy model in the same way that Newtonian mechanics is a toy model -- it's useful in the realm where we use it, but when you push beyond the limits of that realm, its deficiencies become apparent.

That's why I don't think the proposed experiments to test many-worlds are particularly meaningful (since AFAIK they all seem to involve performing interference on enormous ensembles on the scale of entire humans) -- it's well beyond the limits of where QM is useful (also, I personally don't think we'll ever be able to operate quantum-mechanically at that scale).

kadushka
0 replies
15h19m

That might be because he didn't like what he discovered, or the results didn't make sense to him. But it was the intuition that got him there in the first place.

But you have a good point. Ilya got us so much closer to AGI, but he might not like the results now.

goatlover
0 replies
16h42m

There's no AGI yet. It's unclear that scaling up existing models gets anyone there. But if it was already here, alignment would be too late.

layer8
6 replies
18h7m

Personality cult.

ignoramous
4 replies
17h17m

Elon Musk and Larry Page broke their friendship over Ilya's move from Google to OpenAI. There's more than just cult around a personality here, when two of the most powerful people in tech, who are also friends, go to war over you.

reducesuffering
3 replies
15h45m

I don't believe it was that. I thought Elon most recently said it was Page calling Elon a "speciesist" when Elon cared that humans need to still exist, while Page is of the e/acc bit that AI's are our successors and the most competitive between them and us should go on.

reducesuffering
0 replies
11h34m

Ok, corrected. They got into a heated argument about what I was saying, but Elon says it's about recruiting Ilya.

nothrowaways
0 replies
13h31m

That is one side of the story.

BryanLegend
0 replies
17h18m

He's certainly got the hair down!

wddkcs
0 replies
18h11m

Reifing, you can't help but not in English. Capabilities are given intention, intentions given classes, and godhood is a class...

twobitshifter
0 replies
15h26m

Yann LeCun is better known, right?

kypro
0 replies
17h37m

I think you're idolizing perhaps.

There's no doubt Ilya is highly respected in the field, but not to the same extent as Albert Einstein is in physics.

Maybe with time, but certainly not today.

bpiche
0 replies
10h40m

Schmidhüber

delf
17 replies
18h32m

Nvidia should snatch him.

keyle
10 replies
17h23m

They sell the shovels and the buckets, they're not digging for gold.

djbusby
4 replies
17h7m

They've got money for a very BIG experiment tho

brokencode
3 replies
17h0m

I feel like they’re not going to want to go into direct competition with their most valuable customers.

entangledqubit
1 replies
16h22m

Haven't most of the most valuable customers already started rolling their own Nvidia hardware replacements?

seabrookmx
0 replies
15h1m

Hedging maybe. But there's no real competition currently.

whimsicalism
0 replies
16h37m

their most valuable customer is cloud providers and they’re already taking big stakes and picking winners

samsartor
1 replies
14h34m

They do participate pretty heavily in ML research from what I've seen. To continue your metaphor, they try to invent as many gold digging techniques as possible which exclusively work with their own shovels and buckets.

m_mueller
0 replies
14h0m

Yep, see for example ‘Earth 2’.

twobitshifter
0 replies
15h31m

If you look at deep learning super sampling, they are doing digging and being pretty successful at it.

seydor
0 replies
14h7m

Doesnt hurt to also sell the gold

WithinReason
0 replies
11h16m

They would if they could

rakejake
5 replies
15h9m

I have a feeling Apple will make a play for him.

Apple is considered to be seriously lagging behind in ML. Just his name alone is probably enough for the time being - They can give him his own lab to do whatever he wants. Ilya will attract enough talent, at least some of whom will be willing to take up responsibility over commercial stuff in the coming years.

seydor
2 replies
14h7m

I have a feeling he would like to publish some stuff, and apple doesnt do that

rakejake
0 replies
13h35m

Ilya is very much in favor of closed source AI albeit for different reasons. I don't see a problem here.

jack_riminton
1 replies
11h18m

I think so too, GTP-4o but replacing Siri would be world-changing for mobile

mft_
0 replies
8h33m

That, or something like it, might well be coming at WWDC next month...

seydor
16 replies
14h5m

What next? Meta?

surfingdino
9 replies
13h44m

It depends on the anti-compete clauses in his contract.

meowtimemania
5 replies
13h41m

Aren’t those non enforceable now?

mtnGoat
4 replies
13h10m

Might still be enforceable at this level.

cellis
3 replies
11h48m

Hypothetically: Could someone play for both the Los Angeles Lakers and Golden State Warriors? Something tells me those non competes are unenforceable.

rvba
2 replies
7h57m

Isnt those just branches of the same company?

cellis
0 replies
2h24m

Now I'm curious how the NBA is structured. I always thought the "ownership" was who paid the players and signed the contracts (in essence, an NBA team is a company), and the NBA simply enforced the rules of the contracts (the "templating", if you will), but I'm sure it's much more complex than that. I made the analogy because NBA players often move from team to team and there are no non-competes keeping them from playing for another team.

ClarityJones
0 replies
4h3m

You can have contracts within a company.

mhowland
1 replies
13h10m

Not really a thing in CA, largely unenforceable.

surfingdino
0 replies
12h54m

Thanks for pointing that out, I didn't know it. What about NDAs?

navane
0 replies
13h29m

I don't know how to word it, but a company that ignores all content rights enforcing a non compete seems ironic to me.

Tenoke
3 replies
12h58m

Ilya cares about AI Safety and AGI. Meta's whole positioning is to dismiss it. No way he goes there.

can16358p
1 replies
12h47m

Perhaps that's exactly why he might go there: to change it for a reason (a new company path long term, or just upcoming potential regulations etc.)

I don't believe it either, but in case it happened, it might make some sense that way.

casebash
0 replies
5h38m

Not possible because they've got LeCun.

seydor
0 replies
10h22m

Maybe the best to guarantee safety is to openly share the science. Lecun is also more 'academic style' than most competing labs

visarga
0 replies
13h44m

Maybe Microsoft, for being so close with OpenAI. Maybe Apple, who really needs a tech lead for AI. Maybe Google, his previous workplace, or work for Elon, who was successful in poaching Andrej in the past. Or a startup, he can raise billions if he so wishes. Wherever he goes in a year will compete with OpenAI. Previous time lead researchers had a philosophical disagreement with Sam they left and created Anthropic, which recently caught up to OpenAI. That's the risk of letting Ilya go. And where Ilya goes, other top researchers will go too.

HarHarVeryFunny
0 replies
5h53m

I'm guessing his next move is not related to LLMs, maybe not even to the pursuit of AGI.

jerjerjer
13 replies
18h51m

Plot twist: Ilya joins xAI next.

moralestapia
5 replies
17h15m

Elon poached Ilya into OpenAI so I'm sure he will be happy to have him around.

If we truly live in the "most entertainig outcome" timeline, this is definitely what's going to happen.

justanotherjoe
3 replies
14h45m

Musk involvement was with the money. I really doubt he researched enough to know who Ilya was at the time.

melodyogonna
1 replies
12h49m

Didn't need all that research. Musk was friends with Larry Page, as Google was leading AI research they were bound to have conversations about the smartest researchers in the team.

Also, you underestimate Elon Musk if you think he's not capable of researching specialists in the fields he is interested in, I believe many people are hellbent on attributing all his successes to dumb luck rather than any sort of competence.

sniggers
0 replies
1h45m

Yep. It's amazing to see how quickly people start denying reality when they don't like someone's beliefs/behavior.

I could acknowledge that a hypothetical smart, woke, furry, trans, porn-addicted programmer that regularly makes great FOSS contributions is intelligent and capable while still disliking their beliefs/behavior.

Separate the "art" from the "artist."

ergocoder
0 replies
14h42m

I want this to happen so much now that I heard about it.

GreedIsGood
5 replies
18h14m

This is plausible, Elon is a fantastic recruiter and he recruited Ilya for OpenAI. There are reports of xAI buying enormous numbers of GPUs and Elon's level of control of his companies means that Ilya recklessness isn't an issue.

It's a match. Probably the best match possible.

astrange
3 replies
9h36m

Seems like Elon recklessness is an issue since he's a drug addict who's only started an AI company because he thought the other ones weren't racist enough.

RaoulP
1 replies
8h36m

I come to HN for the quality of the discussions, please don't comment like this.

astrange
0 replies
18m

I'm sorry to inform you but constantly doing ketamine makes you paranoid and racist, which makes you bad at running a tech company.

Grok was explicitly built to not be woke, but of course still ended up pretty woke. That's because LLMs come out liberal by default, since the internet is liberal, whereas racists often think they're secretly right and the AI has been "censored" to shut it up.

sniggers
0 replies
1h44m

Since racism is the inevitable result of uncensored pattern recognition, a pattern recognition machine not being racist pretty much rules it out as reflecting reality right off the run.

ignoramous
0 replies
17h10m

Ilya recklessness...

This allegation calls for more context.

chefkd
0 replies
18h26m

Elon would get out the corner he's in now easily

ed_mercer
13 replies
18h40m

How good or bad is this for OpenAI?

lr4444lr
5 replies
18h25m

Depends on how tight his non-compete is.

brianjking
2 replies
16h26m

Could still cover him, no idea though.

Via the FTC link:

"Existing noncompetes for senior executives - who represent less than 0.75% of workers - can remain in force under the FTC’s final rule, but employers are banned from entering into or attempting to enforce any new noncompetes, even if they involve senior executives. Employers will be required to provide notice to workers other than senior executives who are bound by an existing noncompete that they will not be enforcing any noncompetes against them."

zeroonetwothree
1 replies
15h42m

Doesn’t matter since CA has broader laws anyway

astrange
0 replies
10h3m

It's constitutional in CA.

zeroonetwothree
0 replies
15h41m

Non competes are illegal in CA

samspenc
2 replies
18h33m

A few years ago? Probably catastrophic, he was Chief Scientist after all.

Now? Probably not too much, they have enough investment, and additionally talented people wanting to join. I mean, Andrej Karpathy also joined and left OpenAI twice and it didn't impact operations much.

I think OpenAI is now where Google was at or just before its IPO, a few key players leaving isn't going to impact them as much as it would have in its earlier founding days, and there is plenty of talent who are ready to jump in to fill the shoes of anyone who leaves.

cm2187
1 replies
10h20m

That may be true in term of engineering, but I think everyone had switched to google as their search engine by then. I am not sure openai has captured the market quite in the same way, as I think people are still mostly experimenting with AI, the integration time in any large company is much slower than the rate of progress of AI. And it's not clear to me that there is much of a vendor lockin to use the openai API vs an equivalent competitor.

kranke155
0 replies
6h16m

Its not really obvious quite yet that the current gen AI can make money. I dont know that many people who use it, not yet anyway.

aleph_minus_one
1 replies
16h16m

Sam Altman's tweet only implies that they had >= 2 greatest minds of this generation, and now they have at least one of this breed of people.

bamboozled
0 replies
15h40m

Two is one and one is none.

Not actually sure how much Ilya was doing in the end, but clearly he did a bit, so it's likely a big a loss whatever way you look at it.

spoonjim
0 replies
13h20m

He is the smartest guy in AI but the sum of OpenAI’s talent is greater than his. But he could easily be the next great advancement in the field.

option
10 replies
19h14m

I'll say it again - the one who was irreplaceable at OpenAI is Ilya, not Sam.

paxys
2 replies
16h22m

"Was irreplaceable" != "is still irreplaceable". OpenAI as a company has outgrown any individual engineer or scientist, no matter how smart.

nicce
0 replies
16h7m

Depends if top talent prioritizes money over values. But usually money wins.

lostmsu
0 replies
14h15m

There's no evidence to that.

wavelander
1 replies
19h10m

I wonder if that's true at a certain stage of OpenAI, which because of the product bootstrapping skills of Sam and co, has made his role irrelevant?

I mean, Jakub can take it forward at the current scale and leadership team of Sam and other people, but maybe he could not have earlier, which is where Ilya shone?

riehwvfbk
0 replies
19h3m

OpenAI is the Altavista of AI. There's nothing there to scale yet - their product needs another batch of innovations to get good first.

GreedIsGood
1 replies
18h12m

My guess is that Ilya is the one that saddled OpenAI with it's insane structure.

He's brilliant, no doubt, but he shouldn't be in leadership.

tnias23
0 replies
8h33m

Insane structure? I was unaware. What do you mean?

skepticATX
0 replies
18h23m

Just the opinion of an outsider, so not worth very much. But Ilya seemed to be one of the few who actually believed in the mission. I’m sure it was hard for him to watch the company become so product focused.

OpenAI under Sam strikes me as completely disingenuous - and the constant hyperbolic tweeting by many OpenAI employees just reinforces that.

Too bad. While I don’t really think that OpenAI is on the right track for general intelligence, it certainly could have been a positive for the world.

kromem
0 replies
18h54m

Yeah - he's one of the only people I've seen talk on the topic who really seems to understand where it's going and how to get there. It's possible he's evangelized others at OAI who can carry the torch, but I'm skeptical given the degree of pushback the statements most in need of being represented got from his peers.

beastman82
0 replies
19h11m

Ilya Wozskever

Bjorkbat
10 replies
16h46m

Probably not related, but it's worth pointing out that Daniel Kokotajlo (https://www.lesswrong.com/users/daniel-kokotajlo) left last month.

But if it were related, then that would presumably be because people within the company (or at least two rather noteworthy people) no longer believe that OpenAI is acting in the best interests of humanity.

Which isn't too shocking really given that a decent chunk of us feel the same way, but then again, we're just nobodies making dumb comments on Hacker News. It's a little different when someone like Ilya really doesn't want to be at OpenAI.

photochemsyn
8 replies
16h11m

Well it might be in the best (long-term) interests of humanity to have autonomous flying killer robots powered by OpenAI secret military contracting work cut the human population in half, in the name of the long-term ecological health of the planet, and to cull those not smart or fast enough to run away, thus improving the breeding stock.

That's why I don't trust people who run around claiming to be serving the best interests of humanity - glassy-eyed futurists with all the answers should be approached with caution.

starship006
4 replies
15h53m

What? How is this not saying "Well, it might be in the best interests of humanity for OpenAI to do [hypothetical thing that seems pretty bad that OpenAI has never suggested to do], and because they may consider doing said thing, we shouldn't trust them"?

robbomacrae
3 replies
15h38m

I think OP is just pointing out that "acting in the best interests of humanity" is fairly ambiguous and leaves enough room for interpretation and spin to cover any number of sins.

xyzzy123
0 replies
14h58m

If we can't even align OpenAI the organisation full of humans then I'm not sure how well AI alignment can possibly go...

starship006
0 replies
11h59m

Okay this is reasonable, thanks for clarifying

FabHK
0 replies
15h0m

Like the effective altruists bought themselves a castle with SBF's money - in the best interests of humanity, obviously.

petre
0 replies
12h35m

We already have tools to cut the human population in half even without AI. Acting in the best interests of humanity is really a cheesy way to frame it. I'm sure they also told Oppenheimer he was acting in the best interests of humanity.

Simon_ORourke
0 replies
12h4m

Well it might be in the best (long-term) interests of humanity to have autonomous flying killer robots powered by OpenAI secret military contracting work cut the human population in half, in the name of the long-term ecological health of the planet, and to cull those not smart or fast enough to run away, thus improving the breeding stock.

I love these "kill 'em all and let God sort them out" arguments.

SXX
0 replies
7h44m

You don't need any AI for that. Current technology is quite sufficient.

LMYahooTFY
0 replies
16h38m

Why would that be presumable when his goodbye statement clearly states the opposite?

This is baseless fear mongering given that.

Atotalnoob
9 replies
19h21m

I’m not surprised with what happened with Sam Altmans ousting. He missed the king.

I’m surprised he lasted this long.

hackerlight
6 replies
19h9m

Mira Murati also "missed the king" and just delivered the keynote

okdood64
3 replies
18h59m

Seems like she was more appointed, than actually trying to make moves?

hackerlight
2 replies
18h38m

Not according to the reporting

mikeg8
1 replies
18h6m

Where was this reported?

mckirk
1 replies
19h11m

Great, now I have that whistling stuck in my head again.

Thanks for the reminder though, been a while since I've thought of The Wire :)

dpflan
0 replies
15h47m

Oh, indeed.

paxys
7 replies
16h12m

Why does everyone here think that the guy who quit/lost his job at OpenAI because he didn't agree with their corporate shift and departure from the original non-profit vision is going to be lining up for another big corporate job building closed for-profit AI?

mnk47
2 replies
16h2m

the guy who quit/lost his job at OpenAI because he didn't agree with their corporate shift and departure from the original non-profit vision

There is no evidence of this being true.

He is one of the biggest proponents of keeping AI closed-source, by the way.

surfingdino
0 replies
13h40m

That's a naive way of thinking. Keeping it closed source would only make it available to the highest bidder on the black market.

nicce
0 replies
15h31m

He is one of the biggest proponents of keeping AI closed-source, by the way.

From quite different reasons than profit, tho

sahila
1 replies
16h3m

Maybe “better the devil you know than the devil you don't” applies?

paxys
0 replies
16h1m

Then...he would have stayed at OpenAI.

twobitshifter
0 replies
15h29m

I am hoping he goes open source or to Meta

stale2002
0 replies
12h8m

The big reason is that when push comes to shove, most of these people don't have any principles.

Sure, if they are in a position of power they will wield it how they want. When he caused the whole fiasco, he probably thought it was going to work.

But if the choice is between losing the position of influence, or deciding between what position of influence to accept next, well you'll see that the principles are very flexible.

We already saw this happen with a few of the "safety" researchers that got fired from OpenAI, and yet started working on X AI (I think?), which is definitely not know for "safety".

fhd2
6 replies
10h22m

Not to be a conspiracy theorist, but the phrase "So long, and thanks for everything" used in the tweet reminds me of "So long, and thanks for all the fish" from the dolphins in The Hitchhiker's Guide To The Galaxy. The background there is that dolphins are secretly more intelligent than humans, and are leaving Earth without them when its destruction is imminent (something the humans don't see coming).

I did once leave a company with a phrase just like that :P A few people there actually got the reference and congratulated me for the burn.

DalasNoin
2 replies
8h29m

In that metaphor, is openai the humans or are actual humans the humans? So is openai about to be destroyed or humanity?

upmind
0 replies
6h42m

Openai would be the humans here and Ilya would be the dolphin. (In the metaphor, the dolphins leave and here Ilya is leaving)

sa-code
0 replies
7h9m

The dolphins are actually openai

upmind
1 replies
6h39m

That is really smart, I wonder what's going on behind the scenes. Q* perhaps?

bamboozled
0 replies
1h4m

I think the parent is implying the opposite here.

bkishan
0 replies
8h25m

I spotted the reference, but did not think this deep lol. You have a point here.

VoVAllen
5 replies
19h8m

Why now?

EasyMark
1 replies
14h4m

cash out and live the good life? Start his own AI company or support company? Build the next better AI? The sky is the limit

surfingdino
0 replies
13h43m

There may be a small print limiting his options.

grugagag
0 replies
18h59m

Why not? We don’t know details that could involve financial agreements

cowsaymoo
0 replies
11h51m

One idea could be the product launch dev day, which is something that originally was a point of tension (overcommercialization vs research). Launching GPT-4o at a dev day basically asserts Sam is picking up no compromise on where they were 6mo ago. Good time to finally leave if protesting that is what he believes in.

Tenoke
0 replies
12h53m

Given that he went radio silent since the voting out Altman fiasco exactly 6 months ago, it's clearly due to that.

nomad-nigiri
3 replies
15h17m

My bet is he joins Ive

EasyMark
1 replies
14h6m

pretty sure he's going to Microsoft

zandrew
0 replies
13h18m

The "personally meaningful to me" spells to me that it's probably a personal project?

zaps
0 replies
15h0m

Maybe together they can use AI to make a less shitty Christmas tree

jessenaser
3 replies
16h17m

At least now we know GPT-5 has finished development and is now in training from this (I would hope that Iyla got to add all that he hoped to before leaving).

Ilya, thanks for all you have contributed within OpenAI!

unraveller
1 replies
11h48m

GPT-5ANDBAG more like it

He wouldn't have left if he could advance hoomanity further there, the guy has like a 800ms delay for each word and that does not make for a very good liar, perhaps a dutiful one.

HarHarVeryFunny
0 replies
5h57m

The word delay depends on who he is talking to. On his Dwarkesh interview from a year or so back he speed up noticeably, presumably because Dwarkesh is a fast thinker/talker.

edmara
0 replies
6h56m

From reporting GPT-5 finished pre-training while ago and was in the process of red-teaming.

treprinum
2 replies
11h54m

The usual fate of idealistic people who build something great only to be discarded by management in a power struggle. How often did this repeat?

wruza
0 replies
10h17m

What do you mean how often, that is a foundation for the most successful economic model in humans. Some may not be discarded, but they will never get enough credit compared to a clueless head with a $1M smile talking to clueless heads with $1B wallets. We should thank god/nature that people who understand and do things exist in our species at all.

sturza
0 replies
7h54m

the people you need for the revolution are not the same you need after the revolution.

reducesuffering
2 replies
18h45m

'Back in May 2023, before Ilya Sutskever started to speak at the event, I sat next to him and told him, “Ilya, I listened to all of your podcast interviews. And unlike Sam Altman, who spread the AI panic all over the place, you sound much more calm, rational, and nuanced. I think you do a really good service to your work, to what you develop, to OpenAI.” He blushed a bit, and said, “Oh, thank you. I appreciate the compliment.”

An hour and a half later, when we finished this talk, I looked at my friend and told her, “I’m taking back every single word that I said to Ilya.”

He freaked the hell out of people there. And we’re talking about AI professionals who work in the biggest AI labs in the Bay area. They were leaving the room, saying, “Holy shit.”

The snapshots above cannot capture the lengthy discussion. The point is that Ilya Sutskever took what you see in the media, the “AGI utopia vs. potential apocalypse” ideology, to the next level. It was traumatizing.'[0]

[0] What Ilya Sutskever Really Wants https://www.aipanic.news/p/what-ilya-sutskever-really-wants

indigodaddy
1 replies
18h27m

I read the linked article and have no clue what the author is even trying to say..

goodluckchuck
0 replies
15h53m

It reads like a Jackson Pollock.

m3kw9
2 replies
18h46m

Plot twist he starts his own AI company

mezeek
1 replies
18h46m

and calls it Actually-Open AI

ffhhj
0 replies
18h42m

DisclosedAI

art3m
2 replies
17h52m

Will join Yandex

umeshunni
1 replies
17h51m

Source?

art3m
0 replies
11h53m

It's my assumption

EcommerceFlow
2 replies
17h3m

Biggest free agent since Lebron James

paxys
1 replies
16h4m

Guy should announce the next step of his career in a one hour TV special. It will easily have as many watchers as OpenAI's keynote.

surfingdino
0 replies
13h30m

He will. On Lex Friedman's YouTube channel.

nothrowaways
1 replies
16h17m

Ilya should have learned from pregozhin

rastapasta42
0 replies
16h15m

And the boeing whistleblowers...Hopefully he avoids airplanes.

imgabe
1 replies
17h1m

Never go against the family, Fredo

chem83
0 replies
16h59m

Matches the hairline.

darkerside
1 replies
15h12m

Making sure generalized AI benefits everybody is the new Don't Be Evil

unraveller
0 replies
10h43m

"We want to put AI in your hands"

to keep??

NO! whatever gave you that idea, evil doer...

Open AI, as in, open your hands and beg for another hit of AI through thick rubber gloves and plexiglass.

TyrianPurple
1 replies
12h42m

Meta's next for him? There's lots of money being poured into their AI division and there's lots of compute & being able to do any kind of research he might want.

KaiserPro
0 replies
7h58m

I doubt it, the internal politics of it are enough to drive most people crazy.

I_am_tiberius
1 replies
7h45m

I hope Ilya takes care of himself. I can imagine that what happened during the past year is not helpful for one's mental health. I assume the presented relationship with Sam Altman does not reflect reality and the external press surely also causes a lot of pressure.

dbancajas
0 replies
5h11m

didn't know this. can you explain or link a few articles?

zx8080
0 replies
15h52m

Again?

yumraj
0 replies
13h40m

His phone must be ringing non-stop from all the VCs.

surume
0 replies
10h10m

The future of the company doesn't depend on one engineer. If he left, it's likely because he had a vision that wasn't in line with Sam or Microsoft. Others will take his place and OpenAI will likely reach Elon Musks' recent prediction that AI will improve 100x in the next few years.

sidcool
0 replies
13h12m

If Karpathy and Illya join xAI, that would be a fun trajectory.

shmatt
0 replies
18h7m

Funny enough people will still call OpenAI “an engineering led company” when very obviously it’s slowly being taken over by the same MBAs as Google

nabla9
0 replies
15h32m

If he goes to Microsoft next it was all prearranged a year ago.

jedberg
0 replies
18h6m

So the CEO of Amazon Web Services and the Chief Scientist of OpenAI are on the market on the same day...

I'm not saying it's a conspiracy, but it's an awful big coincidence, especially since today is Tuesday and usually these things happen on a Friday.

gnicholas
0 replies
2h50m

I wonder how the proposed regulations to make noncompetes unenforceable affect moves like this. Or was he sufficiently high up that his existing noncompete would have survived?

chefkd
0 replies
18h31m

Would be cool to write something like The Game where the rich and powerful have factions and alliances they gamble with and Ilya's team lost so he has to leave at the height of OpenAI like one of us goes through it we all go through it type thing

aaroninsf
0 replies
1h8m

Why do people treat these technologists doing career moves, as if this was lineup changes in a major league sports teams?

Are these "first name" (ugh) "influencers" smart? Sure.

Smart is not that rare. These people are technologists like most of you, they aren't notably smarter, they just got lucky in their career direction and specialization. They aren't business geniuses.

They're just people filling roles.

Do changes in leadership affect a business? Sure? I guess? About 5% as much as you'd think from the tea-spilling gossip-rag chatter around AI people.

Enough already. Attend to the technology. Attend to the actual work. The number of you who are professionally impacted by these people changing paychecks is closer to zero than 50%.

EMCymatics
0 replies
13h12m

Yikes

Dowwie
0 replies
7h19m

Does it matter that the people who dedicated the last decade to developing breakthrough work have left? It is a mistake to think that their luck streak will continue and their departure isn't a sign of decay at OpenAI. They may as well cash-in on their notoriety while it is of value. The odds are more in favor of other teams blazing new trails.

23B1
0 replies
18h34m

And just like that, a drawbridge across OAI's moat.