Interesting, both Karpathy and Sutskever are gone from OpenAI now. Looks like it is now the Sam Altman and Greg Brockman show.
I have to admit, of the four, Karpathy and Sutskever were the two I was most impressed with. I hope he goes on to do something great.
Top 6 science guys are long gone. Open AI is run by marketing, business, software and productization people.
When the next wave of new deep learning innovations sweeps the world, Microsoft eats whats left of them. They make lots of money, but don't have future unless they replace what they lost.
I don’t feel that OpenAI has a huge moat against say Anthropic. And I don’t know OpenAI needs Microsoft nearly as much as Microsoft needs OpenAI
But is it even clear what is the next big leap after LLM? I have the feeling many tend to extrapolate the progress of AI from the last 2 years to the next 30 years but research doesn't always work like that (though improvements in computing power did).
Not saying it’s going to be the same, but I’m sure computing progress looked pretty unimpressive from, say, 1975 to 1990 for the uninitiated.
By the 90s they were still mainly used as fancy typewriters by “normal” people (my parents, school, etc) although the ridiculous potential was clear from day one.
It just took a looong time to go from pong to ping and then to living online. I’m still convinced even this stage is temporary and only a milestone on the way to bigger and better things. Computing and computational thought still has to percolate into all corners of society.
Again not saying “LLM’s” are the same, but AI in general will probably walk a similar path. It just takes a long time, think decades, not years.
Edit: wanted to mention The Mother of All Demos by Engelbart (1968), which to me looks like it captures all essential aspects of what distributed online computing can do. In a “low resolution”, of course.
Computing progress from 78 to 90 was mind-blowing.
1978: the apple ][. 1mhz 8 bit microprocessor, 4kb of ram, monochrome all-,caps display.
1990:Mac IIci, 25mhz 32-bit CPU, 4MB ram, 640x480 color graphics and an easy to use GUI.
Ask any of us who used both of these at the time: it was really amazing.
They were amazing, and the progress was incredible, but both of those computers - while equally exciting and delightful to people who saw the potential - were met with ‘but what can I actually use it for?’ from the vast majority of the population.
By 1990 home computer use was still a niche interest. They were still toys, mainly. DTP, word processing and spreadsheets were a thing, but most people had little use for them - I had access to a Mac IIci with an ImageWriter dot matrix around that time and I remember nervously asking a teacher whether I would be allowed to submit a printed typed essay for a homework project - the idea that you could do all schoolwork on a computer was crazy talk. By then, tools like Mathematica existed but as a curiosity not an essential tool like modern maths workbooks are.
The internet is what changed everything.
Broadband. Dial-up was still too much of an annoyance, too expensive.
Once broadband was ubiquitous in the US and Europe, that's when the real explosion of computer usage happened.
A big obstacle was that everything was on paper. We still had to do massive amounts of data entry.
For some strange reason html forms is an incredibly impotent technology. Pretty standard things are missing like radioboxes with an other text input. 5000+ years ago the form labels aligned perfectly with the value.
I can picture it already, ancient Mesopotamia, the clay tablet needs name and address fields for the user to put their name and address behind. They pull out a stamp or a roller.
Of course if you have a computer you can have stamps with localized name and address formatting complete with validation as a basic building block of the form. Then you have a single clay file with all the information neatly wrapped together. You know, a bit like that e-card no one uses only without half data mysteriously hidden from the record by some ignorant clerk saboteur.
We've also failed to hook up devices to computers. We went from the beautiful serial port to IoT hell with subscriptions for everything. One could go on all day like that, payments, arithmetic, identification, etc much work still remains. I'm unsure what kind of revolution would follow.
Talking thinking machines will no doubt change everything. That people believe it is possible is probably the biggest driver. You get more people involved, more implementations, more experiments, more papers, improved hardware, more investments.
I agree. Likewise, early AI models to GPT4 is breathtaking progress.
Regular people shrug and say, yeah sure, but what can I do with it. They still do this day.
Honestly mobile totally outstrips this.
One day at work about 10-15 years ago I looked at my daily schedule and found that on that day my team were responsible for delivering a 128kb build of Tetris and a 4GB build of Real Racing.
mobile internet and smartphones were the real gamechanger here, which were definitely not linear.
They became viable in the 2000's, let's say 2007 with the iPhone, and by late 2010's everyone was living online, so "decades" is a stretch.
To make the 2000s possible, decades of relatively uninteresting progress was made. It quickly takes off from there.
It was only 11 years from pong to ping.
Extrapolating 2 years might give you a wrong idea, but extrapolating the last year suggests making another leap that was GPT3 or GPT4 is much, much more difficult. The only considerable breakthrough I can think of is Google's huge context window which I hope will be the norm one day, but in terms of actual results they're not mind-blowing yet. We see little improvements everyday and for sure there will be some leaps, but I wouldn't count on a revolution.
Unlike AI in the past, there is now massive amounts of money going into AI. And the number things humans are still doing significantly better than AI is going down continously now.
If something like Q* is provided organically with GPT5 (which may have a different name), and allows proper planning, error correction and direct interaction with tools, that gaps is getting really close to 0.
AI in the past (adjusted for 1980s) was pretty well funded. It's just that fundamental scientific discovery bears little relationship to the pallets of cash.
Funding in the 1980s was sometimes very good. My company bought me an expensive Lisp Machine in 1982 and after that, even in “AI winters” it mostly seemed that money was available.
AI has a certain mystique that helps get money. In the 1980s I was on a DARPA neural network tools advisory panel, and I concurrently wrote a commercial product that included the 12 most common network architectures. That allowed me to step in when a project was failing (a bomb detector we developed for the FAA) that used a linear model, with mediocre results. It was a one day internal consult to provide software for a simple one hidden layer backprop model. During that time I was getting mediocre results using symbolic AI for NLP, but the one success provided runway internally in my company to keep going.
That funding may have felt good at the time compared to some other academic fields.
But compared to the 100s of billions (possibly trillions, globally) that is currently being plowed into AI, that's peanuts.
I think the closest recent analogy to the current spending on AI, was the nuclear arms race during the cold war.
If China is able to field ASI before the US even have full AGI, nukes may not matter much.
You are right about funding levels, even taking inflation into account. Some of the infrastructure, like Connection Machines and Butterfly Machines seemed really expensive at the time though.
They only seem expensive because they're not expected to generate a lot of value (or military/strategic benefit).
Compare that the 6+ trillions that were spent in the US alone on nuclear weapons, and then consider, what is of greater strategic importance: ASI or nukes?
A tiny fraction of the current funding. 2-4 orders of magnitude less.
Heavy funding may not automatically lead to breakthroughs such as Special Relativity or Quantum Mechanics (though it helps there too). But once the most basic ideas are in place, massive is what causes the breakthroughs like in the Manhatten Project and Apollo Program.
And it's not only the money itself. It's the attention and all the talent that is pulled in due to that.
And in this case, there is also the fear that the competition will reach AGI first, whether the competition is a company or a foreign government.
It's certainly possible the the ability to monetize the investments may lead to some kind of slowdown at some point (like if there is a recession).
But it seems to me that such a recession will have no more impact on the development of AGI than the dotcom bust had for the importance of the internet.
Operational costs were correspondingly lower, as they didn't need to pay electricity and compute bills for tens of millions concurrent users.
There is no reason to think that the ideas are in place. It could be that the local optimum is reached as it happened in many other technology advances before. The current model is mass scale data driven, the Internet has been sucked dry for data and there's not much more coming. This may well require a substantial change in approach and so far there are no indications of that.
From this pov monetization is irrelevant, as except for a few dozen researchers the rest of the crowd are expensive career tech grunts.
That depends what you mean when you say "ideas". If you consider ideas at the level of transformers, well then I would consider those ideas of the same magnitude as many of the ideas the Manhatten Project or Apollo Program had to figure out on the way.
If you mean ideas like going from expert system to Neural Networks with backprop, then that's more fundamental and I would agree.
It's certainly still conceivable that Penrose is right in that "true" AGI requires something like microtubules to be built. If so, that would be on the level of going from expert systems to NNs. I believe this is considered extremely exotic in the field, though. Even LeCun probably doesn't believe that. Btw, this is the only case where I would agree that funding is more or less irrelevant.
If we require 1-2 more breakthroughs on par with Transformers, then those could take anything from 2-15 years to be discovered.
For now, though, those who have predicted that AI development will mostly be limited by network size and the compute to train it (like Sutskever or implicitly Kurzweil) have been the ones most accurate in the expected rate of progress. If they're right, then AGI some time between 2025-2030 seems most likely.
Those AGI's may be very large, though, and not economical to run for a wider audience until some time in the 30's.
So, to summarize: Unless something completely fundamental is needed (like microtubules), which happens to be a fringe position, AGI some time between 2025 and 2040 seems likely. The "pessimists" (or optimists, in term of extinction risk) may think it's closer to 2040, while the optimists seem to think it's arriving very soon.
The majority of the developers may know what LLMs are in an abstract sense, but I meet very few that really realize what these are. These LLMs are an exponential leap in computational capability. The next revolution is going to be when people realize what we have already, because it is extremely clear the majority do not. RAG? Chatbots? Those applications are toys compared to what LLMS can do right now, yet everyone is dicking around making lusty chatbots or naked celebrities in private.
Enlighten us
It is both subtle and obvious, yet many are missing this: if you want/need a deep subject matter expert in virtually any subject, write a narrative biography describing your expert using the same language that expert would use to describe themselves; this generates a context within the LLM carrying that subject matter expertise, and now significantly higher quality responses are generated. Duplicate this process for several instances of your LLM, creating a home brewed collection of experts, and have them collectively respond to one's prompts as a group privately, and then present their best solution. Now there is a method of generating higher reliability responses. Now turn to the fact that the LLMs are trained on an Internet corpus of data that contains the documentation and support forums for every major software application; using the building blocks described so far, it is not difficult at all to create agents that sit between the user and pretty much every popular software application and act as co-authors with the user helping them use that application.
I have integrated 6 independent, specialized "AI attorneys" into a project management system where they are collaborating with "AI web developers", "AI creative writers", "AI spreadsheet gurus", "AI negotiators", "AI financial analysts" and an "AI educational psychologist" that looks at the user, the nature and quality of their requests, and makes a determination of how much help the user really needs, modulating how much help the other agents provide.
I've got a separate implementation that is all home solar do-it-yourself, that can guide someone from nothing all the way to their own self made home solar setup.
Currently working on a new version that exposes my agent creation UI with a boatload of documentation, aimed at general consumers. If one can write well, as in write quality prose, that person can completely master using these LLMs to superior results.
Ah yes, "it's so obvious no one sees it but me". Until you show people your work, and have real experts examining the results, I'm going to remain skeptical and assume you have LLMs talking nonsense to each each other.
I'm at a law firm, this is in use with attorneys to great success. And no, none of them are so dumb they do not verify the LLM's outputs.
How can no one see what we have today? You only need six instances of an LLM running at the same time, with a system to coordinate between them, and then you have to verify the results manually anyway. Sign me up!
If a certain percent of the work is completed through research synthesis and multiple perspective alignment, why is said novel approach not worth lauding?
I've created a version of one of the resume GPTs that analyses my resume's fit to a position when fed the job description along with a lookup of said company. I then have a streamlined manner in which it points out what needs to be further highlighted or omitted in my resume. It then helps me craft a cover letter based on a template I put together. Should I stop using it just because I can't feed it 50 job roles and have it automatically select which ones to apply to and then create all necessary changes to documents and then apply?
The point is these characters are not doing the work for people, it co-authors the work with them. It's just like working with someone highly educated but with next to no experience - they're a great help, but ya gotta look at their work to verify they are on track. This is the same, but with a collection of inexperienced phds. The LLMs really are idiot savants, and when you treat them like that they respond with expectations better.
I agree with OP, I think we still have no idea yet what dreams may come of the LLM's we have today. So no one will be able to "enlighten us" — perhaps not until we're looking in the rear-view mirror.
I would say instead, stay tuned.
I don’t think we’re even close to exhausting the potential of transformer architectures. gpt4o shows that a huge amount can be gained by implementing work done on understanding other media modalities. There’s a lot of audio that they can continue to train on still and the voice interactions they collect will go into further fine tuning. Even after that plays out there will be video to integrate next and thanks to physics simulations and 3D rendering there is a potentially endless and readily generated license free supply of it, at least for the simpler examples. For more complex real world video they could just set up web cams in public areas around the world where consent isn’t required by law and collect masses of data every second. Given that audio seems to have enabled emotional understanding and possibly even humour, I can’t imagine what all might fall out of video. At the least it’s going to improve reasoning since it will involve predicting cause and effect. There are probably a lot of others you could add though we don’t have large datasets for them.
In just a couple of generations each training cycle will cost close to $10 billion. That's a lot of cheddar that you have to show ROI on.
IMO their next big leap will be to get it cheap enough and integrated with enough real time sources to become the default search engine.
You can really flip the entire ad supported industry upside down if you integrate with a bunch of publishers and offer them a deal where they are paid every time an article from their website is returned. If they make this good enough people will pay $15-20 a month for no ads in a search engine.
I don't think it particularly matters right now (practically speaking). It's going to take years for businesses and product companies to commoditize applications of LLMs, so while it's valuable for the Ilyas & Andrejs of the world to continue the good work of hard research, it's the startups, hyperscalers and SaaS companies who are creating business applications for LLMs that going to be the near term focus.
LLM is all you need
Attention and scale is all you need
Anything else you do will be overtaken by LLM when it builds its internal structures
Well, LLM and MCTS
The rest is old news. Like Cyc
I Don't know. Being able to get azure credits has payed out really well for openai as a business in constant need of computer.
Which is a very short term advantage. And Anthropic gets aws credits which would you rather have?
Given Amazon's no show in the AI space? Azure. By a mile.
Except if you're Anthropic or OpenAI you don't care about what your compute provider has done in the AI space - you care about the compute power they can give you.
That's exactly what I'm talking about: https://i.imgur.com/sZ3tniY.jpeg
But how many of those are ordered specifically for OpenAI, and are on order as a result of them to begin with? Do you think if we were in a parallel universe where OpenAI ended up partnering with Google or Amazon instead, the GPU shipments would look the same? I think they would reflect wherever OpenAI ended up doing all their compute showing a pretty similar lion's share.
Your claim was that people should care about compute based on what the provider has done in the AI space, but Microsoft was pretty far behind on that side until OpenAI - Google was really the only player in town. Should they have wanted GCP credits instead? Do you care about their AI results or the ex post facto GPU shipments?
Or, if what you actually want to argue is that Anthropic would be able to get more GPUs with Azure than AWS or GCP then this is a different argument which is going to require different evidence than raw GPU shipments.
The claim being implied was that Anthropic was in a better position because they had partnered with AWS versus Azure and thus they would have more access to GPU.
That isn't the case, at all. All I'm stating is what the chart clearly shows - Azure has invested deeply in this technology and at a rate that far exceeds AWS.
Never discount the value of short term advantages.
Being first at the start (i.e. first mover advantage) is huge.
There are no moats in deep learning, everything changes so fast.
They have the next iteration of GPT Sutskever helped to finalize. OpenAI lost it's future unless they find new same caliber people.
How do you know that they have the next GPT?
How do you know what Sutskever contributed? (There was talk that the most valuable contributions came from the less well known researchers not from him)
sha256:e33135417f7f5b8f4a1c98c28cf26330bea4cc6b120765f59f5d518ea0ce80e5
What should this mean?
Isn't access to massive datasets and computation the moat? If you and your very talented friends wanted to build something like GPT-4, could you?
It's going to get orders of magnitude less expensive, but for now, the capital requirements feel like a pretty deep moat.
How do you know massive datasets are required? Just because that’s how current LLMs operate, doesn’t mean it’s necessarily the only solution.
Then the resources needed to discover an alternative to brute-forcing a large model are a huge barrier.
I think academia and startups are currently better suited to optimize tinyml and edge ai hardware/compilers/frameworks etc.
OpenAI most definitely needs the compute from MSFT. It could certainly swap out to another service but given that microsoft invested via credits it would be problematic. They have enmeshed their future.
They seem to have a huge "money moat" now. Partnerships with Apple and MS mean they have a LOT of money to try a lot of things I guess.
Before the Apple partnership, maybe it seemed like the moat was shrinking, but I'm tno sure now.
Likely they have access to a LOT of data now too.
If we look at history of innovation and invention it’s very typical the original discovery and final productization are done by different people. For many reasons, but a lot of them are universal I would say.
E.g. Oppenheimer’s team created the bomb, then following experts finetuned the subsequent weapon systems and payload designs. Etc.
Except OpenAI hasn’t yet finished discovery on its true goal: AGI. I wonder if they risk plateauing at a local maximum.
I'm genuinely curious: what do you expect an "AGI" system to be able to do that we can't do with today's technology?
Some first ideas coming to mind:
Engineering Level:
Science: Misc:The problem with CO2 levels is that no one likes the solution not that we don't have one. I highly doubt adding AGI to the mix is going to magically make things better. If anything we'll just burn more CO2 providing all the compute resources it needs.
People want their suburban lifestyle with their red meat and their pick-up truck or SUV. They drive fuel inefficient vehicles long-distances to urban work environments and they seem to have very limited interest in changing that. People who like detached homes aren't suddenly affording the rare instances of that closer to their work. We burn lots of oil because we drive fuel inefficient vehicles long distances. This is a problem of changing human preferences which you just aren't going to solve with an AGI.
Assuming embedded AI in every piece of robotics - sometimes directly, sometimes connected to a central server (this is doable even today) - it'll revolutionize industries: human-less mining, processing, manufacturing, services, and transportation. These factories would eventually produce and install enough solar power or build sufficient nuclear plants and energy infrastructure, making energy clean and free.
With abundant electric cars (at this future point in time) and clean electricity powering heating, transportation, and manufacturing, some AIs could be repurposed for CO2 capture.
It sounds deceptively easy, but from an engineering standpoint, it likely holds up. With free energy and AGI handling labor and thinking, we can achieve what a civilization could do and more (cause no individual incentives come into play).
However, human factors could be a problem: protests (luddites), wireheading, misuse of AI, and AI-induced catastrophes (alignment).
Having more energy is intrinsically dangerous, though, because it's indiscriminate: more energy cannot enable bigger solutions without also enabling bigger problems. Energy is the limiting factor to how much damage we can do. If we have way more of it, all bets are off. For instance, the current issue may be that we are indirectly cooking the planet through CO2 emissions, so capturing that sounds like a good idea. But even with clean energy, there is a point where we would cook the planet directly via waste heat of AI and gizmos and factories and whatever unforeseen crap we'll conjure just because we can. And given our track record I'm far from confident that we wouldn't do precisely that.
This exactly. Every self replicating organism will eventually use all the energy available to it, there will never be an abundance. From the dawn of time, mankind has similarly used every bit of energy it generates. From the perspective of a subsistence farmer in the 1600s, if you told them how much energy would be available in 400s year they would think we surely must live in paradise with no labor. Here we are, still metaphorically tilling the land.
Do you believe the average human has general intelligence, and do you believe the average human can intellectually achieve these things in ways existing technology cannot?
Yes, considering that AI operates differently from human minds, there are several advantages:
The first AGI will probably take way too much compute to have a significant effect, unless there is a revolution in architecture that gets us fast and cheap AGI at once the AGI revolution will be very slow and gradual.
A model that is as good as an average human but costs $10 000 per effective manhour to run is not very useful, but it is still an AGI.
The incentives aren't structured properly for these things to happen, it has always been a sci-fi fairy tale that AGI would achieve these things.
now do magical flying unicorn ponies, which I understand there is also considerable demand for
An AGI could replace human experts at tasks that doesn't require physical embodiment, like diagnosing patients, drafting contracts, doing your taxes etc. If you still do those manually and not just offload all of it to ChatGPT then you would greatly benefit from a real AGI that could do those tasks on their own.
And no, using ChatGPT like you use a search engine isn't ChatGPT solving your problem, that is you solving your problem. ChatGPT solving your problem would mean it drives you, not you driving it like it works today. When I hired people to help me do taxes they told me what papers they needed and then they did my taxes correctly without me having to look it through and correct them, an AGI would work like that for most tasks, it means you no longer need to think or learn to solve problems since the AGI solves them for you.
How come the goal posts for AGI are always the best of what people can do?
I can't diagnose anyone, yet I have GI.
Reminds me of:
Not the best, I just want it to be able to do what average professionals can do because average humans can become average professionals in most fields.
You can learn to, an AGI system should be able to learn to as well. And since we can copy AGI learning it means that if it hasn't learned to diagnose people yet then it probably isn't an AGI, because an AGI should be able to learn that without humans changing its code and once it learned it once we copy it forever and now the entire AGI knows how to do it.
So, the AGI should be able to do all the things you could do if we include all versions of you that learned different fields. If the AGI can't do that then you are more intelligent than it in those areas, even if the singular you isn't better at those things than it is.
For these reasons it makes more sense to compare an AGI to humanity rather than individual humans, because for an AGI there is no such thing as "individuals", at least not the way we make AI today.
People with severe Alzheimer's cannot learn, but still have general intelligence.
If they can't learn then they don't have general intelligence, without learning there are many problems you wont be able to solve that average (or even very dumb) people can solve.
Learning is a core part to general intelligence, as general intelligence implies you can learn about new problems so you can solve those. Take away that and you are no longer a general problem solver.
Let's take a step back from LLMs. Could you accept the network of all interconnected computers as a generally intelligent system? The key part here that drives me to ask this is:
I had a very bad Reddit addiction in the past. It took me years of consciously trying to quit in order to break the habit. I think I could make a reasonable argument that Reddit was using me to solve its problems, rather than myself using it to solve mine. I think this is also true of a lot of systems - Facebook, TikTok, YouTube, etc.
It's hard to pin down all computers as an "agent" in the way we like to think about that word and assign some degree of intelligence to, but I think it is at least an interesting exercise to try.
Companies are general intelligences and they use people, yes. But that depends on humans interpreting that data reddit users generates and updating their models, code and algorithms to adapt to that data, the computer systems alone aren't general intelligences if you remove the humans.
An AGI could run such a company without humans anywhere in the loop, just like humans can run such a company without an AGI helping them.
I'd say a strong signal that AGI has happened are large fully automated companies without a single human decisionmaker in the company, no CEO etc. Until that has happened I'd say AGI isn't here, if that happens it could be AGI but I can also imagine a good enough script to do it for some simple thing.
The simplest answer, without adding any extraordinary capabilities to the AGI that veer into magical intelligence, is to have AI assistants that can seemlessly interact with technology the way a human assistant would.
So, if you want to meet with someone, instead of opening you calendar app and looking for an opening, you'd ask your AGI assistant to talk to their AGI assistant and set up a 1h meeting soon. Or, instead of going on Google to find plane tickets, you'd ask you AGI assistant to find the most reasonable tickets for a certain date range.
This would not require any special intelligence more advanced than a human's, but it does require a very general understanding of the human world that is miles beyond what LLMs can achieve today.
Going only slightly further with assumptions about how smart an AGI would be, it could revolutionize education, at any level, by acting as a true personalized tutor for a single student, or even for a small group of students. The single biggest problem in education is that it's impossible to scale the highest quality education - and an AGI with capabilities similar to a college professor would entirely solve that.
The examples you're providing seem to have been thoroughly solved already.
I'm at the European AI Conference for our startup tomorrow, and they use a platform that just booked me 3 meetings automatically with other people there based on our availability... It's not rocket science.
And you don't even need those narrow tools. You could easily ask GPT-4o (or lesser versions) something along the lines of :
Do you work in education? Because I don't think many who do would agree with this take.
Where I live, the single biggest problem in education is that we can't scale staffing without increasing property taxes, and people don't want to pay higher property taxes. And no, AGI does not fix this problem, because you need staff to be physically present in schools to deal with children.
Even if we had an AGI that could do actual presentation of coursework and grading, you need a human being in there to make sure they behave and to meet the physical needs of the students. Humans aren't software to program around.
This is definitely an interesting way to look at it. My initial reaction is to consider that I can enhance the capabilities of a system without increasing its inteligence. For example, if I give a monkey a hammer, it can do more than it could do when it didn't have the hammer, but it is not more intelligent (though it could probably learn things by interacting with the world with the hammer). That leads me to think: can we enhance the capabilities of what we call "AI systems" to do these things, without increasing their intelligence? It seems like you can glue GPT-4o to some calendar APIs to do exactly this. This seems more like an issue of tooling rather than an issue of intelligence to me.
I guess the issue here is: can a system be "generally intelligent" if it doesn't have access to general tools to act on that intelligence? I think so, but I also can see how the line is very fuzzy between an AI system and the tools it can leverage, as really they both do information processing of some sort.
Thanks for the insight.
self consciousness?
And how do you test for that?
Probably the best thing we can do at the moment is compile a list of ways in which we shouldn't test for intelligence.
A working memory that can preserve information indefinitely outside a particular context window and which can engage in multi-step reasoning that doesn't show up in its outputs.
GPT4o's context window is 128k tokens which is somewhere on the order of 128kB. Your brain's context window, all the subliminal activations from the nerves in your gut and the parts of your visual field you aren't necessarily paying attention to is on the order of 2MB. So a similar order of magnitude though GPT has a sliding window and your brain has more of an exponential decay in activations. That LLMs can accomplish everything they do just with what seems analogous to human reflex rather than human reasoning is astounding and more than a bit scary.
I'm curious what resources led you to calculate a 2MB context window, I'd like to learn more.
Looking up an estimate of the brain's input bandwidth at 10 million bits per second and multiplying by the second or two a subliminal stimuli can continue to affect a person's behavior. This is a very crude estimate and probably an order of magnitude off, but I don't think many orders of magnitude off.
Prove the Riemann Hypothesis
Can you do that, are you not generally intelligent, or is that a bad metric for general intelligence? At least one of these is true.
how do you know that is actually currently their true goal. it appears to me that the goal has shifted, and the people that cared about the original goal are leaving.
I’m pretty sure Altman still cares about full AGI, if only because of the power inherent in achieving that goal.
That's probably a good thing
You don't really need to look at history, that's basically science vs engineering in a nutshell.
Maybe history could tell us if that's an accident or a division that arose out of 'natural' occurrence, but I suppose a question for an economist or psychologist or sociologist how natural that could really be anyway or if it's biased by e.g. academics not financially motivated because it happens that there isn't money there; so they don't care about productising; leaving it for others who are so motivated.
With abombs for weapons systems design they needed people who just got huge kicks out of explosions (not kidding here). I guess it’s partially about personal internal motivations, and it might be more of a chance wether the thing you are intrinsically motivated to do falls under engineering or science (in both cases you get the feeling the greats did stuff they wanted to do regardless of the categorizations applied to their discipline - you get more capital affinity in engineering ofc).
How important are top science guys though? OpenAI has a thousand employees and almost unlimited money, and llm are better understood, I would guess continous development will beat singular genius heroes?
You could say the same about Google - and yet they missed the consequences of their own discovery and got behind instead of being leaders. So you need specific talent to pull this off even if in theory you can hire anybody.
I am just curious how it happened to Google? Like who were the product managers or others who didn't see an opportunity here exactly where the whole thing was invented and they had huge amounts of data already, whole web basically and the amount of video that no one else can ever hope to have?
I’m 100% positive lots of people at Google were chomping at the bit to productize LLMs early on.
But the reality is, LLMs are a cannibalization threat to Search. And the Search Monopoly is the core money making engine of the entire company.
Classic innovators dilemma. No fat-and-happy corporate executive would ever say yes to putting lots of resources behind something risky that might also kill the golden goose.
The only time that happens at a big established company, is when driven by some iconoclastic founder. And Google’s founders have been MIA for over a decade.
Golden goose is already being hoisted upon a spit — and your company is not even going to get even drippings of the fat. I am surprised by the short-sightedness of execs.
I don’t work there, I’ve just worked for lots of big orgs — they are all the same. Any claimed uniqueness in “Organizational structure” and “culture” are just window dressing around good ol’ human nature.
It’s not short sightedness, it’s rational self-interest. The rewards for taking risk as employee #20,768 in a large company are minimal, whereas the downside can be catastrophic for your career & personal life.
It's hard to invest millions in employees who are likely to leave to a competitor later. That's very risky, aka venture.
So the alternative is to...?
A lot of it was the unwillingness to take risk. LLMs were, and still are, hard to control, in terms of making sure they give correct and reliable answers, making sure they don't say inappropriate things that hurt your brand. When you're the stable leader you don't want to tank your reputation, which makes LLMs difficult to put out there. It's almost good for Google that OpenAI broke this ground for them and made people accepting of this imperfect technology.
I think the discovery of the power of the LLM was almost stumbled upon at OpenAI, they certainly didn't set out initially with the goal of creating them. Afaik they had one guy who was doing a project of creating an LLM with amazon review text data and only off the back of playing around with that did they realise its potential
Well for one, Ilya was poached from Google to work for OpenAI to eventually help build SOTA models.
Fast forward to today and we a discussing the implications of him leaving OpenAI on this very thread.
Evidence to support the notion that you can’t just throw mountains of cash and engineers at a problem to do something truly trailblazing.
Data volume isn't that important, that's becoming clearer now. What OpenAI did was paid for a bunch of good labelled data. I'm convinced that's basically the differentiator. It's not a academic or fundamental thing to do which is why google didn't do it, it's a pure practical product thing.
Difficult to quantify but as an example the 2017 scientific paper “Attention is all you need” changed the entire AI field dramatically. Without these landmark achievements delivered by highly skilled scientists, OpenAI wouldn’t exist or only be severely limited.
And ironically even the authors did not fully grasp at the time the paper importance. Reminds me of when Larry Page and Sergey Brin, tried to sell Google for $1 million ...
It depends on your views on LLMs
If your view is that LLMs only need minor improvements to their core technology and that the major engineering focus should be placed on productizing them, then losing a bunch of scientists might not be seen as that big of a deal.
But if your view is that they still need to overcome significant milestones to really unlock their value... then this is a pretty big loss.
I suppose there's a third view, which is: LLMs still need to overcome significant hurdles, but solutions to those hurdles are a decade or more away. So it's best to productize now, establish some positive cashflow and then re-engage with R&D when it becomes cheaper in the future and/or just wait for other people to solve the hard problems.
I would guess the dominant view of the industry right now is #1 or #3.
Talk to Microsoft
They definitely need them to find new approaches which you won’t find normally.
OpenAI has less than 800 employees
Most of every large business isn't science but getting organized, costs controlled, products made, risk managed, and so forth.
Agreed - it's good to have some far thinking innovation but really that can be acquired as needed so you really just need a few people with their pulse on innovation which there will always be more of outside a given company than within it.
Right now it's all about reducing transaction costs, small-i innovating, onboarding integrations, maintaining customer and stakeholder trust, getting content, managing stakeholders, and selling.
AI has now evolved beyond just the science and it's biggest issue is in the productization. Finding use cases for what's already available ALONG with new models will be where success lies.
ChatGPT is the number 1 brand in AI and as such needs to learn what it's selling, not how its technology works. It always sucks when mission and vision don't align with the nerds ideas, but I think it's probably the best move for both parties.
“Its biggest issue is in the productization.”
That’s not true at all. The biggest issue is that it doesn’t work. You can’t actually trust ai systems and that’s not a product issue.
if the ai is the product, and the product isnt trustable, isnt that a product issue??
It’s a core technology issue.
The AI isn’t the product, e.g. the ChatGPT interface is the main product that is layered above the core AI tech.
The issue is trustworthiness isn’t solvable by applying standard product management techniques on a predictable schedule. It requires scientific research.
I don't know about that, it seems to work just fine at creating spam and clone websites.
It works fine for some things. You just need a clearly defined task where LLM + human reviewer is on average faster (ie cheaper) than a human doing the same task themselves without that assistance.
Ilya is one of the Founders of the original nonprofit. This is also an issue. It does look like he was not the Founder or in any control of the for profit venture.
I'm not as in tune as some people here so: don't they need both? With the rate at which things are moving, how can it be otherwise?
I guess their point is you already have a lot out there to create new products, and you can still read papers you just won't be writing them.
They do need both but it seems like they have enough engineering talent to keep improving. Time will tell now that Ilya is out but I expect they have enough cultural cache to attract excellent engineers even if they aren’t as famous as Ilya and Karpathy.
They have a strong focus on making the existing models fast and cheap without sacrificing capability which is music to the ears of those looking to build with them.
To this end, OpenAI is already off track. Their “GPT marketplace” or whatever they’re calling it is just misguided flailing from a product perspective.
Ain’t there this pattern that innovations comes in waves and that the companies of the first wave most often just die but the second and third wave a build upon their artefacts and can be successful in a longer run?
I see this coming for sure for open ai and I do my part by just writing this comment on HN.
Or they were experimenting with people defining agentic A.I. slightly before it became more widely popular.
Not for long. They have no moat. Folks who did the science are now doing science for some other company, and will blow the pants off OpenAI.
I think you massively underestimate the power of viral media coverage and the role it plays in building a “brand.” You’ll never replicate the Musk/Altman/Satya soap opera again. ChatGPT will forever be in the history books as the Kleenex of LLM AI.
Pretty weak take there bud. If we just look at the Gartner Hype Cycle that marketing and business people love so much it would seem to me that we are at the peak, just before the downfall.
They are hyping hard to sell more, when they should be prepping for the coming dip, building their tech and research side more to come out the other side.
Regardless, a tech company without the inventors is doomed to fail.
I'm siding with you here. The same is happening at Google, but they definitely have momentum from past decades, so even if they go "full Boeing", there's a long way to fall.
Meanwhile, OpenAI (and the rest of the folks riding the hype train) will soon enter the trough. They're not diversified and I'm not sure that they can keep running at a loss in this post-ZIRP world.
Agree in general. While there remains issues on making/using AI, there is plenty of utility that doesn't require new science but maturation of deployment. For those who say its junk, I can only speak for myself and disagree.
Eh maybe from a company point of view.
But this race to add 'AI' into everything is producing a lot of nonsense. I'd rather go fullsteam ahead on the science and the new models, because that is what will actually get us something decent, rather than milking what we already have.
that won't happen, the next scam will be different
it was crypto until FTX collapsed then the usual suspects led by a16z leaned on OpenAI to rush whatever they had on market hence the odd naming of ChatGPT 3.5.
When the hype is finally realized to be just mass printing bullshit -- relevant bullshit, yes, which sometimes can be useful but not billions of dollars of useful -- there will be something else.
Same old, same old. The only difference is there is no new catchy tunes. Yet? https://youtu.be/I6IQ_FOCE6I https://locusmag.com/2023/12/commentary-cory-doctorow-what-k...
Crypto currencies has the potential to grow the world economy by about 1-3%, as banking fees go down. Add other uses of crypto may double or triple that, but that's really speculative.
AI, on the other hand, has a near infinite potential. It's conceivable that it will grow the global economy by 2% OR MORE per MONTH for decades or more.
AI is going to be much more impactful than the internet. Probably more than internal combustion, the steam engine and electricity combined.
The question is about the timescale. It could take 2 years before it really starts to generate profits, or it could take 10 or even more.
AI? Yes.
LLMs pretending to be AI? No.
What you call "AI" is generally named AGI. LLM's are alredy a kind of AI, not just generic enough to fully replace all humans.
We don't know if full AGI can be built using just current technology (like transformers) given enough scale, or if 1 or more fundamental breakthroughs are needed beyond just the scale.
My hypothesis has always been that AGI will arrive roughly when the compute power and model size matches the human brain. That means models of about 100 trillion params, which is not that far away now.
We absolutely do and the answer is such a resounding no it's not even funny.
Actually, we really don't. When GPT-3.5 was released, it was a massive surprise to many, exactly because they didn't believe simply scaling up transformers wouldn't end up with something like that.
Now using transformers doesn't mean they have to be assembled like LLM's. There are other ways to stich them together to solve a lot of other problems.
We may very well have the basic types of lego pieces needed to build AGI. We won't know until we try to build all the brain's capacities into a model of size of a few 100 trillion parameters.
And if we actually lack some types of pieces, they may even be available by then.
All crypto"currencies" with a transaction fee are negative sum games and as such , they are a scam. It's been nine years since the Washington Post admittedly somewhat clumsily but still drawn attention to this and people still insist it's something other than a scam. Despite heady articles about how it's going to solve world hunger, it's just a scam.
This round of AI is only capable of producing bullshit. Relevant bullshit but bullshit. This can be useful https://hachyderm.io/@inthehands/112006855076082650 but it doesn't mean it's more impactful than the Internet.
I agree, 1-3% was a best case. While I agree it's a net zero, even those who argue for it really don't claim much more than a couple of %.
I actually expected objections on the opposite direction. But then, this is not twitter/X.
The point is that something that can easily generate 20%-100% growth per year (AGI/ASI) is so much more important that the best case prediction for crypto's effect on the economy are not even noticeable.
That's why comparing the crypto bubble to AI is so meaningless. Crypto was NEVER* going to be something hugely important, while AI is potentially almost limitless.
*If crypto had anything to offer at all, it would be ways to avoid fees, taxes and the ability to trace transactions.
The thing is, if crypto at any point seriously threatens to replace traditional currencies as stores of value in the US or EU, it will be banned instantly. Simply because it would make it impossible for governments to run budget deficits, prevent tax evasion and sever other things that governments care about.
LLM is not AGI and there's no way to AGI from LLM. Put down the kool-aid.
I never claimed llm's are agi. Not all neural nets are llm's.
I agree with you on the AI point, but with crypto not all is what it seems.
Yes you may have short term growth, this is solely due to there being less regulation.
Despite what many people think regulation is a good thing, put in place to avoid the excesses that lead to lost livelihoods. It stops whales from exploiting the poor, provides tools for central banks to try avoid depressions.
Costs wise, banks acting as trust authorities actually can theoretically be cheaper too.
Well, I agree with all that. The 1-3% was meant to come off as a tiny, one-time gain, and an optimistic estimate of that. Not at all worth the hype.
Basically, crypto is more like gold rush than a tech breakthrough. And gold rushes rarely lead to much more than increased inflation.
Absolutely
Bank fees don't disappear into the ether when they're collected, so I doubt they have this much affect.
Oh, made my very first retail purchase with Bitcoin the other day. While the process was pretty slick and easy, the network charged $15.00 in fees. Long way to go until "free".
1-3% was intended as a ceiling for what cryptocurrency could bring to the economy, after adjusting for the reduction in inflation once those costs are gone.
Do you have a source for any of these numbers or is this just your speculation? I haven’t seen any estimates from well-known institutions that reference any of the numbers your are pointing to.
"Productization". You mean "enshitification".
Depends on who is controlling product.
Will they though? Last I heard OpenAI isn't profitable, and I don't know if it's safe to assume they every will be.
People keep saying that LLMs are an existential threat to search, but I'm not so sure. I did a quick search (didn't verify in any way if this is a feasible number) to find that Google on average makes about 30 cents in revenue per query. They make a good profit on that because processing the query costs them almost nothing.
But if processing a query takes multiple seconds on a high-end GPU, is that still a profitable model? How can they increase revenue per query? A subscription model can do that, but I'd argue that a paywalled service immediately means they're not a threat to traditional ad-supported search engines.
I honestly think that is the best course of actions for humanity. Even less chance to see AGI anytime soon if he leaves.
What an absurd thing to say.
John Schulman is still at OpenAI. As are many others.
AKA 'the four horsemen of enshitification'.
Jakub Pachocki is taking over as chief scientist. https://analyticsindiamag.com/meet-jakub-pachocki-openais-ne...
I don’t think people give Dario enough credit
Yeah, I think him leaving was a huge blow to OpenAI that they have maybe not yet recovered from. Clearly there is no moat to transformer-based LLM development (other than money), but in terms of pace of development (insight as to what is important) I think Anthropic have the edge, although Reka are also storming ahead at impressive pace.
Jakub Pachocki is still in OpenAI though
Karpathy is still a mountain in the area of ML/AI, one of the few people worth following closely on Twitter/X.
I love Karpathy. He's like a classical polymath, a scholar and a teacher.
Greg Brockman is a very good engineer. And that's maybe even more important in the current situation.