Implementing a ChatGPT-like LLM from scratch, step by step

I honestly cannot fathom why anyone working in the AI space would find $50 too much to spend to gain a deeper insight into the subject. Creating educational materials requires an insane amount of work, and I can promise, no matter how successful this book is, if rasbt were do the math on income generated over hours spent creating it wouldn't make sense as an hourly rate.

Plenty of other people have this understanding of these topics, and you know what they chose to do with that knowledge? Keep it to themselves and go work at OpenAI to make far more money keeping that knowledge private.

If you want to live in a world where this knowledge is open, at the very least refrain from publicly complaining about a book that cost roughly the same as a decent dinner.

Neural Networks: Zero to Hero[1] by Andrej Karpathy

[1] https://karpathy.ai/zero-to-hero.html

https://jaykmody.com/blog/gpt-from-scratch/ for a gpt2 inference engine in numpy

then

https://www.dipkumar.dev/becoming-the-unbeatable/posts/gpt-k... for adding a kv cache implementation

I added notes to the Jupyter notebooks, I hope they are also readable as standalone from the repo.

I'd go with https://course.fast.ai/

It's much more accessible to regular developers, and doesn't make assumptions about any kind of mathematics background. It's a good starting poing after which other similar resources start to make more sense.

Automatic differentiation is why we are able to have complex models like transformers, it's arguably the key reason (in addition to large amounts of data and massive compute resources) that we have the revolution in AI that we have.

Nobody working in this space is hand calculating derivatives for these models. Thinking in terms of differentiable programming is a given and I think certainly counts as "from scratch" in this case.

Any time I see someone post a comment like this, I suspect the don't really understand what's happening under the hood or how contemporary machine learning works.

at least it wasn't

   from transformers import

Lol ok, otherwise it would probably be not very readable due to the verbosity. The book shows how to implement LayerNorm, Softmax, Linear layers, GeLU etc without using the pre-packaged torch versions though.

They should probably

    import universe 

I don't think implementing autograd is relevant or in-scope for learning about how transformers work (or writing out the gradient for transformer by hand, I can't even imagine doing that).

To code from scratch you should first fab your own semiconductors

I'd say my primary motivation is an educational goal, i.e., helping people understand how LLMs work by building one. LLMs are an important topic, and there are lots of hand-wavy videos and articles out there -- I think if one codes an LLM from the ground up, it will clarify lots of concepts.

Now, the secondary goal is, of course, also to help people with building their own LLMs if they need to. The book will code the whole pipeline, including pretraining and finetuning, but I will also show how to load pretrained weights because I don't think it's feasible to pretrain an LLM from a financial perspective. We are coding everything from scratch in this book using GPT-2-like LLM (so that we can load the weights for models ranging from 124M that run on a laptop to the 1558M that runs on a small GPU). In practice, you probably want to use a framework like HF transformers or axolotl, but I hope this from-scratch approach will demystify the process so that these frameworks are less of a black box.

You can't understand it unless you already know most of the stuff.

I've watched it many times to understand well most of it.

And obviously you must already know pytorch really well, including the matrix multiplication, backpropagation etc. He speaks very fast too...

Haven't fully watched this but from a brief skimming, here are some differences that the book has:

- it implements a real word-level LLM instead of a character-level LLM

- after pretraining also shows how to load pretrained weights

- instruction-finetune that LLM after pretraining

- code the alignment process for the instruction-finetuned LLM

- also show how to finetune the LLM for classification tasks

- the book it overall has a lots of figures. For Chapter 3, there are 26 figures alone :)

The video looks awesome though. I think it's probably a great complementary resource to get a good solid intro because it's just 2 hours. I think reading the book will probably be more like 10 times that time investment.

Try OpenAI's spinning up: https://spinningup.openai.com/en/latest/

This book seems to focus on large language models, for which RLHF is sometimes a useful addition.

To learn more about RL, most people would advise the Sutton and Barto book, available at: http://incompleteideas.net/book/the-book-2nd.html

That's exactly what the Q-learning lab in this course does:

https://www.ida.liu.se/~TDDC17/info/labs/rl.en.shtml

Sorry, in that case I would rather recommend a dedicated RL book. The RL part in LLMs will be very specific to LLMs, and I will only cover what's absolutely relevant in terms of background info. I do have a longish intro chapter on RL in my other general ML/DL book (https://github.com/rasbt/machine-learning-book/tree/main/ch1...) but like others said, I would recommend a dedicated RL book in your case.

Reinforcement learning is an entirely separate area of research from LLMs and, while often seen as part of ML (Tom Mitchell's classic Machine Learning has a great section on Q learning, even if it feels a bit dated in other areas) it has little to do with contemporary ML work. Even with things like AlphaGo, what you find is basically work in using deep neural networks as an input into classic RL techniques.

Sutton and Barto's Reinforcement Learning: An Introduction is widely considered a the definitive intro to the topic.

I enjoyed "Grokking Deep Reinforcement Learning"[0]. It doesn't include anything about transformers though. Also, see Python's gymnasium[1] library for a lunar lander environment, it's the one I focused on most while I was learning and I've solved it a few different ways now. You can also look at my own notebook I used when implementing Soft Actor Critic with PyTorch not too long ago[2], it's not great for teaching, but maybe you can get something out of it.

[0]: https://www.manning.com/books/grokking-deep-reinforcement-le... [1]: https://gymnasium.farama.org/environments/box2d/ [2]: https://github.com/DevJac/learn-pytorch/blob/main/SAC.ipynb

The ETA for the last chapter is August if things continue to go well. It's usually available in the MEAP a few weeks after that, some time in September. And print version should be available early 2025 I think.

It's actually less risky. The author may be able to reap the benefits of writing a book without actually finishing it. Ideally, maybe not much more than Chapter 1.

It kind of is, but it's also kind of motivating :)

Thanks for the support! There's the official Manning Forum for the book, but you are also welcome to use the Discussions page on the GitHub page.

The book's forum on manning

It's in progress still. I have most of the code working, but it's not organized into the chapter structure, yet. I am planning to add a new chapter every ~month (I wish I could do this faster, but I also have some other commitments). Chapter 4 will be either uploaded by the end of this weekend or by the end of next weekend.

Depending on your level, it could take a lot of weeks to go through the already available material (code and pdf), so I'd suggest to purchase it anyway... It makes no sense to wait until the end, if you're interested in the subject.

Thanks, comparing positional encodings, MoEs, kv-caches etc are all good topics that I have in mind for either supplementary material and/or a follow-up book. The reason why it probably won't land in this current book is the length and time line. It's already going to be a big book as it is (400-500 pages). And I also want to be a bit mindful of the planned release date. However, these are indeed good suggestions.

Thanks! And please don't hesitate to reach out via the Forum or the GitHub Discussions if you have any feedback or questions.

Glad to hear and thanks for the support. Chapter 3 should be in the MEAP soonish (submitted the draft last week). Will also upload my code for chapter 4 to GitHub soonish, in the next couple of days, just have to type up the notes.

That was pretty smooth. They reached out whether I was interested in writing a book for them (probably because of my other writings online), I mentioned what kind I book I want to write, submitted a proposal, and they liked that idea :)

Thanks :)

The attention mechanism we implement in this book* is specific to LLMs in terms of the text inputs, but it's fundamentally the same attention mechanism that is used in vision transformers. The only difference is that in LLMs, you turn text into tokens, and convert these tokens into vector embeddings that go into an LLM. In vision transformers, instead of regarding images as tokens, you use an image patch as a token and turn those into vector embeddings (a bit hard to explain without visuals here). In both text or vision context, it's the same attention mechanism, and it both cases it receives vector embeddings.

(*Chapter 3, already submitted last week and should be online in the MEAP soon, in the meantime the code along with the notes is also available here: https://github.com/rasbt/LLMs-from-scratch/blob/main/ch03/01...)