This Stanford course seems really advanced and intensive
This Stanford course seems really advanced and intensive
I agree. Someone mentioned there weren't prereqs, I think, but man I was totally lost in the first class.
If you're interested in Deep Learning or any area of ML it's fairly safe to assume you have a background in linear algebra, probability, calculus and obviously some basic programming.
If you're not interested in learning these areas, it's also safe to say you aren't really interested in deep learning either. Which is not to say if you don't already know these areas you aren't interested in deep learning, but if you don't know them and are interested in deep learning you're likely already studying them.
I say this because deep learning and the vast majority of ML really just boil down to an application of these basic tools. Deep learning/ML without the linear algebra, probability theory, calculus and coding isn't really anything at all.
background in linear algebra, probability, calculus
Curious. What does 'background' mean in this sentence. You can spend years studying just one of these in depth. How much is "enough" for ML?
The basics. 1 semester course for each.
I'd say slightly more. Maybe it's just because I attended a state school, but I think my first semester calculus class was all single variable (20 years ago now, so my memory is rusty). You really to understand gradients and jacobians for ML, which I think was calc III for me. But you can skip curl and div part I guess.
thanks! I wonder if someone has compiled a resource with just enough math for ML.
You will have to know the basics of calculus, linear algebra, and probability. A few months of study.
Are you talking about the submission course or Stanford?
Because There is a list of pre-reqs for the submitted course [1] and to be honest I feel like they are the standard requirements for you to fully understand DL may except signal processing stuff that might be taken as optional.
Oh yes, there are, anyone who says there are not is fooling others, or assuming you're a CS grad.
Machine Learning is broader than Deep Learning.
The Stanford course doesn't go as deep as, say, transformers.
Any courses you’re aware of that do?
Yes the posted one contains them (fleuret.org).
Also recommended: https://karpathy.ai/zero-to-hero.html
Oh good. Sorry I thought the implication was that this one fell short (as some of the other comments seem to suggest).
Thanks!
It's actually one of the more approachable ones you'll find.
Sorry but them there the facts. This stuff is hard. Otherwise it probably would have been done in the 1950s
Are there any good, in-depth courses that don't require watching videos?
Thank you so much!
I think this one is a bit out of date....
It looks like it covers the basics pretty well. Any pointers to alternatives?
The "Understanding Deep Learning" book covers more recent models as well: https://udlbook.github.io/udlbook/ (free PDF and Jupyter notebooks available)
IMVHO, GANs are entirely optional.
For others, https://web.stanford.edu/~jurafsky/slp3/ will take you a decent way to understanding transformer architecture.
An earlier edition of this book was the textbook for a computational linguistics course I took back in ~2002 (!), it's amazing how much has been added
For specifically understanding transformers, this (w/ maybe GPT-4 by your side to unpack jargon/math) might be able to get you from lay-person to understanding enough to be dangerous pretty quickly: https://sebastianraschka.com/blog/2023/llm-reading-list.html
This is my favourite text! It really drove home a lot points on many architectures for me. Their math appendix is simply amazing as well!
The handouts and slides are there. They are fully self-contained, not mere bullet points that the lecturer then talks about.
That's fabulous! Thanks for pointing that out
The deep learning book is a great choice, as many have mentioned.
I've been making a course that has a little less theory, and a little more application here - https://github.com/VikParuchuri/zero_to_gpt . Videos are all optional (cover the same content as the text).
See also: Practical Deep Learning for Coders https://course.fast.ai/
When Jeremy Howard wasn't named on the top 100 AI list, it blew my mind. This course is glorious.
Instead Anthropic co-founder siblings the Amodeis made it. Couldnt help but notice that Marc Benioff who owns Time also is the Major investor(think Khosla for OpenAI) into Anthropic. So i would take "times 100" with a side of pickle.
These Deep Learning (and ML) courses are turning into the equivalent of productivity tools. Many are very high quality but won't turn you into a ML/DL expert. The core issue is you need to be spend the time to complete, learn and apply in your own settings. That internal drive is something beyond what these can do and that's the crux of the issue. People keep hoping some magic course will do it for them, but just like the productivity tools, there is no golden or silver bullet. Good old fashioned sit on your butt and do the work ;-).
What does this have to do with productivity tools? This is based on a university course that teaches you the fundamentals of deep learning. Of course it won't turn you into a ML/DL expert, that's not the point. Anyone completing this course on their own in their free time definitely has the internal drive that you say is lacking so I honestly don't get your comment at all.
I followed this course in person a few years ago. I highly recommend it.
I (we?) would welcome any details from you on how to assess it (and the many alternatives that pop up in response)?
I kind of want to get into this as a rusty full stack developer of several years, but I have no idea how feasible it is to try to get into this field in any capacity with 6 months of study.
If you remember what derivatives are and are decent with math (and some probability), you have what it takes (!), and can pick up most of this easier than say React, in the sense that the incline of the ramp is much less significant, but it will take more time overall (so more of a marathon than a sprint).
Another great resource is NYU's Deep Learning course by Yann LeCun and Alfredo Canziani that is fully available on youtube
https://atcold.github.io/NYU-DLSP20/ https://www.youtube.com/playlist?list=PLLHTzKZzVU9eaEyErdV26...
François, thanks for this.
Is there a course that includes the math pre-requisites?
I noticed on the prerequisites page
basics in signal processing (Fourier transform, wavelets).
Are wavelets really basics in signal processing? We definitely didn't cover this my signals and systems class in EE in either grad school or undergrad.
For those interested in this course, be sure to check out his Little Book of Deep Learning as well! https://fleuret.org/francois/lbdl.html
i'll throw in this set of lectures from Andrej Karpathy. The first lecture is very accessible from a beginner perspective.
https://youtube.com/playlist?list=PLTKMiZHVd_2KJtIXOW0zFhFfB... another entirely approachable course (if you know some Python or similar language), by Sebastian Raschka.
So many options. I just started andrew ng coursera last week. What is the difference between all these free options.
Lots of great resources listed here. But I think there is "Understanding Deep Learning" missing from the list [0]. In my opinion, Simon J.D. Prince accomplished a true feat with his book, not only through the material itself but also with the notes attached to each chapter, linking directly to advanced references (free literature review), exercises that really challenge your understanding of the material, and great notebooks with code that truly materialized the concepts learned (free exercises to give to students if you teach a DL class, but this community is probably not the targeted audience for that).
See also Stanford's YouTube channel, where they post the entire machine learning lecture series (19 videos)
https://m.youtube.com/playlist?list=PLoROMvodv4rNyWOpJg_Yh4N...
They've posted a significant volume of CS lectures if you go to their channel. They're pretty good.