17000 into 30 dimensions is quite a bit of space - eg, if each dimension is only 2 values, 30 dimensions gives you a billion unuique locations (2^30)
Andrej Karpathy
446K subscribersWe implement a multilayer perceptron (MLP) character-level language model. In this video we also introduce many basics of ...
This is amazing. Using just a little bit of what I was able...
35 Comments
Wes Roth
4.7k views • 11 hours ago
TheAIGRID
11.2k views • 20 hours ago
TheAIGRID
31.4k views • 1 day ago
Wes Roth
39.6k views • 1 day ago
TheAIGRID
22.7k views • 3 days ago
Wes Roth
51.0k views • 4 days ago
TheAIGRID
26.5k views • 4 days ago
TheAIGRID
9.2k views • 4 days ago
TheAIGRID
35.6k views • 6 days ago
Wes Roth
62.8k views • 6 days ago
Shelf will be hidden for 30 daysUndo
TheAIGRID
45.1k views • 6 days ago
Wes Roth
27.1k views • 1 week ago
Wes Roth
55.4k views • 1 week ago
AI Jason
81.4k views • 1 week ago
TheAIGRID
36.6k views • 1 week ago
TheAIGRID
6.3k views • 1 week ago
TheAIGRID
19.6k views • 1 week ago
TheAIGRID
13.6k views • 1 week ago
Wes Roth
51.9k views • 1 week ago
AI For Humans
598 views • 1 week ago
Wes Roth
60.9k views • 1 week ago
TheAIGRID
147.6k views • 1 week ago
TheAIGRID
42.4k views • 1 week ago
AI For Humans
3.0k views • 2 weeks ago
TheAIGRID
30.8k views • 2 weeks ago
Wes Roth
48.9k views • 2 weeks ago
Wes Roth
39.1k views • 2 weeks ago
TheAIGRID
32.0k views • 2 weeks ago
Wes Roth
59.6k views • 2 weeks ago
TheAIGRID
33.5k views • 2 weeks ago
TheAIGRID
85.7k views • 2 weeks ago
Wes Roth
92.3k views • 2 weeks ago
Morningside AI
8.0k views • 2 weeks ago
Wes Roth
139.6k views • 2 weeks ago
AI For Humans
763 views • 2 weeks ago
TheAIGRID
28.6k views • 2 weeks ago
Wes Roth
33.4k views • 2 weeks ago
AI Explained
123.1k views • 2 weeks ago
TheAIGRID
457.7k views • 2 weeks ago
AI For Humans
1.4k views • 3 weeks ago
TheAIGRID
21.7k views • 3 weeks ago
Wes Roth
28.5k views • 3 weeks ago
Wes Roth
48.6k views • 3 weeks ago
TheAIGRID
398.9k views • 3 weeks ago
Wes Roth
19.7k views • 3 weeks ago
TheAIGRID
31.3k views • 3 weeks ago
AI Jason
38.3k views • 3 weeks ago
Wes Roth
60.8k views • 3 weeks ago
Wes Roth
20.6k views • 3 weeks ago
TheAIGRID
18.1k views • 3 weeks ago
TheAIGRID
53.2k views • 3 weeks ago
Wes Roth
46.5k views • 3 weeks ago
Wes Roth
33.5k views • 3 weeks ago
Wes Roth
83.3k views • 3 weeks ago
Wes Roth
35.4k views • 3 weeks ago
TheAIGRID
40.1k views • 3 weeks ago
Wes Roth
37.4k views • 3 weeks ago
Wes Roth
16.2k views • 3 weeks ago
TheAIGRID
26.4k views • 3 weeks ago
AI Explained
115.9k views • 3 weeks ago
AI For Humans
2.9k views • 4 weeks ago
TheAIGRID
41.8k views • 4 weeks ago
TheAIGRID
40.3k views • 4 weeks ago
AI Jason
68.2k views • 1 month ago
TheAIGRID
40.6k views • 1 month ago
AI For Humans
373 views • 1 month ago
TheAIGRID
51.3k views • 1 month ago
TheAIGRID
19.7k views • 1 month ago
TheAIGRID
31.6k views • 1 month ago
TheAIGRID
103.6k views • 1 month ago
AI For Humans
3.5k views • 1 month ago
TheAIGRID
45.8k views • 1 month ago
TheAIGRID
56.0k views • 1 month ago
AI Explained
115.3k views • 1 month ago
AI For Humans
2.3k views • 1 month ago
AI For Humans
1.7k views • 1 month ago
AI Jason
27.4k views • 1 month ago
AI For Humans
334 views • 1 month ago
AI For Humans
2.6k views • 1 month ago
AI Explained
105.5k views • 1 month ago
AI Explained
130.2k views • 1 month ago
AI For Humans
1.5k views • 1 month ago
AI Jason
203.2k views • 1 month ago
AI For Humans
1.4k views • 1 month ago
AI Explained
177.3k views • 2 months ago
AI Jason
33.0k views • 2 months ago
AI Explained
150.6k views • 2 months ago
Andrej Karpathy
453.8k views • 2 months ago
AI Explained
240.7k views • 2 months ago
AI Jason
62.2k views • 2 months ago
AI Explained
187.2k views • 2 months ago
AI Explained
160.9k views • 3 months ago
AI Jason
87.8k views • 3 months ago
AI Explained
270.8k views • 3 months ago
AI Jason
60.7k views • 3 months ago
AI Explained
96.5k views • 3 months ago
AI Jason
7.1k views • 3 months ago
AI Explained
145.7k views • 3 months ago
AI Explained
133.0k views • 4 months ago
AI Explained
79.4k views • 4 months ago
AI Jason
16.2k views • 4 months ago
AI Explained
84.0k views • 4 months ago
AI Explained
74.5k views • 4 months ago
AI Explained
144.7k views • 5 months ago
AI Jason
68.7k views • 5 months ago
Morningside AI
4.1k views • 5 months ago
AI Explained
83.7k views • 5 months ago
AI Jason
134.4k views • 5 months ago
AI Jason
33.3k views • 5 months ago
Morningside AI
9.7k views • 5 months ago
AI Explained
229.0k views • 5 months ago
Andrej Karpathy
1.8M views • 5 months ago
AI Explained
112.8k views • 5 months ago
AI Explained
167.2k views • 5 months ago
AI Explained
156.8k views • 5 months ago
Morningside AI
26.0k views • 5 months ago
AI Jason
16.1k views • 6 months ago
AI Explained
96.4k views • 6 months ago
AI Jason
68.8k views • 6 months ago
AI Explained
120.6k views • 6 months ago
AI Jason
52.6k views • 6 months ago
AI Jason
19.1k views • 7 months ago
AI Jason
52.7k views • 7 months ago
AI Jason
28.6k views • 7 months ago
AI Jason
13.6k views • 7 months ago
AI Jason
183.6k views • 8 months ago
AI Jason
48.3k views • 8 months ago
AI Jason
26.5k views • 8 months ago
AI Jason
14.4k views • 8 months ago
Andrej Karpathy
4.2M views • 1 year ago
Andrej Karpathy
153.6k views • 1 year ago
Andrej Karpathy
167.4k views • 1 year ago
Andrej Karpathy
240.3k views • 1 year ago
Andrej Karpathy
269.9k views • 1 year ago
35 Comments
17000 into 30 dimensions is quite a bit of space - eg, if each dimension is only 2 values, 30 dimensions gives you a billion unuique locations (2^30)
Andrey thank you for this great series of lectures. you are a great Educator! 100% GOLD Material to Learn
I'm confused at 56:17 why care must be taking with how many times you can use the test dataset as the model will lear     See More
at 21:24 I think it's supposed to be first letter not first word. It's first word in the paper but first letter i     See More
Love all the tips and explanations on pytorch, training efficiency, and educational purposed errors. I was writing both code and notes and rewatching and enjoyed it and felt having a fruitfu     See More
Can't thank you enough. It's such a satisfying feeling to understand the logic under the ML models clearly. Thank you!
This is amazing. Using just a little bit of what I was able to learn from part 3, namely the Kaiming init, and turning back on the learning rate decay, I was able to achieve 2.03 and 2.04 in     See More
17000 into 30 dimensions is quite a bit of space - eg, if each dimension is only 2 values, 30 dimensions gives you a billion unuique locations (2^30)     See Less
Andrey thank you for this great series of lectures. you are a great Educator! 100% GOLD Material to Learn     See Less
I'm confused at 56:17 why care must be taking with how many times you can use the test dataset as the model will lear     See More Is this because there is no equivalent of 'torch.no_grad()' for LLMs - will the LLM always update the weights when given data?    See Less
i love u     See Less
at 21:24 I think it's supposed to be first letter not first word. It's first word in the paper but first letter i     See More ple    See Less
Love all the tips and explanations on pytorch, training efficiency, and educational purposed errors. I was writing both code and notes and rewatching and enjoyed it and felt having a fruitfu     See More r finished. It's like I was learning with a kind and insightful mentor sitting next to me. Thanks so much Andrej.    See Less
It is an absolute honor to learn from the very best. Thanks Andrej.     See Less
thank you andrej     See Less
Can't thank you enough. It's such a satisfying feeling to understand the logic under the ML models clearly. Thank you!     See Less
This is amazing. Using just a little bit of what I was able to learn from part 3, namely the Kaiming init, and turning back on the learning rate decay, I was able to achieve 2.03 and 2.04 in     See More nd validation with a 1.89 in my training loss with just 300k iterations and 23k parameters. I set my block size to 4 and my embeddings to 12 and increased my hidden layer to 300 while decaying my learning rate exponent from -1 to -3 linear space over the 300k steps. All that without even using batch normalization yet. After applying batch norm, was able to get these down to 1.99 and 1.98 with training loss in the 1.7s after a little more tweaking. Really good content in this lecture, it really has me feeling like a chef in the kitchen almost, cooking up a model with a few turns of the knobs...This sounds like a game or a problem that can be solved with an AI trained on turning knobs.    See Less