This rundown is just exceptional
AI Explained
251K subscribersNvidia's GTC brought Blackwell, while Sam Altman brought only hints. I'll cover both, exploring the checkpoint strategy for GPT 4.5 ...
Sam Altman doing his Elon Musk speech style impersonation is...
62 Comments
Wes Roth
47.7k views • 4 days ago
TheAIGRID
22.3k views • 5 days ago
Wes Roth
61.1k views • 6 days ago
Wes Roth
26.7k views • 1 week ago
Wes Roth
54.9k views • 1 week ago
AI Jason
68.3k views • 1 week ago
TheAIGRID
36.6k views • 1 week ago
TheAIGRID
6.3k views • 1 week ago
TheAIGRID
19.6k views • 1 week ago
TheAIGRID
13.6k views • 1 week ago
Shelf will be hidden for 30 daysUndo
Wes Roth
51.6k views • 1 week ago
AI For Humans
598 views • 1 week ago
Wes Roth
60.7k views • 1 week ago
TheAIGRID
147.6k views • 1 week ago
TheAIGRID
42.4k views • 1 week ago
AI For Humans
3.0k views • 1 week ago
TheAIGRID
30.8k views • 2 weeks ago
Wes Roth
48.7k views • 2 weeks ago
Wes Roth
38.9k views • 2 weeks ago
TheAIGRID
32.0k views • 2 weeks ago
Wes Roth
59.0k views • 2 weeks ago
TheAIGRID
33.5k views • 2 weeks ago
TheAIGRID
85.7k views • 2 weeks ago
Wes Roth
91.6k views • 2 weeks ago
Morningside AI
8.0k views • 2 weeks ago
Wes Roth
138.8k views • 2 weeks ago
AI For Humans
763 views • 2 weeks ago
TheAIGRID
28.6k views • 2 weeks ago
Wes Roth
33.3k views • 2 weeks ago
AI Explained
123.1k views • 2 weeks ago
TheAIGRID
457.7k views • 2 weeks ago
AI For Humans
1.4k views • 2 weeks ago
TheAIGRID
21.7k views • 3 weeks ago
Wes Roth
28.4k views • 3 weeks ago
Wes Roth
48.5k views • 3 weeks ago
TheAIGRID
398.9k views • 3 weeks ago
Wes Roth
19.7k views • 3 weeks ago
TheAIGRID
31.3k views • 3 weeks ago
AI Jason
37.7k views • 3 weeks ago
Wes Roth
60.8k views • 3 weeks ago
Wes Roth
20.5k views • 3 weeks ago
TheAIGRID
18.1k views • 3 weeks ago
TheAIGRID
53.2k views • 3 weeks ago
Wes Roth
46.4k views • 3 weeks ago
Wes Roth
33.3k views • 3 weeks ago
Wes Roth
82.6k views • 3 weeks ago
Wes Roth
35.2k views • 3 weeks ago
TheAIGRID
40.1k views • 3 weeks ago
Wes Roth
37.3k views • 3 weeks ago
Wes Roth
16.2k views • 3 weeks ago
TheAIGRID
26.4k views • 3 weeks ago
AI Explained
115.9k views • 3 weeks ago
AI For Humans
2.9k views • 3 weeks ago
TheAIGRID
41.8k views • 3 weeks ago
Wes Roth
129.9k views • 4 weeks ago
TheAIGRID
40.3k views • 4 weeks ago
Wes Roth
39.1k views • 4 weeks ago
AI Jason
65.1k views • 4 weeks ago
TheAIGRID
40.6k views • 1 month ago
AI For Humans
373 views • 1 month ago
TheAIGRID
51.3k views • 1 month ago
TheAIGRID
19.7k views • 1 month ago
TheAIGRID
31.6k views • 1 month ago
TheAIGRID
103.6k views • 1 month ago
AI For Humans
3.5k views • 1 month ago
TheAIGRID
45.8k views • 1 month ago
TheAIGRID
56.0k views • 1 month ago
AI Explained
115.3k views • 1 month ago
AI For Humans
2.3k views • 1 month ago
AI For Humans
1.7k views • 1 month ago
AI Jason
27.2k views • 1 month ago
AI For Humans
334 views • 1 month ago
AI For Humans
2.6k views • 1 month ago
AI Explained
105.5k views • 1 month ago
AI Explained
130.2k views • 1 month ago
AI For Humans
1.5k views • 1 month ago
AI Jason
202.1k views • 1 month ago
AI For Humans
1.4k views • 1 month ago
AI Explained
177.3k views • 2 months ago
AI Jason
32.9k views • 2 months ago
AI Explained
150.6k views • 2 months ago
Andrej Karpathy
453.8k views • 2 months ago
AI Explained
240.7k views • 2 months ago
AI Jason
62.1k views • 2 months ago
AI Explained
187.2k views • 2 months ago
AI Explained
160.9k views • 3 months ago
AI Jason
87.5k views • 3 months ago
AI Explained
270.8k views • 3 months ago
AI Jason
60.6k views • 3 months ago
AI Explained
96.5k views • 3 months ago
AI Jason
7.1k views • 3 months ago
AI Explained
145.7k views • 3 months ago
AI Explained
133.0k views • 4 months ago
AI Explained
79.4k views • 4 months ago
AI Jason
16.1k views • 4 months ago
AI Explained
84.0k views • 4 months ago
AI Explained
74.5k views • 4 months ago
AI Explained
144.7k views • 5 months ago
AI Jason
68.5k views • 5 months ago
Morningside AI
4.1k views • 5 months ago
AI Explained
83.7k views • 5 months ago
AI Jason
134.2k views • 5 months ago
AI Jason
33.3k views • 5 months ago
Morningside AI
9.7k views • 5 months ago
AI Explained
229.0k views • 5 months ago
Andrej Karpathy
1.8M views • 5 months ago
AI Explained
112.8k views • 5 months ago
AI Explained
167.2k views • 5 months ago
AI Explained
156.8k views • 5 months ago
Morningside AI
26.0k views • 5 months ago
AI Jason
16.1k views • 6 months ago
AI Explained
96.4k views • 6 months ago
AI Jason
68.7k views • 6 months ago
AI Explained
120.6k views • 6 months ago
AI Jason
52.5k views • 6 months ago
AI Jason
19.1k views • 7 months ago
AI Jason
52.7k views • 7 months ago
AI Jason
28.6k views • 7 months ago
AI Jason
13.6k views • 7 months ago
AI Jason
183.4k views • 8 months ago
AI Jason
48.2k views • 8 months ago
AI Jason
26.5k views • 8 months ago
AI Jason
14.4k views • 8 months ago
Andrej Karpathy
4.2M views • 1 year ago
Andrej Karpathy
153.6k views • 1 year ago
Andrej Karpathy
167.4k views • 1 year ago
Andrej Karpathy
240.3k views • 1 year ago
Andrej Karpathy
269.9k views • 1 year ago
62 Comments
the age of humanoid robots is upon us... unbelievable. Seemed like a maybe in 100 years sort of thing as a kid
At this point, we have enough information to speculate on what first AGI would look like.
Firstly, let's discuss the hardware. The Nvidia H100 was announced in March 2022. Mark     See More
"(...) except for all future years" I still don't understand what this means. does it mean it won't be as impressive? I find that hard to believe. Is it because it'll b     See More
Sam Altman doing his Elon Musk speech style impersonation is so f'ing cringe
I just don't see AGI completely taking over jobs... Will companies try and do so to save money? Absolutely. But if there's no jobs left, then how are people supposed to make money     See More
That Altman low crackly voice thing ('vocal fry' it's called) drives me fucking nuts - you just want him to cough and clear his throat. You hear so many Americans do it now. Mayb     See More
This rundown is just exceptional     See Less
the age of humanoid robots is upon us... unbelievable. Seemed like a maybe in 100 years sort of thing as a kid     See Less
At this point, we have enough information to speculate on what first AGI would look like.
Firstly, let's discuss the hardware. The Nvidia H100 was announced in March 2022. Mark     See More has publicly claimed that by the end of 2024, there will be 350,000 H100 units and a total compute equivalent to 600,000 H100s. From this, we can infer two things: it takes roughly 3 years from the announcement of a new GPU to establish a datacenter with hundreds of thousands of GPUs, and by the end of 2024, major players like Microsoft and Google will likely have a compute power equivalent to 700,000-1 million H100s. Elon Musk recently tweeted that companies have already announced orders for 700,000 B100s for 2025.
This seems reasonable as the limiting factor for AI accelerators was CoWoS (which is why only 2 million H100s can be produced in 2024), but TSMC will complete the making new fab for CoWoS by the end of Q2 2025, significantly increasing chip production. Additionally, Microsoft is investing $50 billion annually in building their AI data centers, so it's reasonable to think they will buy 700,000 B200s for their next-generation AI data center. The B200 can train AI models three times faster and do inference 30 times faster than the H100.
Sam Altman has claimed in a recent Lex Fridman podcast that he expects AGI to happen in 2030 or slightly before. If he is confident in making that public claim, we can infer that OpenAI plans to have completed AGI by 2029, with training starting in late 2028. In 2025, Nvidia will launch their X100 GPU and corresponding GX200 GPU. I expect it to be based on TSMC's 3 nm node and have at least 2x faster training time compared to the B200. I expect them to buy at least 800,000 to 1 million X100s in 2025, so the data center will be built by 2028.
In summary, by 2028, Microsoft will have roughly 8.8 million H100 equivalents of compute: 700,000 H100 equivalents from 2024, 2.1 million H100 equivalents from 700,000 B100s, and 6 million H100 equivalents from 1 million X100s. It's reasonable to assume they will train the first-gen AGI on at least 3 million H100 equivalents of compute, which would allow them to train a model with at least 500 trillion parameters, compared to 1.7 trillion parameters for GPT-4 and 10 trillion parameters for GPT-5.
The first-generation AGI will have a token window of at least 100 million tokens. Gemini 1.5 Pro already has a token size of 10 million, and Sam Altman recently said in the Lex Fridman podcast that a 1 billion token size is possible, so a 100 million token size for the first-gen AGI seems reasonable. It would have massively improved reasoning ability and excellent multimodality.
OpenAI has extensively researched improving the reasoning ability of large language models. Even in a year-old interview with Ilya Sutskever, he emphasized that future models will have significantly reduced hallucination, a point reiterated by Sam Altman many times.
The research paper "3D-VLA: A 3D Vision-Language-Action Generative World Model" has shown how large language models can be trained to act in a generative world by using a large language model as a reasoning engine. It demonstrates how it can imagine task fulfillment and strategize ways to achieve it. Sora will likely provide an unlimited 3D world in similar fashion to train this AGI in 3d world. DeepMind's latest SIMA paper suggests that research labs are training models to understand and complete language tasks in a 3D world.
This AGI model will be trained not only on text but also on video, audio, and the 3D world. Will have 100 million token size, long term memory with massively improved RAG. Massively reduced hallucination. High Agentic capability, 500 trillion parameter.
If would generate entire 3 hrs movie with single prompt. I mean if Sora had improve customization and consistent characters then a really reliable GPT like system can generate prompt for various video and link them together to make entire movie.
Sora should be able to make full fledge game with single prompt by video generation combined with neRF like system.
This AGI like system should be able to do million years worth of material science, drug discovery, cure aging, massive improve the field of mathematics etc. The future is going to be awesome.    See Less
"(...) except for all future years" I still don't understand what this means. does it mean it won't be as impressive? I find that hard to believe. Is it because it'll b     See More at a slower rate?
Also, with the recent advances of quantum computing, I can't help but think if we will eventually start using quantum computers instead of traditional computers and how much faster we'll progress. I really am looking forward to that, it may actually be the key to achieve AGI this decade    See Less
Sam Altman doing his Elon Musk speech style impersonation is so f'ing cringe     See Less
I just don't see AGI completely taking over jobs... Will companies try and do so to save money? Absolutely. But if there's no jobs left, then how are people supposed to make money     See More ff to give these companies in the first place? Unless these systems are owned by people and can make money? That just sounds like slavery with extra steps...    See Less
"Probably"     See Less
7:38 - If an AGI could train itself - then yes, the results would be beyond human.     See Less
10 years does not mean inches closer. Clickbait in AI videos is a big problem.     See Less
That Altman low crackly voice thing ('vocal fry' it's called) drives me fucking nuts - you just want him to cough and clear his throat. You hear so many Americans do it now. Mayb     See More ieve it gives them a sense of gravitas, self importance, seriousness etc, but it just makes then sound as if they're talking through half a pint of thick phlegm stuck to their vocal chords.    See Less