This rundown is just exceptional     See Less

the age of humanoid robots is upon us... unbelievable. Seemed like a maybe in 100 years sort of thing as a kid     See Less

At this point, we have enough information to speculate on what first AGI would look like.

Firstly, let's discuss the hardware. The Nvidia H100 was announced in March 2022. Mark     See More has publicly claimed that by the end of 2024, there will be 350,000 H100 units and a total compute equivalent to 600,000 H100s. From this, we can infer two things: it takes roughly 3 years from the announcement of a new GPU to establish a datacenter with hundreds of thousands of GPUs, and by the end of 2024, major players like Microsoft and Google will likely have a compute power equivalent to 700,000-1 million H100s. Elon Musk recently tweeted that companies have already announced orders for 700,000 B100s for 2025.

This seems reasonable as the limiting factor for AI accelerators was CoWoS (which is why only 2 million H100s can be produced in 2024), but TSMC will complete the making new fab for CoWoS by the end of Q2 2025, significantly increasing chip production. Additionally, Microsoft is investing $50 billion annually in building their AI data centers, so it's reasonable to think they will buy 700,000 B200s for their next-generation AI data center. The B200 can train AI models three times faster and do inference 30 times faster than the H100.

Sam Altman has claimed in a recent Lex Fridman podcast that he expects AGI to happen in 2030 or slightly before. If he is confident in making that public claim, we can infer that OpenAI plans to have completed AGI by 2029, with training starting in late 2028. In 2025, Nvidia will launch their X100 GPU and corresponding GX200 GPU. I expect it to be based on TSMC's 3 nm node and have at least 2x faster training time compared to the B200. I expect them to buy at least 800,000 to 1 million X100s in 2025, so the data center will be built by 2028.

In summary, by 2028, Microsoft will have roughly 8.8 million H100 equivalents of compute: 700,000 H100 equivalents from 2024, 2.1 million H100 equivalents from 700,000 B100s, and 6 million H100 equivalents from 1 million X100s. It's reasonable to assume they will train the first-gen AGI on at least 3 million H100 equivalents of compute, which would allow them to train a model with at least 500 trillion parameters, compared to 1.7 trillion parameters for GPT-4 and 10 trillion parameters for GPT-5.

The first-generation AGI will have a token window of at least 100 million tokens. Gemini 1.5 Pro already has a token size of 10 million, and Sam Altman recently said in the Lex Fridman podcast that a 1 billion token size is possible, so a 100 million token size for the first-gen AGI seems reasonable. It would have massively improved reasoning ability and excellent multimodality.

OpenAI has extensively researched improving the reasoning ability of large language models. Even in a year-old interview with Ilya Sutskever, he emphasized that future models will have significantly reduced hallucination, a point reiterated by Sam Altman many times.

The research paper "3D-VLA: A 3D Vision-Language-Action Generative World Model" has shown how large language models can be trained to act in a generative world by using a large language model as a reasoning engine. It demonstrates how it can imagine task fulfillment and strategize ways to achieve it. Sora will likely provide an unlimited 3D world in similar fashion to train this AGI in 3d world. DeepMind's latest SIMA paper suggests that research labs are training models to understand and complete language tasks in a 3D world.

This AGI model will be trained not only on text but also on video, audio, and the 3D world. Will have 100 million token size, long term memory with massively improved RAG. Massively reduced hallucination. High Agentic capability, 500 trillion parameter.
If would generate entire 3 hrs movie with single prompt. I mean if Sora had improve customization and consistent characters then a really reliable GPT like system can generate prompt for various video and link them together to make entire movie.
Sora should be able to make full fledge game with single prompt by video generation combined with neRF like system.
This AGI like system should be able to do million years worth of material science, drug discovery, cure aging, massive improve the field of mathematics etc. The future is going to be awesome.    See Less

"(...) except for all future years" I still don't understand what this means. does it mean it won't be as impressive? I find that hard to believe. Is it because it'll b     See More at a slower rate?

Also, with the recent advances of quantum computing, I can't help but think if we will eventually start using quantum computers instead of traditional computers and how much faster we'll progress. I really am looking forward to that, it may actually be the key to achieve AGI this decade    See Less

Sam Altman doing his Elon Musk speech style impersonation is so f'ing cringe     See Less

I just don't see AGI completely taking over jobs... Will companies try and do so to save money? Absolutely. But if there's no jobs left, then how are people supposed to make money     See More ff to give these companies in the first place? Unless these systems are owned by people and can make money? That just sounds like slavery with extra steps...    See Less

"Probably"     See Less

7:38 - If an AGI could train itself - then yes, the results would be beyond human.     See Less

10 years does not mean inches closer. Clickbait in AI videos is a big problem.     See Less

That Altman low crackly voice thing ('vocal fry' it's called) drives me fucking nuts - you just want him to cough and clear his throat. You hear so many Americans do it now. Mayb     See More ieve it gives them a sense of gravitas, self importance, seriousness etc, but it just makes then sound as if they're talking through half a pint of thick phlegm stuck to their vocal chords.    See Less