Glad to see the bat signal worked     See Less

You can't solve hallucination in humans let alone machines. People and machines make mistakes.     See Less

i tried 3.1 pro. it infinite looped in the thinking phase and wasted countless tokens. then finally after like 5 mins the code it wrote had syntax errors     See Less

The TLDR from Google Gemini's summary
-
The video discusses the release of Gemini 3.1 Pro and the growing confusion surrounding AI model benchmarks (    See More 2_DPnzoiHaY">0:00). The speaker explains that post-training (1:01) is now the dominant stage in LLM development, leading to models excelling in specific domains rather than universally (1:39).

Here's a breakdown of the key points:

Domain Specialization and Benchmarks (1:39): The video highlights that models optimized for specific domains may perform differently in other areas. For example, Claude Opus 4.6, despite being strong in coding, performed poorly in a chess puzzle benchmark (2:01). This shows that older paradigms, where strong performance in one area meant strong performance in all, no longer apply.

ARC-AGI 2 Caveat (3:42): Gemini 3.1 Pro shows impressive results on ARC-AGI 2, outperforming other models (3:42). However, this is tempered by the observation that models might use "unintended arithmetic patterns" from numerical encodings, leading to accidentally correct solutions (4:22).

Simple Bench Record (5:54): Gemini 3.1 Pro set a new record on the speaker's private "Simple Bench," a test of common sense reasoning (5:57). This performance brings it within the margin of error of human average baseline, marking a significant threshold in AI capabilities in text-based tests (6:06).

Hallucination Caveat (8:22): The video addresses the issue of hallucinations (factual inaccuracies) in models. While Gemini 3.1 Pro appears to have a lower percentage of incorrect answers being hallucinations compared to some other models, it's noted that hallucinations are "definitely not a solved problem" (9:36).

Model Card Insights (9:54): The speaker touches on the Gemini 3.1 model card, highlighting that features like "deep think mode" might not always enhance capability despite the inference costs (10:18). However, it also shows examples where 3.1 Pro significantly reduces runtime in machine learning R&D tasks (10:38).

Exponential Growth and Anthropic's Strategy (11:12): The video discusses the rapid revenue growth of AI companies like Anthropic and OpenAI (11:32). Dario Amodei, CEO of Anthropic, suggests that by specializing in enough "specialisms," models might generalize to all specialisms, potentially achieving super intelligence without extensive continual learning (12:55).

The Elusive "One True Benchmark" (15:10): The video questions the existence of a single benchmark for general intelligence, noting that labs are incentivized to create their own benchmarks, which can lead to bias (15:52). Forecasting the future is presented as a truly objective benchmark, with models approaching human forecaster levels (16:01).

Other Metrics and Realism (17:02): The video briefly touches upon other benchmarks like speed, showcasing a model that can generate full answers in milliseconds (17:09). Finally, it highlights the increasing realism in video generation with examples like "Seed Dance 2.0" (17:44).    See Less

LLMS were promised to generalise. Turns out it's whack-a-mole after all. Feel the AGI.     See Less

Boring. The last ten videos just pick arguments with this and that. It’s all about nothing, really.

Buffett used to say the taste of cola doesn’t age — he keeps wanting more. Ot     See More �t have that quality; after a while, you can’t stand their taste.

Your narrative, videos feel the second type now 😟    See Less

Yeah i stopped trusting benchmarks after the release of Gemini Pro 3.0. It killed it in benchmarks and gave me the impression it was the best model. I've been comparing it against ChatGP     See More de subs extensively for the past 2 months using all 3 in tandem with the same prompts (trying to figure out which sub to choose), i found that Gemini 3 is by far the most "confidently wrong" most often. It never says "i don't know", it will answer incorrectly and worst of all it will do it confidently & persuasively. And when you try to correct it it doesn't get the message into its head, it'll either push back or dismiss it. For this reason i started to find Gemini hard to trust with certain things (like troubleshooting etc) because i could never know if it was giving me misinformation or not. There were times that i found that if i hadn't put the effort into verifying what it told me then i would've gone ahead believing what it said was true when it wasn't. You can't trust that.

The conclusion i make from that is that Google must be training it to be good at benchmarks rather than real-world usability.

So these days i default to Claude or ChatGPT (still trying to figure out which one i prefer) simply because if they don't know something they will say so, Claude probably moreso than ChatGPT in this regard, but neither of them are as bad as Gemini 3.

Don't get me wrong, Gemini 3 is better for certain things (e.g. multimodal stuff, and i prefer the way Gemini explains/words things), but until they fix the confident hallucination i can't trust it. At this point the only reason i'm still subscribed is because of NotebookLM    See Less

What about Grok 4.20? Is that one running among the smartest now?     See Less

ive grown cold to ai
seems like incremental improvements won't get us to agi and labs do nothing interesting in that regard     See Less