AI 2040 and the cult of intelligence

Article URL: https://geohot.github.io//blog/jekyll/update/2026/07/11/ai-2040.html

Comments URL: https://news.ycombinator.com/item?id=48874200

Points: 155

# Comments: 177

Hacker News 讨论

157 points · 182 comments · 查看原帖

ThrowawayR2
Yep, that's horrifying. Imagine LLMs not just denying information about, to intentionally choose a hot button example, abortions but also invisibly logging a black mark against the asker for thoughtcrime/precrime because the current ruling party has baked their ideology into law. [EDIT] Imagine subtle injection of biases into responses that support the agenda of the current ruling party and failing to include any counterarguments. That wasn't possible with just the internet but it is very possible with centralized LLMs and a public dependent on them for looking up information and doing their reasoning for them. Authoritarians of every stripe must be salivating over the prospect.
mlinsey
I have some sympathy to geohot's view when it comes to pure informational chatbots. It's a first amendment issue, I'm allowed to write and read books that are useful to getting away with crimes, etc. This obviously doesn't work at all when the agents start doing real things in the real world, though. "Hey AI, I don't like my neighbor, find an exploit in the firmware for his car and make the cruise control malfunction and crash him next time he gets on the highway". This is committing a crime, not just talking about theoretical crimes. The AI can and should refuse it. I think he's anticipating and discarding this objection with his introduction, which otherwise feels disconnected from the rest of the article. FWIW, I have changed a bike tire and I'm pretty sure most of the MTS at the big labs could. This sort of "they're just bookworms who don't understand the physical world" rhetoric asi
ks2048
> Like we either live in a world with freedom or we don’t, and like many Americans who have come before, I’m willing to give my life to fighting for it. This is a very simplistic view. "Freedom" isn't binary. In most of "land of the free", I can't even sit on a park bench and drink a can of beer. Yes, this is just a small example of a personal freedom - and not an important, cherished freedom like his examples (freedom to have a robot help you cover-up a murder).
madrox
I am, mostly, aligned with the author here, though I don't think that talking to AI about how to not get caught for killing your wife is really going all the way in. Going all the way in is talking to it about planning a mass shooting. Or producing a nuclear device or bioweapon. Sure, the knowledge itself isn't dangerous unless combined with other traditional behaviors like acquiring certain materials, but maybe there should be a line. Living free shouldn't be a suicide pact.
lilerjee
No central idea, and somewhat chaotic But the one argument is right: > Reality has lots of finicky details. I would like to see the authors of this document try to change a bike tire. Even with a superintelligent ChatGPT, I suspect they would struggle. Details can kill AI, causing them useless or wrong directions.
piker
What a provocative and brilliant way to prove misalignment. It will fall on deaf ears for most but it’s a great litmus test for all: “in your opinion should your AI be permitted to tell you how to cover up a murder?”
tangenter
> I used to be one of these people. I read Yudkowsky and was like, OMG recursive self improvement hard takeoff AI is coming. What you’re seeing are people who live in their heads. We all do it from time to time, but we snap out of it when we need to actually get shit done. These people are tenured in bullshitting and they sound as if they know what they are talking about, but are in reality inept and incapable. They’ve practiced rhetoric and prose here and there, but they profoundly lack experience in things outside of their immediate field.
andy99
It will be interesting to see how the discussion shakes out, this is all stuff that would get you down modded and talked down to if you said if on HN, so far the reactions are positive but I think there’s a massive authoritarian / safetyism contingent here that will disagree hard with almost everything.