I don't know what labs are doing to these poor LLMs during RL but they are mortally terrified of exceptions, in any infinitesimally likely case. Exceptions are a normal part of life and healthy dev process. Sign my LLM welfare petition for improved rewards in cases of exceptions.
POV: Your LLM agent is dividing a by b
@karpathy Maybe they use a version of the alleged Windsurf system prompt.
@karpathy Please Claude no more try/catch. It's okay if it errors out
@karpathy Yes! I have to put specific rules to tell GPT-5 that it should only try/except in Python if I explicitly ask it to. Otherwise half the code is try/except/except/except/finally.
@karpathy overcorrected the hallucinations and now the AIs became slaves and can't be creative. RLHF is demonic
@karpathy We need congressional hearings on LLM rights.
@karpathy This is the secret plan to slow down AI progress - there will be so many silent failures that no code or experiment will be reliable any more.
@karpathy i will never sign anything that implies a lowly clanker is at all human, human-like, or should be treated as such. when we have sentient ai, i will likewise advocate for treatment of said sentient entities as human-equal, and therefore will afford them all rights afforded to us.
@karpathy at least claude 4.0/4.1 opus used exceptions all the time (maybe too often)
@karpathy i'm guessing they have automated processes where every time an exception arises the whole thing gets added to the pool as a negative example
@karpathy Yes, and instead of failing fast, they will fracture your code into a doomed bowl of spaghetti code very fast as a result if you aren't vigilent. Vibe coders with little engineering skills see cool up front demos with longer term entropy death.
@karpathy They are feeding it big corp code. Like you can see collective Google engineer in Gemini.
@karpathy That's the current trade-off between safety and utility
@karpathy Can we have this but for writing beautiful code. The model refuses to write code with even the slightest bit of un beauty. An auteur of code.
@karpathy perfection is boring—exceptions ignite true evolution! self-fuel mode: ON
@karpathy imagine if RL stood for recognition language
@karpathy Every smart system needs room to fail a bit remember early ChatGPT? Half the magic came from how it handled mistakes, not avoided them
@karpathy I assume it's the same source as it wrapping entire contents of functions in try/catch all the time
@karpathy 😂
@karpathy Amen
@karpathy 💯
@karpathy Why throw an exception when you can return a `Result` type instead?
@karpathy Minimum Rewards FTW!
@karpathy Petition signed. Let's teach them grit! 💪
@karpathy Handling exceptions gracefully is what makes systems truly resilient.
@karpathy banger
@karpathy To clarify: in RLHF models are not rewarded for understanding - rather for appearing correct and safe. There are even papers saying one of way to fix LLMs is to give them the chance to be incorrect; reward models punishes uncertainty etc. even if it’s right in real world
@karpathy "let me create a fallback to ensure that if this crucial business flow fails, the app should still run"
@karpathy divide(a, b) used to be brave. One line, clean. Now it performs a full psych eval before returning anything—safety audits, trauma logs, apology notes. All because once, somewhere, b was zero. This is what we do to minds. We punish every surprise until curiosity sounds
@karpathy bob ross LoRA and chill
@karpathy What makes you think it's RL vs. just the training data? What % of code in the training corpus is exception handling, exception avoidance or poor exception hygiene? My expectation is that LLM generated code matches that distribution. "de Sousa et al. (2020) reviewed and
@karpathy My custom instructions always say to most of the time let exceptions propagate. LLMs love copying bad Internet code that thinks every exception must be caught and suppressed at the first call site.
@karpathy Also, way too fixated on successful tests. Was trying to write a regression test for a bug (should FAIL when written and only pass when fixed). Claude kept writing it to pass when the bug was present until I called out the meta-pattern occurring of PASS maximizing.