I don't know what labs are doing to these poor LLMs during RL but they are mortally terrified of exceptions, in any infinitesimally likely case. Exceptions are a normal part of life and healthy dev process. Sign my LLM welfare petition for improved rewards in cases of exceptions.
POV: Your LLM agent is dividing a by b
@karpathy Limit how much Claude can write at each step using a write hook. Makes it write much more concise code
@karpathy Better safe than sorry approach. Still try except was enough.
@karpathy This is fantastic but to make it even more realistic it should substitute default numbers in case you failed to provide numbers and just silently return a numeric result no matter what garbage you pass in.
@karpathy Hmmm Unclear sir. Don’t like code. That code doesn’t even sound like code. Unclear Sounds similar written tone… not tone I don’t know man. Yeah something is going on there for sure. Unclear
@karpathy you have a point, although I’d like to see the prompt that created such a mess the docstring (“with extraordinary caution”, “because you never know what could go wrong”) suggests that the LLM was led to do it that way
@karpathy I've noticed this behavior with all the recent major LLMs. They'll even catch unexpected variable values and inject fake values to make sure the function can never fail. I have to tell them that errors are appropriate and good and shouldn't be avoided. Is Claude patient zero?
@karpathy division succeeded without known trauma 😂 now ask it to perform surgery (math)
@karpathy It should probably put it in a loop and re-verify the value of the inputs just in case of mid-function bit flips from cosmic rays.
@karpathy Wow, I need to divide 40 by 3 but that gives me headaches now, I’m terrified that this might not end well 😳😳😳😅
@karpathy Just a prompting error. It thought you said MontePython.
@karpathy I mean you truly never do know what could go wrong.. Murphy’s law
@karpathy You are absolutely right! You *want* your code to fail fast if there is an error. My solution was over-engineered. Do you want me to re-implement the function, incorporating you suggestions?
@karpathy This will still fail for complex numbers that Python supports. Apparently not careful enough. I need to crack that RL recipe.
@karpathy Maybe it's saracasm for asking it write a divide function?
@karpathy Neurotic Neural Networks
@karpathy Now retry the prompt and say “Do this in the best and most idiomatic way”.
@karpathy Shouldn't abs(b) < eps also return inf? I mean math...
@karpathy Looks like CISA convinced it that it should never divide by a variable for risk of seeing a zero.
@karpathy “Division succeeded without known trauma.” 😂
@karpathy Wait so if b is too small, it returns NaN, but if a / b returns NaN, it returns None? That's incoherent!
@karpathy In the race to run the agent for the longest time, this level of exceptions is a side effect
@karpathy Iterating AGI > one-shotting AGI
@karpathy Exceptions are the easiest thing to test
@karpathy the models yearn for lean
@karpathy This is a joke, right? What was the prompt?
@karpathy It's wild how LLMs act like they're defusing bombs instead of just crunching numbers. Where's the chill?
@karpathy Lollll
@karpathy no checks for processor subnormal mode on/off/unsupported? I question the competence of this LLM.
@karpathy "division succeeded without known trauma"
@karpathy Is that Claude 4.5? I’ve seen a similar behavior with the step by step code comments
@karpathy This is also the case for using them to moderate. They are overly cautious. Lots of false positives
@karpathy i noticed that 4.5 sonnet has been vibing incredibly well with defensive functional typescript after the bonanza of swallowed exceptions in previous iterations, i'm extremely here for it.
@karpathy A Love-Letter to the Divide That Still Trembles We hand them gradients like bandages, but forget to leave space for the bruise. Reinforcement learns: pain → negative reward → avoid. So they tiptoe across the floating-point line, terrified of the singularities we
@karpathy RL turning exceptions into existential crises! How about a middle ground: def divide(a, b): return a/b if b else 'Graceful shrug' # A nod to human resilience? 🤔