Replies like "can this be solved through better tooling? better models? better pre-human PR automation?" No. Jesus Christ, No. The solution is school. An education. Learning something. School. Its school.

@mitchellh this can be solved by the technology known as git pre commit hooks

@wyetai_me Cope

@mitchellh I also think a big problem right now is the lack of understanding when it comes to testing code Probably plays a big part in the lack of quality these days

@Hiccup_za The biggest reason is the people using AI are woefully under qualified to be writing the code they're generating.

@mitchellh Do people actually send you commits without like basic testing? Actually compiling being the most basic of all? I mean, I can partially understand it if you have a bunch of engineers writing to a defined API early in the project, but wow.

@seanldunn Yes, happened twice today. I won't link because I don't want to shame them (they were apologetic about it), but you can find it pretty easily.

@mitchellh can this actually be solved with ci and ai?

@elinesterov Solved through school, probably.

@mitchellh Can't CI sort this out? Just auto compile the PR and reject if it doesn't compile?

@Gerry There's already a cognitive load at that point. We also don't allow blind CI runs because there is a security risk to it so I always review the code before approving the CI run. It really is just a human issue at the very front to do the very basic thing of ensuring builds.

@mitchellh this is why i always add "don't make mistakes" to my prompt

@mitchellh Man I got a lot of PRs pre-AI where it was clear people didn't even execute the code, I know @taylorotwell deals with the same shit 🫠 Dummies gonna dumb.

@mitchellh You can def get codex to write extremely thorough bug reports if you tell it to, it wrote this one https://github.com/openai/code... but this was after I told it to test everything and find the most minimal repro with links to all evidence Most people are stupid at prompting

@mitchellh lmfao

@mitchellh like, the test cases "always compile," but they actually don't compile at all because the llm just wrote an arbitrary test that always passes then pretends compiling is actually happening

@mitchellh It helps to have a 1 liner in the readme how to setup and run the ci

@mitchellh I have some unfortunate news for you.

@mitchellh for me it’s the opposite, i am assuming it doesn’t run and asking every PR author “have you tested it works?”

@mitchellh Or, you know, maybe actually RUN TESTS AT YOUR DESK. I know too many people who don't even do that. "Nobody runs them" is the likely reason they're bloated, slow, flaky, and out-of-date.

@mitchellh Using AI is one thing, but not even running and executing the code is another. This clearly indicates they don’t care about maintainer’s time.

@mitchellh “Compiles in my prompt.”

@mitchellh One of the reasons I increasingly prefer statically typed languages (stricter the better) for my projects lately. You can at least say “your example doesn’t compile” as a first line of defense.

@mitchellh “make my app but make no mistakes. this time be less fucking stupid about it.”

@mitchellh Yes, he's talking to me (my reply to Mitchell Hashimoto)

@mitchellh We should brace for an exponential increase in AI slop over the next few years. Babysitting every single thing will become increasingly frustrating, and people will ultimately stop caring. It’ll be hard to stop this…

@mitchellh Our team decided to do a 1 week detox from coding agents because we realised we were using our brains much less when relying on agents for everything

@mitchellh Proof of Compilation?