I keep throwing away LLM generated code
I've been messing around with AI tools like Codex and Claude for the last few months. I'm a late bloomer.
At several points in the process I found it to be remarkable at understanding the software I'm writing. With my general skepticism of these tools I've been reading every line very carefully and it's getting things generally right.
As I transitioned parts of my workflow to AI I've realized, however, that I throw away a lot more code than what I keep. Recently, while working on React, something that should have plenty of training data, I threw away all of the code and started from scratch again. Maybe this is my fault but I think a problem has emerged; I'm lying to myself about how closely I'm paying attention, the feeling of progress is too stimulating and I get sloppy. I didn't even realize it and professionally that is concerning as I'm paid to pay attention.
I think I've learned that the current generation of AI models is not good at creating abstractions or considering how a project will mature. That is most of what my day-to-day duties require. The LLM doesn't have an intuition about how software will mature it only knows about what has been and what you tell it. That intuition isn't easy to describe and I doubt most engineers are going to try to express that nuance at every prompt.
I've adjusted my workflow to match this limitation. I now get a solid structure of the software written 100% by me. I build the abstractions and core flow. I only reach for AI when I'm ready to flesh out some boilerplate after the LLM has a lot of examples of the structure. The result is that I reach for AI a lot less to build anything.
I have found that having Claude is pretty good as a quick search engine. From Neovim I can select some code, send it to Claude, and ask a question with the results tailored to the project. It has replaced a lot of Google. When writing in Python, Go, JS, TS, SQL, CSS, etc etc it gets hard to remember all the syntaxes for types where each is just a little different. Claude is quick and efficient at that even with older models.
What I've stopped doing is expecting more from it than it can do.