LLM & Outsourced Thinking

I remember when I first used an LLM. It was on a Hugging Face instance, a model called BLOOM. Before that, my experience with generative natural languages has been only with a few markov chain models. They were extremely buggy and incoherent, but had an inexplicable charm to them. BLOOM, on the other hand, was able to predict a prose with keen awareness of contextual information. You can't converse with BLOOM, unlike modern interactive LLMs like ChatGPT. Give it a half complete paragraph, and it will auto-regressively predict the next token iteratively. I tried to simulate conversations, but they often derailed into drivel nonsense. Nevertheless, it was fun to just play around with the stochastic parrot and see what it could spit out.

It's been three years since then. Vibe-coding is all the rage. Imagine writing a prompt, clicking a button, and a fully functional program is ready to ship. People willingfully grant full access to their computers to AI agents for them to do their work. If you can't maintain an AI-generated application, ask your agent to fix it for you. Software engineers are going to be replaced, and the art of programming is dead. In the near future, you will wake up and your agent will help you do everything. How awesome is that? (The last part is a hyperbole, of course.)

Humans are inefficient. They can't spit out lines of code so fast that, when measured in lines per second, yields a human readable number (unless we talk about competitive programming.) Sometimes, they idle, take a break, or consume sustenances. When I'm stuck, I sit there to ponder. A lack of answer, and boredom, are food for the brain, which works in the background constantly rewiring neurons. Get your answer from an LLM? Good luck with that. At least you'll be able to solve a problem much faster, I guess, but what makes you different from an AI agent if you don't think?

The most important part of building a quality program happens, in my opinion, in those moments when you do nothing. Most of my "voila"s came from staring at the monitor or taking a stroll in front of my seat: the structure doesn't seem right after implementation even with the diagram already written; there must be a better way to solve this, consider cache locality; i need a break. These are behaviors LLMs can't replicate. Do they know what taking a break is? Do they even know anything other than the token with the highest probability based on their embeddings? I guess you'll have to take that to philosophers.

Do they even know anything?