blog - git - desktop - contact


Still a "no" to AI from me

Here's an incomplete but hopefully concise list of reasons against AI. Might expand or add sources later.

Preface: It is not about usefulness or quality

I want to make it clear that I'm not questioning usefulness or quality per se. Depending on what you're doing, AI can help you with both or neither.

But these two properties alone are not enough. A lot of tools can be very benificial to you. What I'm questioning is this: Is it a good idea to use every tool available to you, no matter what? How well did this approach work for us in the past? What will be the consequences, short term or long term?

See also:

The list

Climate

AI is more demanding on hardware, at the same time we're facing a bit of a climate issue. Can we really afford to spend more, build more? Shouldn't we try to shrink and be more efficient? Cut out what isn't really needed? Invest brain power to find simple and elegant solutions to problems?

See also:

Dependency

There's the fundamental issue that all these AI tools make you depend on some big corporation. This is already the case in many other areas, but AI makes it much worse, especially when you want me to use it for writing programs. I don't want to buy into another subscription with skyrocketing prices, nor do I want to depend on their availability, their opinions, ... I consider programming to be one of the most fundamental things you can do with a computer, and willingly accepting such a strong dependency is -- frankly -- absurd to me.

And even when you run these things locally, you first need a model, which you cannot feasibly build yourself. You always need a "supplier" of some kind.

"Leaking" data

The issue that most current tools don't run on your devices, so you're shoving lots of valuable data or even credentials to someone else.

Brain drain

There's the (upcoming) brain drain. AI takes over the jobs of juniors. Hence no more juniors. Hence no more seniors -- in a few years. You need to train people and you need to do that on real world projects, otherwise they don't learn the skills that they need.

Unlearning skills

"Use it or lose it." By letting AI do all the tasks, you will eventually no longer be able to do them yourself, because you have forgotten how it works. This will also affect your ability to judge an AI's output: At the moment, you might be a highly skilled developer who can quickly tell if something is right or wrong, but that won't be the case anymore eventually.

Giving up formal languages

There's the issue that "prompts" are very vague because they use natural language instead of well-specified formal language. We willingly give up that precision. Instead of knowing exactly what each word in the prompt means, everything is fuzzy and ambiguous -- because that's a core property of natural languages.

Often times, prompts are quite long and put into files. That is essentially source code, but in an extremely fuzzy way. Reading and understanding this is very hard, building a mental model of the "program flow" is next to impossible.

(This point alone warrants a blog post of its own.)

Vendor lock-in and unclear model update breakage

Prompts use natural language but they get interpreted differently by different AI services. This means you can't just take all your prompts and switch to another service -- you might get very different results. An additional problem here is that you don't know in advance how to avoid this breakage, because nothing is well-defined but it's all fuzzy natural language: This is vastly different from porting a program from, say, C to Rust. The only thing you can do is switch and then fiddle around until it mostly works.

This kind of breakage can also happen "unexpectedly" when staying with the same AI service but that service/model gets an update. (There appears to be a general expectation that "things will only get better", but this is not guaranteed at all.)

Nudge towards carelessness

The issue that double-checking AI output is a painstaking process that takes a long time and a lot of discipline. AI tools use human language and I've observed many times that people have a hard time keeping up their "guards"; the output looks like it was written by an expert, which makes it much more "believable". So, realistically, most people don't verify the output and they resort to "lgtm". We might even blindly trust that thing and call it "vibe coding". How's blindly trusting anything good?

The temptation to let everybody program

AI makes it easier to write code, so there's the temptation that you don't need qualified programmers to begin with. Accountants can write their own tools now, can't they? I think this is very dangerous because untrained personnel cannot properly assess the output of AI tools, nor do they have high-level knowledge of software architecture or best practices.

This is not meant as gatekeeping. If we want to enable more people to write software, we should teach them properly, actually give them the knowledge and the skills. And if it turns out that some pre-existing technologies or frameworks are just "nasty" and only AI makes them somewhat bearable for newcomers, then we should re-asses these technologies and see if we can improve them.

Conflicts with the EU Product Liability Directive

The (upcoming) EU Product Liability Directive demands more care and scrutiny. The two points above are in conflict with that.

The Generation Effect

The Generation Effect: If you let AI do all the work, you won't build as much knowledge as if you did it yourself. If you don't really dig through a problem but only let something or someone else figure it out and then you only read the result, this knowledge will vanish from your brain more quickly.

"Monkey Work" is meaningful

The issue that all work is meaningful, even the so-called "monkey work", like writing tests or documentation. These tasks are part of the thought process and the design process. More often than not, I realize that something's off while doing them. If you skip this because "the AI can do it", the quality of your output will have reduced.

Ignoring software licenses

The issue that I have published software as GPLv3 -- and then the crawlers came along, took that code, and now sell it as part of a humongous corpus, under ... which license exactly?

Don't tell me "this is like a human learning to program". It is not. We're talking about a massive commercial product here, not a human being.

It's also not a case of "this is for the betterment of humanity". AI services are not owned or controlled by The Public but, again, by private companies.

Crawlers

Aggressive crawlers are harmful. They force us to deploy workarounds like Anubis or other forms of throttling, which harm accessibility and/or availability. Frustrated operators might resort to blocking entire ASNs, which greatly threatens one of the core properties of the internet: Access for everybody.

Meta

Why isn't this a Git repo or a wiki?

This would open a huge can of worms. At the end of the day, it's my website and my content -- not a public place for discussion.

If you have feedback or suggestions, feel free to drop me a note. I'll take it under consideration but I can't promise to incorporate it.

Contributors

Credit where credit is due: The following people contributed ideas, links, ...