Or why you may lose your time when you try using agents for doing significant breakthroughs
I hope this document ages as quickly as possible
I can't bear coding agents. They are fully brainwashed by corporate bullshit job output. They are awful yes-men incapable of pushing back. They are trained mostly against American society standards which I can't stand and find oppressive (if you disagree you are part of my problem).
But that's not today's topic.
What happens when something is unspecified or undefined.
That's today's topic, and my conclusions hurt. They hurt BAD. If you have a way around that, then I'm all ears. For now that's a major contributor to project failures.
What does unspecifed and undefined mean
Unspecified means one of these:
- Not written in any file.
- Not part of any prompt.
- Absent from context.
- Left free for interpretation.
- Outcomes and consequences not predicted (which does not mean unpredictable).
- Metaphor: they are plot holes (that can be exploited).
The two are VERY different animals. Both hurt when poorly used. And both are different from implicit, which is what you can deduce objectively from what already exists: implicit should always lead to the same outcome, while neither unspecified nor undefined will.
But here's the catch: Both are desirable characteristics in engineering.
Leaving something unspecified makes development more efficient. If you had to describe every single aspect of one thing to proceed, you would lose time. Leaving things unspecified is what makes specification hierarchy work: anything not specified should adhere to the more general principle stated in the more general document (keep that in a corner of your mind, it will be important later).
Undefined, on the other side, is explicit: you declare something undefined. It's a very powerful optimization tool: not forcing a specific outcome is part of what makes a product flexible, just leave undefined and see how it evolves and define it later if it has to be. Undefined behavior is also desirable when checks are expensive: you can't just check each and every tiny little thing.
What happens when you combine the 2?
Undefined characteristic may come from unspecified characteristic. In fact it's the default state of anything: it is fully unspecified (nothing is written yet) and fully undefined (no outcome is predicted).
Trap: you may think that anything unspecified is automatically undefined, but it's not: it may be defined by more general principles.
The inherently divergent nature of both aspects is one of the most difficult engineering problems to solve. Finding the correct balance to have something that works, does not diverge, and takes a finite amount of effort to describe, is one of the characteristics of good engineering.
How agents fail
Their training set is stronger than their input.
Everything below are consequences of the single observation above.
You can't force the agent to adhere to your solutions if they deviate from their builtin standard solutions, even if your solutions solve issues that you know will happen if you follow the standard. That's because of everything explained above.
If you leave things unspecified, agents will pad with their own default training set. They will fail at following any kind of specification hierarchy you try to build: they will choose the "standard" solution nearly all the time and you will lose your time fighting against it all the time.
For example, an AI will assume that you need correctness checks and validation on all your input: wrong ! If you say that in the system prompt it will be ignored most of the time because input validation has been forcibly drilled deep into AI bechavior. Each time you say "this should look like that" they WILL insert a validation step that you never asked for. No prompt can change that.
It's impossible to specify what you leave unspecified
Yes, that sentence doesn't make any sense to begin with, and that's the core of the issue, and what participates to making me think it's unsolvable.
The only way to hope changing AI behavior is by prompting. You can't just provide a generic prompt and expect the agent to apply it in all circumstances. Or if it works, it will follow that to an absurd extent, just like lawyers pushing word definitions to their narrowest extreme to try to escape issues.
Then just specify your needs
No. We use agents to save time, to code faster. If we need to specify everything by hand, then it takes more time than coding by hand, where everything is already in the developer's mind
What you say doesn't apply to teams, only self-imbued egocentric single-man projects
No. If it was, team building wouldn't exist. Team building (the true one, not the bullshit to make your company pay for karting sessions - btw I don't hate that) is where unspecified turns into implicit: the ultimate top-level specification that you don't have to write - a huge time saver. A good team shares values, solutions, vision, so that it minimizes the need for communication or nitpicking about details, everyone pushes toward the same direction.
You don't get it: you specify the needs and the AI is supposed to find solutions: parallelize agents
Earth does not say thank you. AI costs energy, a lot of it. AI ain't cheap. There is no free meal.
Apart from that: do you really think AI can efficiently find a solution that can both match all your current criteria and your future needs that you still can't state explicitly because they are still a faint far away light that you don't see clearly yet? What if the supermassive AI work solves all current issues in a way that makes your future work impossible to implement?
Then use a better model !
It just applies more standard solutions, and my projects fail even faster. Everything works better at the beginning, I keep adding small touches and refine, then the model starts applying workarounds silently because the standard general architecture is bad at handling that new use-case.
At least, dumb models fail fast, and that's a balance difficult to achieve. In fact I now crave for models that make less assumptions and that are able to follow user-specified design philosophies, rather than "smart" models able to solve problems by themselves. Docile and obedient models are less bad than smart but autonomous models.
What about undefined
It means you won't be ever able to leave purposely undefined behavior in order to leave opportunities for optimization. The AI will always interpret undefined behavior as something to fight against, and will strive to control everything.
Even worse: you can't trust the AI to leave holes on purpose. Ever. No way. It will default to fill in the gaps. Leveraging unspecified behavior with general direction asking to interpret it as undefined behavior will lead to the usual default solution padding, not leaving room for optimization or smart hacks. No way. The AI will never ever exploit undefined behavior as an opportunity to optimize or reduce things, it will exploit it as an opportunity to pad it with its own templates.
And the fact that AI is notoriously bad at following negative requests (do not...) does not help. Fast attention models (and some other modern optimizations) make this even worse, the model is getting even better at ignoring the parts of the context that complexify execution of the prompt.
I want AIs that can participate in team building
Because karting against an AI would be fun...
No, more seriously...
Is that what training is about? I don't know I never explored that way. But if training takes even more energy than coding myself, it's still not worth it. I would prefer AIs that nudge their behavior little by little just by feeding it input, and no commercial offering does that (and no, building a system prompt for you is NOT what I expect, I expect something much stronger than that)