In 2024, I attended ICST in Toronto. I went to attend the first "Artificial Intelligence in Software Testing" workshop (AIST), where I gave a tutorial on a potential competition and participated in a panel (which, turned into a roundtable discussion) about the field. Our concerns at the time were principally about LLMs with two prominent questions: "how can we research this reliably?" and "how much can we reasonably expect to come from this?"

Readers who have visited before will know that I'm not the greatest fan of LLMs, to say the least. The many arguments against it are weary of needing to be repeated so often, and I am weary of repeating them. Back in 2024, the scale of these harms was not yet well-understood, and so the panel itself was mostly concerned with the fact that we could not realistically do research on LLMs themselves, but only the viability of their applications. No empirical result about efficacy would be generalisable, and anything could be denied (in favour of or against LLMs) by simply claiming that the research was studying the wrong targets or applying them in the wrong way.

Ironically, though I had attended ICST specifically to attend AIST, it was not the most impactful part of the conference for me. That title belongs to the second keynote from the main conference track, "We build the world we measure" by Mike Hoye. The keynote, ironically, was mostly not about measurement, but its consequences. The speaker derided the industry for its utter disregard for the well-being of its customers, and the lack of accountability faced by companies that develop software. Ultimately, he called for accountability in software engineering, akin to the measures put into place by other engineering fields, all justified by a simple fact: software is simply too integral to our daily lives, and too consequential.

I barely remember the workshop I had joined for anymore, but I remember where I sat during the keynote, specific slides, the shape of the room, the rapt attention and silence of the audience, and the brief conversation I had with the speaker afterward. I also remember the two Google engineers (one senior, one junior) who joined the conversation -- rather forcefully -- to assert that accountability is toxic, and counter to the freedom of software development. I remember the smugness with which he said this, and the certainty with which he claimed this, and his inability to justify that claim beyond repeating the claim itself or excuses already accounted for in the keynote.

At the time, I just found this infuriating. A quintessential self-report on avoiding responsibility for the software one develops, knowing that people depend on it. Now, I remember it in the context of LLMs.

Our industry is so poisoned by the idea that consent is optional; that responsibility is something to be dismissed in a limitation of warranty; that negligence is excusable in the name of progress; that features are more valuable than stability; that usability is secondary to commercialibility; that somehow, there is no monetary gain too small to justify an indignity to users. The idea that companies are somehow not responsible for the software they create and distribute is pervasive and normalised, while people's lives revolve around programs working as anticipated often without any alternatives.

I'm coming back to the ideas of the keynote more often these days. LLMs have sponsored a radical shift in how programs are developed for many -- mostly for worse. Now, the very same companies which have, for decades now, shirked responsibility for the consequences of their programs, are using these tools to increase the output of code at the cost of maintainability, understanding, and accountability in the software they distribute.

Effectively any programmer can produce code with an LLM that accomplishes a task that can be commercialised. I see our field shifting from one that already has problems with accountability to one where there simply is none. I see a world where individual programmers, who maybe historically had enough stake in their work to see it done well, become disconnected from the code they are responsible for, assigned enormous workloads which can only be accomplished by simply caring less and relying on the code generator more. Where those same programmers are "held accountable" by being replaced when something fails; after all, since nearly any programmer can operate the tools that actually produce the code, we become more expendable, more exchangeable. To me, and certainly to those who truly benefit from the commercialisation of software, this developer sounds like perfect scapegoat to be held accountable for the harms without consequences for the system or the people that actually cause those harms.

When something causes harm in the traditional engineering domains, the company faces severe penalties, and a proper investigation is done to determine the cause of the failure. It is rarely because of an individual's failings alone, but because of systematic failures that induce it. When something causes harm in software engineering, the company abdicates responsibility entirely, rarely faces any consequences, and, if anything ends up happening to hold anyone "accountable", it is most often implemented by simply dismissing the programmers involved. And now, we are entering a paradigm where developers are both encouraged and expected not to feel responsible for their work; to defer responsibility as much as possible to a tool. And maybe someday, we enter a paradigm where product designers simply generate code directly, without the involvement of a programmer at all; where no one feels any personal responsibility and no one is individually accountable for any failure. It simply becomes "the way that it is"; software is simply expected to fail, even when it is critical for the user. Admittedly, for many users, it already feels like we're there. The only difference is that maybe, at one point, that company employed someone who took any amount of personal pride or felt responsible for that code working as anticipated.

Companies which create and distribute software that people depend on must be held to account. For a long time, they've needed to be held to account. Software remained somewhat usable because of the programmers at these companies striving for actual quality, but dysfunction is, and has been for several years now, the norm. Users are suffering now from negligent practices, and the degree to which software is degrading and will degrade in the future is nearly unimaginable. And companies simply shrug.

Without real accountability, I can only imagine it getting a lot worse.