AI has us quickly forgetting humans as the software delivery loop. Now, forget humans in the loop. For Thoughtworks’ distinguished engineer Kief Morris, we need to pay more attention to the human on the loop to keep defining what is good code and what are good systems.
“If you’re using AI to build software, you need to make sure that you’re building production-ready software, and I think the default approaches that people are taking right now don’t include” these engineering best practices, Morris told The New Stack during PlatformCon in London last month.
“Even if you’re not giving the agents free rein, you have to be building into your process [and] you have to be thinking about how to use agents to help you do that in a safe way, and that can’t wait; that’s not like future-pie-in-the-sky,” Morris said. “That’s something you have to do now.”
At the speed of AI, organizations have to focus a lot less on speed to deploy, and a lot more on the guardrails that ensure what you are deploying is safe, secure, and of high quality. Fortunately, that solution goes back to the software delivery best practices that have been accepted for more than a decade now. But only if your CI/CD pipeline is production-ready.
What is “good” anymore, anyway?
The question on the minds of many right now is how to get more out of their AI agents, but, as Morris explored that idea at PlatformCon in London, he noted that we need to step back from what the agents are doing to ask these questions first: “What does that mean for us? How do we keep controlling this?”
Morris is worried that we’ve delegated so much of the code building to AI agents that we’ve become distant from the details of what they’re building, which has us losing track of making sure the AI is building things well.
“That means we get better at defining good, and I don’t think we’ve been really good at this.”
“That means we get better at defining good, and I don’t think we’ve been really good at this,” Morris said in his talk. As he works out how to define that — and it will be different for every team’s and every company’s requirements — he has been “maximizing” or pushing agents to their limits, into an end-to-end, hands-off agentic workflow.
To achieve that, he believes that using continuous delivery pipelines within agentic workflows helps build better software. And we should take the CI/CD approach to agents, too.
“The idea of continuous delivery with DevOps is that when you find a bug in production, you don’t just fix the code in production; you fix it in the source, [and] you work out how they get through my pipeline, what do I need to change – my tests? – and whatever it may take to avoid this kind of mistake slipping through again,” Morris continued.
“It’s very similar when I’m working with agents that if an agent isn’t producing what I want, it’s not that I’m just going to correct it because the next time I use an agent to make a modification of my code, it’s potentially going to undo it.”
Then the next step is figuring out what you can put in place so that you don’t even have to look at the code at all. As code review has quickly become the new bottleneck, Morris offers this question that developers need to answer: “What do I need in order to feel comfortable that the code that is being built is good enough?”
“What do I need in order to feel comfortable that the code that is being built is good enough?”
For many organizations, they are hedging their bets that good enough for the large language model (LLM) is good enough in general. For him, that definition of “good” must always come back to what your users care about.
“I think we can learn a lot by trying to push these approaches to the max, even if it’s even a little bit further than we might be able to safely do in certain business circumstances,” he continues.
“LLMs don’t care about building software to last, they just want to build the next feature, so we need to figure out how to do it anyway.”
“LLMs don’t care about building software to last, they just want to build the next feature, so we need to figure out how to do it anyway,” or it’ll leave the wrong last impact.
The cognitive debt holding us back
The key problem is we often don’t know what we don’t know.
In AI-generated and even AI-deployed code, it’s answering, as Morris describes, “Even if I’m not looking at the code, how do I understand what’s happening, what it has built, and how is it put together, so it doesn’t run ahead of me and then you lose track of it, and it becomes harder to make changes to it and fix things, because you’re not in control anymore.”
Margaret-Anne Storey, co-author of the SPACE framework for developer productivity, recently released her Triple Debt Model, exploring the hidden human cost of AI-accelerated software development within three buckets:
- Technical debt, which sits in the code.
- Cognitive debt, which sits within the people on teams.
- Intent debt, which sits within the externalized knowledge, made up of the underlying rationale, goals, and design constraints that are insufficiently documented.
So how do we avoid this cognitive debt in order to not reach, as Morris calls it, cognitive surrender, while still not having to look at the code? That comes with finding ways to enforce quality conventions, including across citizen developers.
At least that answer has existed for 16 years now — in the CI/CD pipeline. That’s where, he says, you put humans on the loop managing how agents build software that lasts, in a way that closes that cognitive gap.
To achieve that, it comes down to the next layer of continuous delivery called harness engineering.
Harnessing code quality via harness engineering
In harness engineering, software builders — both technical and potentially non-technical — work on the harness around the code.
By Morris’s definition, the harness allows you to prompt an idea, then the AI builds, and then you iterate around the process. Teams work to create guides that tell the agents how to build, including architectural decision records that provide context over time.
In order to achieve that, he says, we must sense operability — the combination of software, plus infrastructure, plus processes — via the classic cross-functional and non-functional metrics:
- Lagging metrics – deliverability metrics including DORA metrics, user satisfaction, business operations metrics (transactions, cost), commercial metrics
- Leading metrics – deployability metrics including failure rates, performance and scalability testing, failure scenario testing, penetration testing, and compliance auditing
These all feed into making sure the codebase is always production-ready.
This isn’t the exciting agentic stuff, he admits, but how many teams are doing at least some of these essential leading metrics? By the time you realize dips in your lagging metrics, it could be too late. These leading indicators can be tedious, but this is also where AI agents usually excel in implementation – so long as the agents have clarity into what “good” looks like.
Then, once these software engineering best practices are finally in place, Morris proposes that your existing continuous delivery pipeline acts as an operability harness, with all the leading metric sensors built right in.
Now is the time to learn as much as we can about AI-driven software development, which, he argues, comes by going all-in. But you can only go all-in if you have the right harness in place, which can only happen if you build the system that builds the system.
“The resistance happens when you get into the specifics, like when I said in the [PlatformCon] presentation that maybe you shouldn’t even look at the code,” Morris says.
Group Created with Sketch.
0 Comments
Log in to join the conversation.No comments yet. Be the first to share your thoughts.