Your AI-built app is getting out of hand. Now what?

It started so well. Your team began building with AI tools, and suddenly everything went faster. A working prototype in days instead of weeks. A new feature before lunch. Management happy, team happy.

And then, slowly, something changed. A small fix breaks something completely unrelated. Nobody dares to touch the part that handles payments. A new developer joins, and after two weeks still can’t explain how things fit together. The speed that made everyone so enthusiastic is quietly turning into the thing that slows you down.

Sounds familiar? You’re not alone.

One of my customers built a system to help with their bookkeeping. It was set up so they could sell it as an online service to other companies. It started with a small proof of concept, which turned into the actual application. It was all built in a very short timeframe. And they got customers. Everybody happy, right? Well, yes, until the first performance complaints came in. But nobody really knew why, and nobody dared to touch the system, afraid that things would get worse. Development basically came to a halt. No new features, angry customers.
That application wasn’t even built with AI, but on a no-code platform. With AI-assisted development, the same thing happens, only faster. And I’ve seen similar situations way too often. (Read the full case.)

Five signs it’s getting out of hand

How do you know whether you’re dealing with a few teething problems or something bigger? These are the signs I look for:

1. Nobody can explain how it works. There’s code in production that no human really wrote, and no human really reviewed. It works, until it doesn’t, and then nobody knows where to start.
2. Small changes, big surprises. You change one thing, and something on the other side of the application breaks. The parts are connected in ways nobody designed.
3. The same logic, five times. AI solves the problem in front of it. It doesn’t remember that it solved almost the same problem last week, in a slightly different way. So you end up with several versions of the same calculation, each with its own small differences.
4. Tests are missing, or don’t test anything. Either there are none, or there are lots of AI-generated tests that happily pass whatever the code does.
5. Security and data surprises. Passwords or keys in the code. Input that’s never checked. Data ending up in places it shouldn’t. AI tools nobody approved, with company data pasted into them.

And a bonus sign, maybe the most telling one: the speed is going down, not up. Every new feature takes longer than the one before, because more and more time goes into fixing what broke along the way.

Why it happens

Here’s the thing: AI doesn’t create the mess. Speed without structure does. AI just makes it possible to build up that mess a lot faster than before.

The causes I see most:

• No architecture rules. Every prompt is a new design decision. AI makes local choices that make sense for the task at hand, and nobody checks whether they fit the whole.
• Prompts instead of specifications. What the software should do lives in a chat history that nobody else can see. Six months later, nobody knows why something was built the way it was.
• No review. Generated code goes into production because it works. Nobody reads it with a critical eye before it does.
• Pressure to deliver. The first results were so fast that the bar moved. Taking a day for structure now feels like falling behind.
• Builders without development experience. AI makes it possible for people without a software background to build working software. That’s great, until that software becomes business-critical without anyone noticing.

If this reminds you of my piece on citizen developers, that’s no coincidence. It’s the same story: the technology is rarely the problem. The missing structure around it is.

Full disclosure: I’m building this website this way

This website is built with AI-assisted development. And I’ve seen these risks up close, even with all the structure I could think of in place: a written specification, a design document, and a rule that every change goes through a reviewed pull request before it goes live.

Things still slipped through. Placeholder text from the design mockups ended up in the real templates; for a while, every article proudly announced a reading time of “[X] min read”, instead of mentioning the actual duration. Small inconsistencies crept in between the specification and the code. None of it was dramatic, and all of it was caught, because every change had to pass a review on a preview site before going live.

That’s the lesson I took from it: the structure didn’t prevent every mistake. It made sure the mistakes were found before visitors did.

What to do this week

The good news: you don’t need a big program to start. If you recognize the signs above, this is where I’d begin, this week:

1. Stop adding to the pile. You don’t have to freeze all development. But from today, nothing new goes into production without someone reviewing it, someone who understands what it does.
2. Make an inventory. What’s there, which parts are critical (money, customer data, anything customers see), and who knows what about each part. You’ll probably find a few parts nobody really owns. Those are your biggest risks.
3. Start writing specifications. Write them down, so you can check what has been developed against the specs, and keep track of what you’re building.
4. Put a safety net around the critical parts. Before you change anything in the most important flows, add tests that capture how they work today. Then you’ll know when you break something.
5. Write down the rules. A short document with your architecture choices, coding conventions, and which AI tools and models are approved. Most AI coding tools can read such a file and follow it, so the rules work for people and AI alike.
6. Check where your data goes. Which AI tools are in use, and what company or customer data has been pasted into them? This one is quick to check and can be expensive to ignore.

For the customer I mentioned before, this is exactly what we did. We took a step back and made an inventory of what was important and who owned what. We started writing specs and checking them against what was already built, so we knew what to work on.
And, critical for this particular project, we started on an architecture and all the rules that would need to be applied.
All of this was done so we could move on to the next step: getting development back under control.

Getting back in control, without stopping delivery

The first week stops things from getting worse. Getting back in control takes longer, but it doesn’t mean stopping everything. What works:

• Specification first, prompt second. Write down what you want before you ask AI to build it. A few sentences are often enough. It forces you to think, and it leaves a record the next person can read.
• Small changes. Small changes are easy to review, easy to test and easy to roll back. One huge AI-generated change is none of those.
• Leave it better than you found it. Every time someone touches a part of the code, they clean up a little: remove a duplicate, add a test, clarify a name. No big refactoring project, just steady improvement while delivering.
• Tests are part of done. AI can write tests, and it’s good at it. But a human checks what those tests actually test.
• Governance, the light version. Approved tools and models, clear rules about where data may go, and clear ownership of each part of the application. Who is accountable when something goes wrong? Decide that before it does.

The customer I mentioned got back in control. The crucial thing we learned: there was no architecture specified. The application started as a proof of concept, to which functionality was added. The lack of architecture exposed severe issues with data handling, which made the system too slow to work with.
So we put an architecture in place, and step by step we checked the specs against what was already built. We either updated existing features or added new ones, fully controlled and based on the right architecture.
Result: the performance issues were fixed in a very short timeframe, and continuous development was now fully under control, also because governance was set up to guide the team.

What not to do

Three reactions I understand, but wouldn’t recommend:

• Starting over from scratch. Tempting, and almost always wrong. Without new structure, you’ll rebuild the same problems, just faster.
• Banning AI tools. You’ll lose the speed, and people will use the tools anyway, just without anyone knowing. That’s shadow IT all over again.
• Blaming the team. This is a structure problem, not a people problem. The team did what everyone was cheering for: delivering fast.

Keep the speed, add the structure

AI-assisted development isn’t the problem. Building without structure is. The teams that get the most out of AI aren’t the ones that generate the most code; they’re the ones that can still explain, test and change that code six months later.

So keep the speed. Just add the structure that lets you keep it.