Designing AI features people can still answer for

An AI feature fails in a quiet way. The demo works. A month later nobody can say why the product gave a particular answer, or who is allowed to overrule it. The model did not make the application unmaintainable. The missing boundary did.

Insight Apps builds applications that people have to keep. AI belongs in that kind of product only when a person can still answer for the outcome. This is a design problem before it is a model problem.

Put a fence around the feature

Name the job the model is allowed to do, in one sentence a colleague outside the team can repeat. Summarise a thread. Draft a reply. Classify an incoming request. Suggest the three fields that are probably wrong. If the sentence needs a comma-separated list of ambitions, the feature is not ready.

The fence also says what the model must not do. It does not move money. It does not delete a record. It does not tell a customer the decision is final. Those actions stay behind an ordinary, tested path in the application, the same path a human would use. The model can propose. The application commits.

The model can propose. The application commits. A person remains able to say why.

That split keeps the rest of the codebase ordinary. You can review it, log it, and reverse it without learning the internals of a provider.

Decide what “good” means before you prompt

Teams often start with a prompt and a handful of flattering examples. The examples that matter are the boring ones from the real workflow: the incomplete form, the angry customer, the record that looks like the last one but is not. Write twenty of them down. Include the ones the feature should refuse.

For each example, write the answer a careful colleague would accept. That set is the evaluation. When you change a prompt, a model, or a retrieval source, you run the set again. You do not argue from a single impressive screenshot. A feature you cannot re-check is a feature you will be afraid to change, which is the opposite of maintainable.

Keep the set in the repository, next to the code. It is a test, even if it is not a unit test. Treat a drop in the set the way you would treat a failing build.

Leave a person in a position to answer

Someone has to be able to explain a result to a customer, an auditor, or their own manager. That person needs three things on the screen, not in a log they cannot open.

They need the input the model actually saw, not a reconstructed guess. They need the output, unchanged. They need a way to accept, edit, or discard it that is as obvious as any other control in the product. If editing the suggestion is harder than retyping the work, people will either trust it blindly or ignore it. Both outcomes waste the feature.

On mobile, this is mostly a layout problem. The suggestion cannot be a modal that hides the record. The record stays visible. The suggestion sits beside it, or under it, and the action is a normal button.

Log enough to learn, and then stop

You will want to know which suggestions were accepted. That is a product metric, and it is fair to store it. You do not need a copy of every customer document “just in case the model gets better.” Retention is a product decision. Write it down: what is stored, for how long, who can read it, and how a person asks for it to be removed.

If the provider trains on customer content by default, turn that off or do not send the content. Say so in the same note as the rest of the data handling. Compliance here is not a paragraph in a sales deck. It is the list of bytes that leave your system. The same habit is what we describe in infrastructure that stays compliant.

Make the model replaceable

Providers change prices, quality, and terms. A feature that can only speak to one vendor’s SDK, in twelve call sites, will outlive the reason you picked that vendor. Hide the call behind a small interface in your application: input, output, and the evaluation set. The prompt can live beside it. The rest of the product should not know the vendor’s name.

This is the same rule as any other technology choice. Prefer the boring path at the boundary, and read how to choose technology you are allowed to leave if the team is about to weld a provider into the domain model.

Ship it as a feature, not as a personality

The interface should say what the feature does. “Draft a reply from this thread” is a feature. A chat box that implies the application now understands the whole business is a promise you cannot keep. People forgive a narrow tool that is wrong sometimes. They do not forgive a product that sounded certain.

When the model has nothing useful to say, the feature should say so and get out of the way. A blank suggestion with an honest reason is more maintainable than a fluent paragraph that invents a policy.

If you are adding AI to an application you expect to keep, start with the fence, the examples, and the person who has to stand behind the result. The model is the last decision, and it should be an easy one to change. If you want a second pair of eyes on that shape, write to Insight Apps.