0
← Journal index

AI / Wanderings 2026

How to Build an AI Workflow That Improves With Use

Choose a measurable outcome, collect useful feedback and make changes you can check.

By Martin Uetz4 min read
A circular mechanical feedback system moves markers through action, observation and adjustment.

An AI workflow can produce a convincing answer and still leave you uncertain whether it helped. To improve it, you need to connect what it does to what happens next.

I use this pattern in my work at humAIne: the system takes an action, we observe the result, and we adjust the process. The feedback matters because it gives us something better than an impression to work with.

A loop has three parts:

  1. Action: the system does something.
  2. Signal: you collect evidence about how it went.
  3. Adjustment: you change the process in response.

The adjustment might be a better prompt, a corrected source, a different routing rule or a decision to involve a person earlier. It can be reviewed and applied by a human. You do not need a model that retrains itself to build a useful feedback loop.

Start with an outcome you can check

“Make our emails better” leaves too much room for interpretation. Replies and useful conversations tell you more about an outreach process than whether the text sounds polished.

For customer support, a quick answer is useful only if it helps resolve the problem. Track whether the customer needs to come back and whether a person had to correct the response.

Choose one outcome for the workflow, then add the checks that protect against improving the number at somebody else’s expense. A support system that closes tickets quickly while leaving customers stuck is doing the wrong job.

Write down the starting position before you change anything. Otherwise, you will struggle to tell whether the new process helped.

Collect feedback that explains the result

A thumbs-up is convenient. A correction that says the answer used an outdated policy is more useful because you can act on it.

Keep enough context to understand a failure: the task, the relevant inputs, the output, the correction and the eventual outcome. Protect sensitive information and retain only what the process needs.

Review a sample of successful cases too. Silence does not necessarily mean the work was correct, and people do not report every mistake.

Three places to start

Meeting preparation

I use AI for research and preparation. After a meeting, the useful question is whether the brief helped with the conversation.

Record what was missing, what was outdated and what took up space without helping. Use those observations to change the sources or the brief’s structure before the next meeting.

If the system repeatedly misses an important fact, investigate why. It may lack access to the source. Rewording the prompt will not solve an access problem.

Content repurposing

We use AI to repurpose content at humAIne. A post can generate several shorter pieces, but producing more versions is only the beginning.

Compare which pieces bring relevant readers to the original article and which lead to useful replies or subscriptions. Look at enough examples to avoid treating one unusually successful post as a reliable pattern.

Then change one element, such as the opening or the format, and watch what happens. Keep the original argument intact. A more attention-grabbing version is a poor improvement if it misrepresents the article.

Customer-support triage

In a support workflow, AI can suggest where a ticket belongs. The feedback comes from what happens after that suggestion: was the ticket rerouted, did it reach the right person, and was the problem resolved?

Review recurring mistakes and update the categories, examples or escalation rules. Some cases should continue to go straight to a person. The loop should help you identify those boundaries as well as improve routine routing.

Change the process deliberately

Someone needs to own the review and the resulting changes. A dashboard on its own will not improve the workflow.

Start with a regular review that fits the volume of work. Examine failures, propose one change and check it against examples you have kept aside. If the change makes other cases worse, revise it or reverse it.

Keep a record of what changed and why. When possible, compare the revised process with the previous version on similar work. Without that comparison, changes in the task mix or the underlying model can be mistaken for progress you caused.

As you gain evidence, some low-risk adjustments may be automated. Changes affecting customers, commitments or sensitive decisions need appropriate limits and review.

Expect uneven improvement

Feedback loops can help a workflow improve, but gains are not guaranteed to compound at a fixed rate. Easy fixes run out. The input changes. A measure that was useful becomes misleading.

When progress stalls, check the signal and the task before changing the model. You may have reached a limit of the process, or you may be measuring something that no longer reflects the outcome you want.

Pick one recurring workflow this week. Define the outcome, capture its current performance and decide who will review the results. Make the first change small enough that you can tell whether it helped.