Processing data with AI models: errors, retries, tracing and costs

AI data processing that works in a demo proves very little. In a demo the network is fast, the data is clean and nobody calls twice. In real work, however, everything else happens. One service is slow, another rejects requests, and an AI model returns text that ignores the format. Then the provider's bill arrives at the end of the month.

Here I describe how I keep AI model calls reliable in the tools I use to process data. I cover five topics: the data contract, errors, retries, tracing and costs.

AI data processing starts with the data contract

The first job is not technical. For each flow I write down where the data comes from, where it must end up and who decides when something breaks. Then I fix the format: required fields, allowed values, what to do with missing data.

Take a hypothetical example. A company receives customer requests from its website and wants them in the CRM, already sorted: I classify them with an AI model and load them into the CRM. There are three systems involved: the website, the model and the CRM. Without a written contract, though, each one makes guesses about the others. As a result, the first error turns into an argument about who is right.

Temporary errors and permanent errors

Not all errors are equal. Some go away on their own; others do not. Robust AI data processing tells them apart at once.

A network error or a service that is briefly down is temporary, so a retry makes sense. By contrast, invalid data is permanent. Retrying ten times just gives you ten identical errors.

Then there is a special case: rate limits. HTTP defines status code 429 "Too Many Requests". The response may include a Retry-After header, defined in RFC 9110, which says how long to wait. When the service sends it, my tools honour it. Otherwise you risk being blocked for longer.

Retries without duplicates

In AI data processing, retrying looks trivial. In fact it hides two traps.

Waiting the right way

The first trap is retrying at once, all together. If a hundred processes fail at the same moment and retry at the same moment, the service stays flooded. The known fix is to wait longer after each attempt, with a random part added. AWS explains it well in a post on exponential backoff and jitter. I also always set a maximum number of attempts. After the last one, the request goes to an error queue that a person can review.

Never doing the same action twice

The second trap, moreover, is worse. For example, a request goes out, the service runs it, but the response gets lost. My tool does not know that, so it retries. The outcome: two requests in the CRM, two emails, two records.

To prevent this, every write operation carries a unique key. The receiver remembers it and, if it sees the key again, does not repeat the action. For HTTP there is an IETF draft for the Idempotency-Key header that describes exactly this. Even when the external service does not support it, you can apply it on your own side of the flow.

AI data processing: the format is not guaranteed

In AI data processing, however, the model is not a service like the others. The same request can give different answers. Therefore every response is validated before use: format, fields, allowed values.

If the classification breaks the schema, the system does not guess. Instead, the request goes to a review queue, with the original text and the model's answer. That way a model error stays a case to look at, not a wrong record in the CRM.

Then there are versions. Specifically, providers update and retire models often. For instance, in the Gemini API release notes of 17 September 2026, Google announced a new version of its agent with renamed tools, and the shutdown of the old one on 5 October. So I pin the model version in the configuration of my tools. I change it only after rerunning the tests on real cases.

Tracing: every call must leave a trail

When a customer asks "what happened to my request?", well-traced AI data processing answers within minutes. In particular, that needs an ID that follows the request through every system: website, model, CRM.

For each call I log at least the time, duration, outcome, number of attempts and the ID. For model calls I add the model version and the tokens used. There is an open standard for this data too: the OpenTelemetry semantic conventions for generative AI. Using them saves you from inventing a format of your own. Moreover, you can switch monitoring tools without rewriting everything.

Costs: measure per call, not at month end

In AI data processing, models are billed by usage, in tokens. The cost therefore depends on how much text goes in and comes out. Two technical choices matter more than others.

The first is reusing the fixed parts of a request, such as instructions and examples. Many providers charge less when those are read back from a cache. For example, on the GPT-6 Sol price list, cached input costs 10% of normal input. On the price list of Claude Opus 5.5, announced on 22 September 2026, cache reads cost $0.20 per million tokens, against $4 for input.

The second choice is deferred processing. If a classification can wait a few hours, it often costs less. Again on the GPT-6 Sol price list, Batch processing costs half the standard rate.

Finally, I set a spending cap and an alert. After all, a faulty retry loop can multiply calls within hours. Better to learn that from an alert than from the invoice.

A checklist for AI data processing

Before I use a procedure on real data, I check six points:

• the data contract is written and each flow has an owner;
• temporary and permanent errors follow different paths;
• retries wait longer each time, add randomness and stop at a limit;
• every write carries a key that prevents duplicates;
• every call leaves a trail you can follow;
• spending and usage have a cap and an alert.

On the page about data conversion and transfer I describe how I approach this work, from the field map to the log of rejected records and corrections.

If data moves between your systems by hand today, or a data transfer stops without warning, write to me. We can look together at where the data gets lost and which fix makes the most sense.