AI Proof of Concept

An AI idea often sounds plausible and is quickly shown in a demo. Whether it works with your own data, in your own workflow and at a reasonable cost only becomes clear under realistic conditions. A proof of concept, or PoC, answers that before bigger investments follow.

We define in advance how success is measured, test the idea on real data, compare several approaches and estimate the cost of running it in production. At the end there is a recommendation to decide on. A PoC presupposes a concrete idea. If you first need to find out which ideas are worth pursuing at all, start with AI consulting.

Start with a question that has a yes-or-no answer

The starting point is a question that can be answered with yes or no, for example: “Can a model assign incoming requests to the right team with sufficient accuracy?” With it come success criteria that are measurable and written down before the start.

Quality is measured first, depending on the task as the share of correct assignments, the share of correct answers with the right source, or the deviation of a forecast. That needs a baseline from today’s process, measured by the same yardstick, because people make mistakes too. Then comes impact, such as handling time per case or the share of cases completed automatically, and finally minimum values at which the solution is usable, along with values at which the initiative ends. If criteria are set only after the test, any result can be interpreted to fit. So we agree on them beforehand with the business team.

A PoC usually runs on a slice of the data, without connection to production systems and with a small team. It clarifies whether the idea works in principle. Whether people will want to work with it every day is something only a pilot shows.

Data: not just the cleanest examples

The data determines how meaningful a PoC is. We choose a sample that reflects daily reality, difficult cases included. It is cleaned: duplicate, outdated or contradictory entries are spotted, documents are made readable, fields are standardized. Where a correct answer exists, experts record it so that results can be checked.

Test data stays strictly separate from the data used to tune prompts or models. Otherwise the result looks better than it is. If the data contains personal data, we clarify hosting, data processing agreements and anonymization with your data protection officer beforehand.

Comparing approaches and measuring

For one task there are usually several routes: a ready-made product or a feature of the software you already use, a language model with prompts and, where needed, a search over documents, or a custom model. The ready-made product is often the quickest route but less adaptable. The language model suits texts, documents and dialogs. A custom model, whether classic machine learning or a tuned language model, comes into question for forecasts, specialized data or high volumes, but needs more data and upkeep.

We build the candidate variants lean. The simplest approach is the baseline, and a more elaborate one is worth it only if it is measurably better. We also test different language models against each other, because leaderboards say little about your own task.

The core of the testing is a collection of test cases with expected results, built together with the business team. It contains typical cases, edge cases and cases where the solution is explicitly meant not to answer or to hand over to a person. Where the result is clear-cut, such as categories or numbers, checking is automatic. For free text, experts score samples, supplemented by a second model as a judge. A single overall rate is not enough. We look at which errors occur and how serious they are, because a misfiled newsletter is judged differently from a wrong answer about a contract. Whenever the model or prompt changes, the test cases run again so that comparisons stay fair.

What is on the table at the end

A PoC with a few calls says nothing yet about ongoing operation. We therefore extrapolate the measured values to realistic volumes. For cost per case, the amount of text processed, the choice of model and the number of steps count, and models that reason longer before answering use more. Operating effort includes monitoring, upkeep of prompts and data, adapting to new model versions and support, along with interfaces to systems, permissions, logging and failure behavior.

From this comes the recommendation with results against the criteria, a cost estimate, the risks and a proposal. Continuing means a plan for pilot and operation. Adjusting means a different approach, better data or a narrower scope. Stopping is right when the criteria are clearly missed, and even then test cases and findings remain useful.

That the road to production is not a given is shown by the numbers. Gartner predicted in 2024 that at least 30 percent of generative AI projects would be abandoned after the proof of concept by the end of 2025. Reasons cited included data quality, inadequate risk controls, escalating costs and unclear business value. According to a Gartner survey of CIOs published in October 2026, the more successful companies are considerably more likely to define in advance when a pilot should not go to production.

In our view a lot hinges on three things: criteria fixed before the test, data that reflects daily reality, and someone in the business who owns the result and also accepts a stop. What runs without restrictions in a test needs permissions and logging in operation. We check that early.