Generative AI for Business

Generative AI arrived in most companies a while ago, just not as a shared tool. Individuals write to chatbots on private accounts, one team has built itself a script, and nobody can say what it brings or where the data goes.

We roll out assistants and build custom applications on language models, with testing, cost control and a short usage policy. Searching company documents and agents that act in systems on their own are separate topics.

Assistant or custom application

Generative AI produces text, code or images from instructions in natural language. In companies this mostly means large language models (LLMs), and there are two ways to reach them. Either someone opens an assistant and works in it, or your own software calls a model through an interface (API).

The assistant fits what piles up in every office: email drafts, proposal text, summaries of meetings and email threads, translations with fixed terminology, research. Development teams use it for code, tests and documentation too. A custom application makes sense when the process is fixed and the result has to go straight into another system. Sorting incoming tickets by topic and urgency is an example. Nobody should have to open a chat for that; the result has to appear in the ticketing system. For general writing and research tasks, that would be too much effort.

Rolling out an assistant

The choice follows the work environment first. If you work in Microsoft 365, look at Microsoft Copilot; on Google Workspace, at the built-in Gemini assistant. With Microsoft it pays to read the licenses carefully, because the names are confusing. The app formerly called Microsoft 365 Copilot is now Microsoft Copilot, while the add-on license keeps the old name. Copilot Chat is available with suitable Microsoft 365 licenses and relies mostly on web data. With the add-on license, Copilot also reaches emails, files and meetings through Microsoft Graph, as far as the person has access.

That is exactly what makes the rollout tricky. Such an assistant shows everything a person can access, and so it exposes poor sharing settings. The folder that has been open to everyone for years suddenly shows up in answers. We clean up those permissions before the rollout.

On data protection, a lot depends on plan and configuration. According to Microsoft, prompts and responses are not used to train the foundation models, and the EU Data Boundary applies to EU customers. We check whether individual models or features are excluded before approval. OpenAI and Google also state that they do not use business data for training. With ChatGPT Enterprise, content can be stored in Europe for eligible workspaces. That does not mean every processing step and every integration stays in the region. We clarify this with your data protection officer before a tool is approved.

Licenses alone are not enough. Handing them to everyone without clarifying use cases means paying for tools that hardly anyone opens. A pilot with a small group and concrete tasks, backed by usage reports, works better. And where no approved tool exists, teams drift to private accounts. A good official offering is the most effective remedy.

What goes into a custom application

The model call is the smallest part; the work is around it. The prompt needs a clear task, a role, a format and examples, and it belongs in version control rather than in scattered chat windows. Instead of free text, the model should return answers in a fixed format, such as JSON following a schema, which several vendors validate themselves. That way results can go straight into other systems.

Guardrails act before and after the model call. Inputs are filtered, outputs are validated against rules, and when the model is unsure a person decides. Protection against prompt injection belongs here too, meaning hidden instructions in texts the model processes. Many models also read images, PDFs and audio, such as a photo of a delivery or a voice message. How reliably that works on your own documents only a test shows. And the integration needs thought: outages, retries and rate limits of the interface have to be handled before results land in the CRM or the data warehouse.

Which model

There is no best model, only one that fits the task. Proprietary models are often stronger on hard tasks and easy to reach through an API or the cloud providers’ platforms. Open models can be run and adapted in-house but need infrastructure and operations, and their license terms differ. Read them before use. Large models give better results on complex tasks, while small ones are faster and cheaper. For classifying or extracting, they are often enough.

Billing is usually by amount of text processed, with input and output priced separately. Long inputs and models that reason longer before answering cost more. Location matters too, whether at the vendor directly, in an EU region of a cloud platform or on your own infrastructure. It affects data protection and cost and is covered in the topic on GDPR-compliant AI.

Model names and rankings change within a few months. We therefore build applications so the model can be swapped, and decide based on our own tests rather than leaderboards.

How to tell whether the answers are good enough

Models phrase things convincingly even when the statement is wrong. Without measurement it stays unclear whether an application is good enough. The basis is a collection of typical inputs with expected results from the business teams. Where the result is clear-cut, such as categories, a script checks automatically. For free text, a second model scores the output, supplemented by spot checks from experts. With every new prompt and every new model version the test cases run again. Otherwise a drop in quality only shows up when users complain.

In operation we watch cost per application and per call, response time, error rate and corrections by users. A prototype with a few calls says little about the bill in production, because long documents and many users change it. Budgets and alerts per application prevent surprises. Content with external impact or legal consequences is still reviewed by a person.

Usage policy and training

A usage policy of one or two pages is enough. It states which tools are allowed, which data may be entered, who reviews results and who stays responsible for them. More important than length is that everyone understands it.

Training shows how to write good instructions, where the limits are and why results must be checked. Measures for AI literacy are also part of the EU AI Act. The details are in the topic on it.