Computer Vision

Computer vision analyzes images and video automatically. That ranges from quality inspection in production, counting objects and taking stock in the warehouse to checking packaging and labels or assessing the condition of equipment from photos. What used to require expensive special-purpose systems can now often be done with deep learning and standard cameras.

A project pays off where visual checks depend on whoever is inspecting, defects only show up at the customer, inventories are counted by hand, or photos from the field service and from building sites are collected but never analyzed systematically. We take such projects from the first feasibility check to operation and pay particular attention to the conditions on site: changing light, dirt, vibration and rare defect images.

What is the task, really?

Computer vision is an umbrella term, and the task determines the method. In classification, the model assigns a whole image to a class, such as “OK” or “reject”. Object detection finds objects and marks them with a box, which makes counts, completeness checks and stock-taking possible. Segmentation outlines objects or defects pixel by pixel and is needed when the size, shape or area of a defect has to be measured. Anomaly detection learns what a flawless part looks like and reports deviations, which helps when there are hardly any images of defects. Add to that tracking objects across several video frames, on a conveyor belt for example, and reading serial numbers, batches and labels directly in the image. Whole documents have a focus area of their own, AI Document Processing.

Before we build anything, we agree how many missed defects and how many false alarms are acceptable. Without that figure, you cannot judge later whether the system is good enough.

Existing model or your own?

For many tasks a model does not have to start from zero. There are freely available models pretrained on large image collections that can be adapted with relatively few images of your own. For object detection the licenses differ considerably. The Ultralytics YOLO models are released under AGPL-3.0 or a commercial license, while smaller RF-DETR variants are under Apache 2.0. That should be checked before use, and the same goes for datasets. Foundation models provide general image features on which custom classifiers can be built from a few examples.

Multimodal language models answer questions about images and describe content without any training of your own. For prototypes and rare analyses that is a good route. For fast, repeatable inspection at line speed they are often too slow and too expensive. We usually need a specially trained model when the task is very specific, such as fine surface defects on a particular material, when the inspection has to run in real time on small hardware, or when the required accuracy is not reached with existing models. So we first test with images from the real application what an existing model can do, and then decide based on the measurements.

The image matters more than the model

The quality of the image data shapes the result more than the choice of architecture. Even before the first training run we settle how the images are produced: camera position, resolution, exposure, background, triggering. Anyone who changes that later usually has to retrain. Constant light and a fixed camera position often improve results more than any model tuning. And training images from the lab do not help if the model has to work on the shop floor. They must come from the real environment.

When annotating, meaning marking objects and defects, the annotators need clear rules. At what point is a scratch a defect, where does the object end? Without shared rules the model learns the annotators’ disagreement. For this we use annotation tools that can also be self-hosted. Models such as Meta’s Segment Anything can pre-annotate, so that specialists only correct the suggestions. Newer versions also find objects from a text description and work on images and video alike.

Edge cases need particular attention. Rare defects are barely represented in the data but are often exactly the ones you want to find. Daylight, shift changes, a dirty lens and new batches change the picture. And if images of the same part appear in both training and test, the model looks better than it is. So we split by part, batch or time period.

At the machine or in the cloud

In the cloud, models are easy to update and can be scaled when needed. That suits analysis without time pressure, such as photos from the field service. Directly at the machine, at the edge of the network, the model runs when the answer has to be there within milliseconds, the connection is not reliable or images must not leave the site.

For edge use there are special machines with a graphics accelerator, also in industrial form factors. Models are often exported to the open ONNX format for this and accelerated with a runtime. That keeps the choice of hardware open, because the model can run on other hardware without retraining. With many sites, it also has to be settled how updates are rolled out and devices are monitored.

When something changes in operation

The quality of an image model can drop even though nothing about the model was changed. New products, a different supplier, a rebuilt hall or a nudged camera are enough. So we monitor the input images (brightness, sharpness, framing), the distribution of the results and how confident the model is in its decisions. Uncertain cases go to a person for review, and their decision flows back as a new training example. How versioning, retraining and monitoring can be set up in general is covered under MLOps.

If people are recognizable in the images, data protection questions come up. We clarify them early with the data protection team and avoid recording people wherever we can. More on this under GDPR-Compliant AI.