Anatomy of a data crisis: 27 systems, one free-text field.

A European fruit and vegetable exporter let us map its data landscape before a planned ERP switch. What we found is worth sharing - because the same architecture, with the same failure points, sits inside most mid-sized industrial companies today.

The situation

An ambitious goal, sitting on a fragile foundation

The company is a successful exporter of fresh produce, with hundreds of millions in revenue and a network of partner cooperatives. Leadership had set a clear target - reliable, real-time data steering the business by 2030 - and a new ERP was planned as part of the journey.

But before changing any system, the more useful question was: what does the landscape actually look like today? The answer: around 27 separate applications, held together by an ever-growing layer of Excel. And at the centre of it, doing a job it was never designed for, a PLM system acting as the defacto product-data hub - fed and drained by hand-built ETL pipelines.

And wherever pipelines didn’t reach, Excel did - most critically at the company’s edge, where partner cooperatives delivered their data as manually prepared spreadsheets, re-keyed and error-prone. The supply side of a multi-hundred-million-euro business entered its systems through attachments.

Failure point #1

The article master: a database inside a text field

The single most expensive problem in the whole landscape was also the least glamorous one: the product reference data. There was no structured article code. Everything that defines a product - variety, calibre, category, origin, packaging - was concatenated into a single free-text designation field, typed by humans, parsed by other humans.

The consequences cascade through everything downstream. You cannot compute margin per variety if “variety” doesn’t exist as a field. You cannot match a cooperative’s delivery line against your own stock if the article strings don’t match. You cannot deduplicate what you cannot parse. One unstructured field at the bottom of the stack quietly invalidates every analysis at the top of it.

One free-text field at the bottom of the stack invalidates every analysis at the top of it.

Failure point #2

A PLM doing a data hub’s job, with ETL as the glue

Because there was no real data backbone, the PLM system had drifted into that role: it held the product data, and hand-built ETL pipelines shuttled copies of it to whichever system needed it next. That architecture fails in three predictable ways:

• Every copy starts ageing the moment it’s made. ETL runs in batches; between runs, systems work on yesterday’s data. Discrepancies between systems aren’t an accident here - they’re built into the design.
• Every new system means new pipelines. Each application connects point-to-point to several others - the classic n×m problem. With ~27 applications, the pipeline count and the maintenance bill grow faster than the landscape itself.
• The hub becomes unreplaceable. The PLM can’t be touched without breaking the pipelines hanging off it - the very lock-in companies fear from a platform, created accidentally by not having one.

What it costs

The bill arrives in stock, in margins, in retirements

None of this stays an IT problem. It surfaces as business numbers:

• Stock management ran on manual processes - and the company itself estimated the resulting stock-outs and inefficient logistics at around €3 million per year.
• Leadership couldn’t get reliable KPIs - margin per sale and stock levels couldn’t be produced consistently across systems, so strategic decisions were made on intuition.
• Critical knowledge sat in a few heads. The undocumented logic that made the landscape work at all - which string means which article, which file goes where - was held by a handful of long-serving employees approaching retirement.

The problem in one image
Ask three departments for the calibre of the same apple and you can get three different answers - procurement grades it to buy, production sorts it to run the line, sales describes it to sell. None of them is wrong; each measures for its own purpose, in its own system. The apple never changed. The shared definition never existed.

The instinct, and why it fails

“The new ERP will clean this up”

With an ERP switch already planned, the tempting conclusion is that the migration will fix the data along the way. It won’t - and this landscape shows exactly why. The new ERP would inherit the same unparseable article strings, the same unmatched cooperative data, the same orphaned ETL logic. The migration gets *harder*, because mapping 27 systems’ worth of inconsistencies onto a new core is precisely the work nobody has done yet. And afterwards, the company’s data model belongs to the new vendor.

The right sequence is the reverse. First, build the data foundation - above the systems, owned by the business:

• Start with the article master. Define one structured article code with explicit attributes - variety, calibre, category, origin - and a governance workflow for creating and changing articles. This is deliberately the first move: it’s the root from which every other figure derives.
• Replace the accidental hub. One integration layer takes over the role the PLM and the ETL pipelines were straining to play: every system connects once, and data is linked at its source, not copied - so nothing ages between batch runs, and discrepancies stop being built in.
• Bring the cooperatives inside the model. Partner data maps into the same structured definitions on arrival, instead of being re-keyed from spreadsheets.
• Encode the tribal knowledge. The mapping logic living in people’s heads becomes part of the model - and survives the retirements.
• Then migrate the ERP - calmly. The new system plugs into a foundation that already holds the truth. The business processes on top don’t move. And the next migration after this one gets cheaper, not harder.

A new ERP inherits your data problems. A data foundation makes the ERP replaceable.

The lesson travels

Check your own landscape for these four signs

The details here come from fresh produce, but the architecture is everywhere - in chemicals, pharma, machinery, energy. The same four signs are worth looking for in your own landscape: a critical attribute living inside a free-text field; a system acting as data hub that was never designed to be one; Excel carrying data across an organisational boundary; and a planned migration expected to “fix the data on the way”. Any one of them is a warning. All four together are this article.

Fixing them is not a tool purchase, and it doesn’t have to wait for the next big programme. It starts with one structured definition of one critical object - and a foundation above your systems that makes every later change cheaper. That foundation is exactly what the boTec DataBridge is built to be.