Master Data Deduplication & Cleanser

The problem

Every long-running ERP accumulates the same rot. The same vendor exists three times — once as "ACME Industrial Ltd", once as "Acme Industrial", once as "ACME IND. LTD" — each with a different payment term, a different bank account, and a share of the spend. The same customer sits under two accounts because a salesperson could not find the first one, so the credit limit is meaningless and the aged debtor report understates exposure. The same physical item carries two item numbers, so on-hand looks short in one place and long in another, and MRP plans for a shortage that does not exist.

The damage compounds quietly. Spend analysis by vendor is wrong, so negotiations are based on a fraction of the real volume. Payments go to a stale bank account on the duplicate record. Credit exposure is understated across split customer accounts. Item duplication corrupts inventory valuation, forecasting, and every warehouse count. And the data quality problem is self-reinforcing: users cannot find the existing record, so they create another one.

Most teams know they have this. What stops them is that merging master data in F&O is genuinely dangerous — a customer account is referenced by sales orders, invoices, payments, ledger transactions, addresses, contacts, agreements, and a dozen module-specific tables. Merge it wrong and you break history.

What ECOSIRE builds

We build a duplicate detection, merge, and prevention engine as an X++ extension package for Dynamics 365 Finance and Supply Chain Management. No overlayering — extension classes, chain-of-command, table and form extensions, and new tables in your own model.

Fuzzy duplicate detection

Detection runs as a batch job across the master data tables in scope — customers, vendors, released products, and any additional entity you need. Matching is multi-attribute and weighted: normalised name comparison that strips legal suffixes, punctuation, and casing; phonetic matching for names entered by ear; tax registration and company registration numbers; bank account details; address normalisation and comparison; email and phone; and for items, product dimensions, manufacturer part number, and barcode.

Each candidate pair gets a composite score with a breakdown showing which attributes contributed. You set the thresholds: above one score the pair is a confirmed duplicate, between two scores it is a review candidate, below it is ignored. Detection is legal entity aware — you decide whether duplicates are hunted within a company, across companies within a data area group, or globally.

Safe merge with reference repointing

This is the part that has to be right. Before any merge, the engine builds a full reference map of every table and field pointing at the record being retired — sales and purchase orders, invoices, payments, settlements, ledger transactions, addresses and contacts through the global address book, agreements, trade agreements, item coverage, warehouse and inventory records, attachments, and any custom table in your model that carries the reference.

The merge itself runs as a single transaction with a survivor record and a retired record. Field-level survivorship is configurable — you decide, per field, whether the survivor keeps its own value, takes the other record's value, or requires a human decision. References are repointed, the retired record is blocked and flagged rather than deleted so history and audit remain intact, and a complete merge log records every table, field, and row that moved.

A dry-run mode produces the entire reference map and the full impact report without touching a single row, so the merge is reviewed before it is executed. Merges are also reversible for a configurable window using the merge log, provided no conflicting transactions have posted since.

Entry-time prevention

Cleaning without prevention just resets the clock. The layer hooks the customer, vendor, and released product creation and modification paths through chain-of-command, running the same matching engine at save time. A user creating a near-duplicate sees the potential matches before the record is committed, with the option to use the existing record instead. Depending on your policy and the user's security role, a high-confidence match can be a warning or a hard block.

The same validation is applied to records arriving through the data management framework and data entities, so an integration or a bulk import cannot inject duplicates that the UI would have caught.

Ongoing data quality

A data quality workspace reports open duplicate candidates, merge history, and quality scores by entity and legal entity. Data entities and OData expose the candidate, merge log, and scoring tables for Power BI, so data stewardship becomes a monitored process rather than a one-off project. Security is delivered as duties and privileges, so detection, review, and merge execution can be separated — a steward can propose, a data owner approves.

Who this is for

Organisations running F&O with master data that has grown through years of use, acquisitions, or multiple legacy migrations. It suits procurement teams that cannot get a true spend picture, finance teams whose credit exposure is split across duplicate customer accounts, and supply chain teams whose on-hand and forecasting are distorted by duplicated items. It is particularly relevant to multi-legal-entity groups where the same trading partner exists in several companies under different names.

It is a poor fit if your master data is small, recently migrated, and already governed by a strict creation process.

How delivery works

1. Scoping call. We look at your actual master data volumes and shape — which entities are in scope, how bad the duplication looks on a sample, which custom tables reference the master records, and what your merge governance needs to be. 2. Fixed quote. You receive a written scope covering entities, matching attributes, merge and survivorship model, prevention policy, and reporting, with a fixed price. 3. Build. We develop the X++ extension package against a sandbox on your F&O version, tuning the matching rules against a copy of your real data. 4. Install in test. The package goes through your LCS pipeline into your sandbox. We run detection over your full data set, review the candidate list with your data owners, and execute dry-run merges so you can see the exact reference impact before anything is merged for real. 5. Production. After sign-off, the same package is promoted through your LCS pipeline to production. Detection runs first; merges are executed under your governance, with a backup taken before any merge run. 6. Support. A support window from go-live covers defects, matching-rule tuning, and support during your first production merge waves.

Lead time is typically two to four weeks from signed scope, depending on how many entities are in scope and how many custom tables reference the master records.

What this is not

There is nothing pre-built to download and no trial. Matching rules that work for one organisation's vendor names produce nonsense for another's, and a merge engine that does not know your custom tables is a data-loss risk. Every build is scoped, tuned, and dry-run against your data before it merges anything.