Retries handle failures the system noticed. Dead letter queues handle failures that exhausted their retries. Neither handles the failure where nothing happened at all.
A webhook that was never delivered. A job accepted by a provider and then stalled. An event dropped during a deployment window. A scheduled export that did not run. In each case there is no error on your side, because on your side nothing occurred.
Reconciliation is the only layer that catches those, and it is the layer most often left out.
What to compare
Pick the things that must agree, and compare them directly rather than through a derived number.
Counts. Paid orders against fulfilment jobs. Records in the source against records in the destination for a period. A count mismatch is the fastest signal available.
Totals. Processor payouts against the sum of recorded transactions and fees. Stock ledger balance against a physical count for a sample of lines.
Specific records. For a recent window, list identifiers on both sides and report what is present in one and absent in the other. This is the version that tells you which order to go and look at, which is what an operator actually needs.
Staleness. Records that entered a state and have not moved within an expected time. An order submitted to fulfilment three days ago that has never reported production is a stalled job, and no event will ever tell you.
Report, do not resolve
The temptation is to have reconciliation fix what it finds. Resist it until the pattern is understood.
An automatic resolution for a discrepancy nobody has diagnosed is a good way to make a small problem symmetrical across two systems. The first version reports, a person investigates, and once a specific class of difference is understood and has a known safe fix, that class can be automated with the action logged.
Where the output goes
A report nobody reads is not a control.
Put it somewhere with the operational queues, showing the differences with enough context to act on: the record, the systems, the nature of the mismatch and a link to each side. Alert when the count of differences exceeds a threshold, or when a difference has persisted across runs.
Zero differences should also be visible. A reconciliation job that silently stopped running looks exactly like one that is finding nothing.
Frequency
Match it to the cost of a delay.
Choose a cadence from the business impact, available reports and source update frequency. A time sensitive fulfilment queue may need a different schedule from master data. Record who reviews discrepancies and when a stale comparison itself should raise an alert.
The window matters as much as the frequency. Comparing the last seven days every day gives you overlap, which catches something that arrives late and would have fallen between two non overlapping windows.
The discrepancies you will find
In practice the same causes recur.
A provider changed a field and the mapping silently produced a default. A permission changed and one direction of a sync started failing authentication. A certificate expired. A deployment restarted a worker mid batch. A record type was added at one end that the other end does not know about. A time zone difference put a day boundary in the wrong place.
None of these announce themselves. All of them show up in a comparison.
Build it at the same time as the integration
Reconciliation is usually deferred and then written after the first incident, which is the most expensive possible order.
Building it alongside the integration lets the team test the comparison before an incident depends on it. It also gives a concrete answer to how operators would notice that the integration had stopped working.
Compare a consistent business window
Two correct systems can disagree temporarily because they were observed at different times. If the source accepts another order while the destination report is being exported, a raw count difference may be normal delay rather than loss. Record the cutoff, report generation time and source cursor with each reconciliation run.
Use an agreed grace period for records still in transit, and show them separately from overdue mismatches. Do not hide them completely: a record that remains in transit across several runs needs attention. Where the source offers a stable snapshot or a supported change cursor, use that instead of treating a moving list as a fixed dataset.
Counts alone are insufficient. Two missing records and two duplicates can cancel each other numerically. Compare stable identifiers and material fields, then present a summary for navigation. An operator should be able to move from the total to the exact records and the values that differ.
Version the comparison rules. A newly added tax or status field can create a wave of differences even though the integration has not changed. Preserve which mapping and normalisation rules produced the report so reviewers can distinguish a genuine divergence from a changed definition.
For an approved correction, create a traceable operation with a reason and an expected result. On the next run, verify that it resolved the intended discrepancy without producing a new one elsewhere. Marking an item resolved manually is not equivalent to observing agreement between the systems.
Retain the last successful comparison time independently of the scheduler. If a run fails before writing its report, the dashboard should show stale evidence rather than an empty list interpreted as no differences. Test this by stopping a run midway through its source read and checking the status visible to operators.