← EXCOIN / INSIGHTS

Fintech data flow: from source to traceable outcome

Map a data flow by business case and ownership of each value. Events may arrive late or more than once; traceability and reconciliation belong in the design.

Begin with the business source of truth

A fintech system rarely has one database containing an entire case. A customer enters an instruction in an application, identity is checked elsewhere, a payment goes through a provider and an internal ledger closes the accounting state. A data-flow map needs to show where each important value originates, who may change it and which side treats it as final. An API diagram shows connections but cannot by itself tell you whether payment acceptance also means completion. Start with one concrete operation and follow its outcome from both the customer's and operator's point of view. Include the moment when each participant first learns of a change.

For every field distinguish its source, copy and derived value. A name may be confirmed by an identity service, an amount entered by the customer and a fee calculated under a particular rule version. When records disagree, the authoritative source and person able to resolve the difference must be known. Data lineage records the origin, transformation and use of a value. In practice it helps explain why a report differs from the operating system. If the path cannot be reconstructed for one case, an aggregate dashboard will not solve the problem. Record assumptions about data freshness and which source can correct a historical mistake.

Draw a reference flow with identifiers

A simple reference flow starts when the customer application creates a request. A domain service assigns a stable identifier, an integration layer sends a command to the partner, and the partner returns technical acceptance followed later by a business outcome. The internal record captures both events, updates case state and feeds reporting. A queue may sit between steps. Every message needs a link to the original case and the version of the data contract. Without it, an outage makes a new operation, a retry and a late confirmation hard to distinguish. The identifier must survive retries and be visible to authorized support staff.

The diagram also needs return paths for rejection, timeout, correction and manual action. Show where a case waiting for the partner is stored and when it is checked again. An arrow back is not enough if nobody owns the next step. An open case needs an owner and an age limit. In a financial flow it matters that a message was delivered technically and whether money was actually processed. The operating interface should name those states clearly so a person does not have to infer the result from an HTTP response code. Include the point where a decision becomes irreversible without a compensating action.

Events loosen coupling but introduce a time gap

Asynchronous events let systems process work at their own pace. Microsoft's overview of event-driven architecture notes that this style commonly introduces temporary inconsistency. That property is acceptable only when the product can display and resolve it. If a customer sees completed before a partner confirms the outcome, the distinction is more than an engineering detail. Define pending, confirmed and unknown states separately. For each item of information set the tolerable delay and the point at which waiting becomes an incident. Explain whether users may proceed while a related value is still catching up.

Events may arrive out of order or more than once. Consumers should safely repeat processing using a stable business-operation ID rather than merely the time of an HTTP attempt. Microsoft's idempotent consumer pattern addresses repeated delivery. The system must also handle a late cancellation after another stage has been confirmed. Do not overwrite a state blindly: check allowed transitions and retain original events so the decision can later be explained. Move unprocessable messages into a repair path with their reason rather than retrying indefinitely. A person needs to know whether the event is waiting, rejected or being investigated.

Audit evidence is different from a debugging log

Audit needs to capture who initiated the operation, what instruction was created, which system accepted it, which rules were used and when the outcome became final. Identifiers, versions, timestamps and relationships between events matter. Full copies of sensitive payloads in every log are unnecessary and may increase leakage risk. Separate records that explain a business decision from detailed technical diagnostics. Each has different permissions and retention. When a person corrects a case, create another traceable event instead of overwriting history until the original mistake disappears. Test whether the evidence is understandable without developer-only knowledge.

OpenTelemetry describes propagating context between processes so traces can connect. A correlation ID can follow a case through API, queue and worker. It does not, by itself, create a reliable audit record: traces may be sampled or kept briefly. Store business-significant events under their own rules independently of diagnostics. In a test, try to reconstruct one successful and one failed case without reading source code. If support must search unrelated tables to see what happened, the flow map is not operationally complete. Redact technical logs while preserving the identifiers needed to join them to authorized evidence.

Reconciliation finds differences between expectation and fact

Reconciliation compares the internal record, the partner's status and perhaps the accounting outcome. It is more than counting messages. For an important transaction check identifier, amount, currency, time, final state and link to the original case. Define frequency and tolerance for differences. Some mismatches resolve when a delayed message arrives; others need intervention. The team must know how long to wait before opening an exception and who owns it. Aggregate totals can agree despite two errors in opposite directions, so individual cases also need comparison. Set a clear cutoff for a daily or other periodic check.

A correction must not silently create another transaction. If a partner completed an operation but your application lost its acknowledgement, establish the actual state before continuing. Distinguish another status query from another instruction. Where compensation is necessary, record its reason and relationship to the original operation. In accounting, an opposite entry may be more appropriate than erasing history. The operational view should show open differences, their age, owner and last action. Reconciliation then becomes a managed process rather than a nightly script whose result nobody reads. Keep a route for escalation when both systems claim to be right.

Test exceptions before optimizing the happy path

The test matrix should include an unavailable partner, lost acknowledgement, duplicate webhook, invalid input, late rejection and altered order. For each case describe expected application state, support evidence and next action. Test concurrent requests against the same resource or limit when the process permits them. If the partner's sandbox cannot produce a condition, simulate it locally and acknowledge the gap. Successful API tests reveal little about what a customer sees after ten minutes of waiting or what an operator does with an unknown outcome. Exercise recovery after a restart as well as a single failed request.

Check changes to data contracts too. A new optional field may be harmless, while a new meaning for a state is not. Assemble examples of real messages for major variants and have both sides confirm them. After release watch failure states, reconciliation differences and manual fixes. The initial period may require more frequent checks. When a new exception appears, add it to tests and update the flow map. The architecture improves from observed operational evidence rather than the number of deployed connectors. Keep the test samples safe to share without exposing real customer records.

Use the map to decide the next stage

A useful map shows data owners, technical boundaries, states, expected waiting time, failure paths and reconciliation. It need not be elaborate: one readable diagram and a short table of contracts can be more useful than a vast model nobody maintains. With every newly connected system ask whether it changes the source of truth or merely reads a copy. If it changes meaning, update the contract and tests before drawing another arrow. Record the decision date and owner so a later team understands why the flow was designed this way. Review the map after a significant partner or product change.

EXCOIN can help map and implement a selected flow, but availability or regulatory compliance cannot be promised without knowing the particular product and partners. A first stage can verify one representative case from instruction to final record, including outage and correction. Measure processing time, unknown states and the time needed to explain a mismatch. If the pilot lets you answer where every operation came from and what happened to it, you have a stronger basis for expansion than a merely successful API connection. The same evidence can also show that a simpler integration is sufficient for the current scale.

Sources

Discuss a project