Start with the meaning of data
Two systems can exchange JSON over HTTP and still disagree about what the data means. One may define a customer as a person with an active contract; another treats every contact in a database as a customer. One timestamp records when a request was created, another when it was approved. These differences initially look like minor field mapping questions. In production they create duplicate records, incorrect states and disputes about which system is wrong. Start with a vocabulary of business terms and a few real examples rather than a list of endpoints.
For each important field, identify the source of truth, permitted changes and owner of corrections. If a CRM creates customers and a billing system owns payment status, both applications should not independently decide the value of the same field without a conflict rule. Include time semantics: when is a change considered complete, and how old may a displayed value be? Only then choose a transport mechanism. A short table of meanings, ownership and events may be enough. Its value comes from precision rather than its size.
Choose a boundary that fits the user journey
A synchronous call suits a step where the user needs a prompt answer and acceptable waiting time is clear. An asynchronous message helps when processing takes longer, demand arrives in bursts or the destination is occasionally unavailable. This is not a competition between technologies. Trace the specific journey: what can the screen truthfully confirm, what may remain “processing”, and how will the user learn the final outcome? Displaying “complete” when only the first request succeeded creates a misleading product state.
Separate commands that change state from queries that merely read it. Record whether the caller requires immediate consistency or can tolerate a later update. For an asynchronous flow, determine how a received message connects to the original request and how an item that repeatedly fails can be repaired. A small project may need only a well-designed API and a simple queue. A complicated platform cannot compensate for a business process whose ownership and outcomes are undefined.
A contract includes errors and change
An API contract is more than the fields in a successful response. It should describe required and optional data, identifiers, pagination, permissions, error codes and the meaning of each state. The OpenAPI Specification can express operations, schemas, callbacks and webhooks. Use such a description as a shared artefact for client, supplier and tests. A specification file does not replace worked examples when the business meaning depends on context. Someone must still agree what “accepted”, “settled” or “cancelled” means.
Every contract change has consumers. Adding an optional field is generally easier than changing the meaning of an existing one, yet it is not automatically safe for every client. Event messages need explicit compatibility rules and a defined transition period. Confluent's Schema Registry documentation distinguishes backward, forward, full and transitive compatibility. The right rule depends on who deploys first and how long older messages remain in circulation. Test a new contract against real consumers and representative stored data, not only against a schema parser.
An unknown outcome is not a failed outcome
The most dangerous integration case occurs when a caller sends a command but receives no response. The destination may have completed the work while its response was lost. A blind retry can create a second payment, order or account. For operations that change state, design an idempotency key or another unambiguous way to recognise the same logical request. Repeating that request should reveal the known result rather than trigger the side effect again. Decide how long the key remains valid and what happens if the same key arrives with different content.
Retry only transient failures, and impose a limit. AWS guidance on retry with backoff discusses idempotency, overload caused by repeated calls and the need to distinguish temporary from persistent failure. A practical design identifies who retries, for how long, at what interval and when a person takes over. Automatically repeating a request with permanently invalid data wastes capacity. The better response is a visible state, an intelligible rejection reason and a safe correction process.
Plan recovery from partial completion
Independent systems rarely share one database transaction. Imagine creating a client in CRM, recording a contract and then establishing a billing profile. The first two steps can succeed while the third fails. Decide whether the earlier steps will be compensated, completed later or left in an explicit pending state. Each option has cost and business consequences; there is no universal answer. The decision belongs in the process design, where stakeholders can see its effect on customers.
Write several failure scenarios at the boundary between steps and assign a decision owner to each. A compensating action is not necessarily an exact reverse of the original one. A sent email cannot be unsent, and a settled payment may require a distinct refund process. If human intervention is needed, an administrator must see the original request, every attempt and the safe actions available now. This work is best done before implementation because it shapes the data model, administrative interface and support procedure.
Security belongs to the whole flow
Integrations often move sensitive information between teams or suppliers. Do not assume that an internal network alone establishes trust. Define the calling identity, permission scope, credential rotation and logging policy for every connection. Least privilege has a concrete form: a service reading an order status should not be allowed to change bank details. For webhooks, verify the origin and message signature according to the provider's actual protocol, and account for repeated delivery rather than assuming an event arrives once.
Do not place full tokens, personal data or payment instructions in application logs. A correlation identifier, timestamp, step, error category and carefully selected metadata usually support diagnosis. Evaluate storage and transport according to the sensitivity of the data. Legal obligations depend on the purpose and location of processing and need a separate assessment. Engineers can make that assessment possible by supplying an accurate inventory of flows and access roles. Security design grows from the process map; it should not be appended after the endpoints are built.
Operations must explain the state
An “API is responding” metric is inadequate when a business operation can disappear between steps. Track accepted and completed requests, age of the oldest unfinished item, proportion requiring manual repair, failures by partner and differences between expected and actual records. Give each metric an owner and a threshold that triggers a real investigation. A dashboard without an action is decoration. Distinguish a technical failure from a legitimate business outcome, such as an application that was correctly declined.
Every flow should be traceable from the user's request to its final step. A correlation identifier helps, but a history of state transitions and contract versions can shorten diagnosis further. An operational runbook should explain how to pause sending, safely reprocess an item, reconcile records and escalate a partner-side fault. Rehearse it with a controlled failure. If only the original developer can execute the procedure, the integration is not yet ready to hand over.
Test real cases, including the inconvenient ones
A successful request is the easiest part to test. Add duplicate delivery, a timeout after successful execution, an old message version, missing required data, incorrect permissions and partial completion. Test migration or replay of older records as well. Verify outcomes in the API response, database, user-visible state and operations record. Those views can disagree even when an individual unit test passes. The integration is correct only when their meanings stay aligned.
Before launch, run a limited pilot with representative data and a prepared rollback. Agree who will approve business correctness and who will check technical measures. Record known limitations, ambiguous states and their resolution procedures. Sound integration architecture does not promise that errors will never happen. It makes them detectable, explainable and repairable, then lets the organisation continue without losing the meaning of its data.