Completion is a local fact
When an API returns success, it proves something narrow: that one system accepted or completed one operation according to its own contract. It does not automatically prove that the intended business outcome occurred.
An email provider can accept a message that never reaches the recipient. A carrier interface can accept an instruction that later fails operationally. A payment request can be created without settlement. A workflow can record a downstream response and still crash before persisting its own state. These are not unusual edge cases. They are ordinary consequences of independent systems, networks, queues, and people acting on different clocks.
Products become misleading when they compress all of those facts into a green “completed” badge. Verification deserves its own state, interface, evidence, and owner.
Separate intent, execution, observation, and proof
A reliable workflow benefits from at least four distinct records:
- Intent — what the user or policy authorized.
- Execution — what command the system attempted and with which identity.
- Observation — what a downstream system reported.
- Verification — what evidence supports the intended real-world result.
The records may be linked, but they should not collapse into one another. An HTTP 201 can be valid execution evidence without becoming proof of delivery. A webhook can be a useful observation without being authoritative for settlement. A manually reviewed document may verify an outcome while also recording uncertainty.
This model introduces more states and more interface copy. That complexity is worthwhile because it matches the world the product is supposed to manage.
Retry changes the product contract
Once a workflow can retry, “send the request” is no longer a single action. The system must decide which failures are retryable, how long to wait, how many attempts are allowed, whether each attempt reuses a stable idempotency key, and what happens when the limit is exhausted.
Exactly-once language is especially dangerous across independent systems. A downstream service may complete an effect while the caller loses the response. Without a shared transaction, the caller cannot always know whether retrying will duplicate the effect. Idempotency can produce an effectively-once boundary when both sides honor it, but that guarantee has a scope and retention period.
The honest product response is to expose the boundary. Persist attempts. Classify responses. Stop after a bounded number. Move unresolved work into a dead-letter state. Give an operator a protected, idempotent recovery path. A retry loop without those decisions is not reliability; it is repeated uncertainty.
Verification needs evidence that can age
Proof is not permanent merely because it was once valid. A document can expire. A status can be superseded. A source system can revise a record. A screenshot can show an interface without proving the data behind it.
Verification therefore needs provenance and time. What source produced the evidence? When was it checked? What scope did the check cover? Can it expire or become temporarily unavailable? Which claim depends on it?
These questions change interface design. Instead of showing only “verified,” a product may need states such as executed-unverified, verified, needs-review, and halted. It may need to display the evidence date beside the business claim and route stale results back to a person.
The user sees more nuance, but the organization gains a system that can admit when it no longer knows.
Recovery is part of the user journey
Failure handling is often documented as an operations concern after the “happy path” is designed. In practice, recovery is a product journey with actors, permissions, decisions, and side effects.
An operator needs to know why the item stopped, what has already happened, what will happen if it is replayed, and whether another operator currently owns the recovery attempt. A customer may need a safe status that does not leak internal or private information. The system needs to prevent an old replay from overwriting a newer decision.
This is why durable ledgers, leases, fences, versioned decisions, and protected approval endpoints matter. They are not infrastructure decoration. They make recovery understandable and prevent the act of fixing a failure from creating a second one.
Verification changes how success is measured
If the product measures only requests completed, it will optimize for throughput. If it measures verified outcomes, stale evidence, duplicate suppression, dead-letter age, and recovery ownership, it will optimize for trustworthy completion.
That change can expose uncomfortable facts. Some outcomes take longer to verify. Some integrations provide weaker evidence than expected. Some “successful” automations need human review. These findings are useful because they reveal the actual operating system rather than the dashboard’s preferred story.
CargoMesh models separate execution and verification states in a logistics reference system. The n8n Reliability Lab demonstrates bounded retry, idempotency, dead-letter handling, approval, and replay using local fixtures. Neither project turns a local test into a production guarantee. Together they make one principle concrete: a product is not done when it acts; it is done when it can responsibly explain what happened next.