Benchmark automation by reconstructability

Automation success rate is easy to report and easy to misunderstand. A platform can mark a run successful when every configured step completes, even if the source event represented the wrong person, a later system overwrote the result or the customer received an inappropriate message. RevOps needs a companion measure: reconstructability. The question is whether an independent reviewer can trace a sampled run from observed trigger through identity, decision, write, delivery, current state and recovery.

This is a local measurement design, not a claim about industry performance. Attio, Customer.io and Braze provide current examples of observable surfaces. Attio exposes workflow steps, changes, results and agent activity. Customer.io documents polygon-geofence entry and exit support in its stable iOS SDK 4.9.0. Braze describes journey construction, threshold alerts and messaging observability. Use the official product evidence to define what can be observed, then test the organization's cross-system chain.

Build the sample frame from consequences rather than convenience. Include customer messages, CRM ownership or lifecycle changes, audience entry, high-impact data writes and held or rejected actions. Stratify by workflow version, trigger type, channel and outcome. Include normal, failed, held, retried and manually overridden runs. A sample of only completed runs cannot measure whether controls and recovery are reconstructable.

Score trigger evidence first. The packet passes when the reviewer can find a stable event ID, subject ID, source system, schema or shape version, observed time, received time and eligibility-relevant context. For a polygon geofence, that includes the deployed boundary version and SDK version. A screenshot of a map is not enough because it does not prove which geometry evaluated the device event.

Score identity resolution separately. Require the stable person or account keys used at decision time, the link between source and destination identities and any ambiguity or merge state. An email address alone should not receive full credit when stronger identifiers exist. If the event is person-level but the action affects an account, the packet must show the association and the rule that justified the change in grain.

Next score decision evidence. Record the workflow, rule or model version; evaluated-at time; inputs used; output; reason or branch; approval state; and stop-condition result. For assisted journey building, preserve the approved deployed configuration rather than only the natural-language instruction. For an agent tool call, show the bounded action and field authority. This makes proposal, approval and runtime distinguishable.

Score write and delivery evidence in two stages. Execution evidence includes the attempted operation, destination, requested values, vendor run ID and response. State evidence independently reads the destination afterward and records the final value or delivery disposition. This prevents a successful request from being mistaken for a correct business outcome. Braze's sent, held or dropped explanations are useful delivery evidence; current customer state still needs a separate check.

Score temporal coherence. A packet passes when observed, received, evaluated, attempted and confirmed times are present and their governing rules are clear. Penalize sequences that can be ordered only by one ingestion timestamp. Delayed mobile events, queue retries and mid-run customer-state changes are precisely the cases where one clock creates misleading causality.

Score recovery readiness before an incident. The packet should identify the pause control, unprocessed cohort, reversible writes, previous values where required, correction owner and verification step. A workflow history may show exactly which records changed, but a useful recovery score also asks whether the team can safely undo or compensate. Customer communication often cannot be unsent, so the recovery plan may require suppression, disclosure or a follow-up rather than reversal.

Use a five-level rubric. Level zero has no stable run evidence. Level one locates the platform execution. Level two connects trigger, identity and deployed version. Level three adds destination reads and customer-contact disposition. Level four includes approval, exceptions and tested recovery. Report the distribution by consequence class rather than averaging everything into one score that lets many low-risk runs hide a critical gap.

Measure review time alongside completeness. Give a trained reviewer a fixed sample and record the time needed to reach a supported conclusion, systems visited, missing links and expert handoffs. Long review time can reveal fragmented evidence even when every fact technically exists. The goal is not an arbitrary speed target; it is a stable local baseline that improves as correlation keys and evidence views mature.

Track disagreement. Two reviewers should independently classify a subset as reconstructed, unresolved or incorrect and record why. Disagreement often exposes ambiguous field authority, unclear eligibility or inconsistent interpretation of a vendor status. Resolve the definition before using the metric for trend reporting. A reproducible rubric matters more than a flattering initial score.

Review the benchmark monthly and after material workflow changes. Publish the sample frame, definitions, missing-evidence rate, destination mismatch rate, median review time, unresolved cases and recovery-test results. Never convert the local baseline into an external product ranking without equivalent implementations and evidence. The measurement is for operating improvement: it should identify the exact layer that prevents a trustworthy explanation.

Add a freshness dimension without folding it into completeness. A packet can contain every field and still describe an obsolete customer state. Record the age of identity, consent, commercial and destination evidence at decision time, then classify whether each value met the local freshness contract. This separates missing evidence from stale evidence and gives the owner a more actionable remediation path.

Reconstructability complements throughput, latency and cost. Attio's credit information can support resource review; Braze thresholds can reveal volume anomalies; geofence precision can improve event relevance. None replaces the evidence chain. A mature scorecard shows how much automation ran, how quickly, at what cost, how often controls intervened and whether the organization can still explain and correct a consequential sample.

Related reading: Research & Benchmarks · Revenue Operations · Data Quality