Smarsh reports Agentforce support gains and shows the measurement boundary
A company-reported support case offers useful evidence for operators, provided its observed results stay separate from universal targets and retention claims.
The reported result
Smarsh said on September 3 that its Agentforce-based customer agent, Archie, achieved 72% self-service deflection across 405 interactions in the second quarter of 2026. The company also described a manual confidence assessment and an internal agent, Emmy, that helps support representatives with account context and case work. These are company-reported results from a specific deployment, not an independently audited industry benchmark.
The useful signal for customer-operations teams is the combination of an external service workflow and an internal assistance workflow. They affect different parts of the customer experience and need different evaluation criteria. The analysis below proposes how another team can inspect a similar implementation without assuming Smarsh's outcome will transfer.
Read the denominator before the percentage
Ask which interactions entered the measurement and which were excluded. A result can change materially depending on whether the denominator includes abandoned sessions, repeat contacts, transfers, unsupported requests and customers who never reached the agent. The right definition depends on the service model, but the definition should be stated before the result is interpreted.
Keep the unit consistent. Sessions, questions, cases and customers are not interchangeable. One customer may ask several questions in one session or return with the same issue in several sessions. A dashboard that moves between these units can make an operational improvement appear larger or smaller without any underlying change in customer experience.
Separate containment from resolution
A conversation that does not reach a human can be a successful self-service outcome. It can also end because the customer gave up, changed channels or decided to return later. A sound review looks for evidence that distinguishes these possibilities. Define a repeat-contact window and explain why it fits the product and the types of questions being handled.
Review a sample of apparently successful sessions alongside escalations. Ask whether the answer addressed the actual question, respected the account context and provided a usable next step. Keep unresolved cases visible instead of assigning them the most convenient label. A human review should have a written rubric so two reviewers can explain disagreement rather than merely average their opinions.
Evaluate internal assistance on a different axis
An internal agent that prepares account context can reduce lookup effort without resolving the customer's underlying problem. Measure whether the representative receives accurate, current and relevant information, and whether the preparation changes the work they must do. Include the time required to verify or correct the output. Apparent speed gained before review can disappear when the evidence is incomplete.
Compare similar work where possible. Complex cases can differ in product, severity, customer configuration and engineering involvement. A simple before-and-after average may reflect a different case mix. Preserve those characteristics and distinguish elapsed case time from active handling time. The two measures answer different operational questions and should not be combined into a single productivity claim.
Inspect the human handoff
The customer should not have to reconstruct the whole issue after escalation. Review whether the receiving person gets the question, relevant evidence, actions already attempted and the reason for transfer. At the same time, avoid allowing an unverified summary to overwrite the customer's actual statements. Keep a trace to the original interaction so the representative can inspect ambiguity.
Assign responsibility for gaps that cross teams. A missing knowledge source may belong to documentation, an account mismatch to CRM operations and an unsupported action to the service owner. A clean handoff includes a route for those internal fixes. Otherwise the frontline team absorbs the cost while the agent's headline metric remains unaffected.
Build a pilot that can fail honestly
Choose a bounded question set with sufficient volume to inspect and a responsible human owner. Write success and stop criteria in advance. Include incorrect associations, missing knowledge, contradictory instructions and requests beyond the supported scope. A pilot that contains only easy questions can prove that the easy path works; it cannot establish readiness for the full customer population.
Report both the promising result and the operating limits. State the cohort, window, review method, exclusions, correction effort and unresolved outcomes. Do not infer revenue retention from service deflection. Customer effort, support quality and renewal decisions can be related, but one metric does not establish the others.
What operators should take away
Smarsh's disclosure is a useful starting point for a measurement discussion. The appropriate next step is to design a local evaluation with comparable clarity about scope, rather than copy the headline into a business case as a guaranteed target. A credible deployment earns confidence through repeatable behavior and inspectable evidence.
For RevOps, the account-context connection deserves particular attention. Service automation is stronger when the customer identity, record authority and handoff are reliable. Those foundations also support post-sales work beyond the immediate case, but the broader commercial value should be evaluated on its own evidence.
Original source
This DailyRevOps article is written in our own words from the source signal and adds RevOps context, workflow analysis, and operator interpretation.
- Original source: Smarsh
- Original publication date:
- Source link: Read the original article