Amplitude adds scheduled stops for experiments and feature flags
Amplitude can now stop an experiment or flag at a scheduled time and freeze the analysis window. Operators still need to reconcile late data, fallback behavior and downstream actions outside Amplitude.
What changed
Amplitude shipped scheduled experiment stops on 1 September. The release says teams can set a start time, an end time or both. At the end time, experiment delivery stops, a flag turns off, the analysis window freezes and the experiment moves to Completed (Pending decision). Owners are notified and the active-experiment slot is released.
Sources: Amplitude official release note
The stop becomes part of the experiment definition
A planned end was previously easy to keep in a calendar or runbook while delivery depended on a person returning to the product. Scheduling turns that intent into a product control. Amplitude also displays a duration summary and says schedules can be edited after an experiment starts.
The operator should still verify time zone, daylight-saving behavior, permission, notification recipients and what happens to subjects active exactly at the boundary. Record the planned and actual stop separately. A correct configuration is not the same as observed execution.
A frozen analysis window improves decision discipline
Amplitude says the analysis window freezes when the scheduled stop fires. That can prevent the reported population from continuing to change while a team reviews the result. It also creates a clear point for preserving the method, cohort, metrics, incidents and result in a decision packet.
Late events still need a rule. Event time and ingestion time can differ, and a warehouse or downstream report may continue to reconcile. Teams should document whether late events are excluded, added to a corrected result or held in a separate quality report. Do not silently change a result after approval.
Pending decision is a useful separation
The status described by Amplitude distinguishes completed delivery from the decision about what to do next. That separation matters because a stopped test can be inconclusive, affected by tracking problems or ready only for a bounded rollout. Completion should not automatically select a winner or trigger a customer action.
Require the owner to record ship, do not ship, rerun, narrow or hold, with the supporting evidence and uncertainty. If the next action changes a flag, customer cohort or CRM workflow, create a new approved version rather than reopening the old result informally.
Turning a flag off is not a universal rollback
The release says flags turn off at the scheduled end. The application still needs a defined fallback value, and client, server, cached or offline evaluation paths can behave differently. Test the exact SDK and environment used in production, including a session that spans the stop boundary.
Any message, task, CRM update or integration job emitted before the stop may have its own queue and terminal state. Map those consumers separately. A product-level stop can be successful while a downstream lifecycle platform continues processing work it already received.
The active-slot effect needs governance
Amplitude says the stopped experiment releases its active-experiment quota slot. That makes capacity easier to reuse, but it can also encourage teams to start the next test before the prior decision and cleanup are complete. Capacity and decision readiness should remain separate states.
Before reusing a slot, confirm that exposure collection has stopped, exceptions are owned, the result packet is preserved, downstream work is reconciled and temporary flags or audiences have a retirement plan. The platform slot is a technical resource, not evidence that the operating cycle is closed.
A practical acceptance test
Create synthetic subjects immediately before and after the stop. Confirm assignment, actual exposure, fallback value, notification, status and frozen window. Submit one duplicate exposure, one late outcome and one subject whose eligibility changes after the schedule was approved. Keep the expected result for every case.
Then inspect connected warehouse, lifecycle, CRM and reporting paths. Reconcile record counts and stable identifiers rather than relying on a screenshot. Route every mismatch to an owner and repeat the test after changing the schedule or evaluation environment.
What operators should verify next
Confirm plan availability, user permissions, edit behavior, notification delivery, time zone, exact flag-off semantics, analysis-window handling, late-event behavior, quota release and log retention in the target account. Review the current documentation because release behavior can evolve.
Scheduled stops reduce dependence on memory and create a cleaner decision boundary. Their operational value comes from the surrounding evidence: observed delivery stop, tested fallback, frozen definitions, downstream reconciliation and a named decision owner.
What the release does not establish
Amplitude's note documents a product capability. It does not prove that an experiment is statistically valid, that a flag is safe in every application, that late data is immaterial or that a measured outcome caused revenue or retention. Those conclusions require the team's own method and system evidence.
For RevOps, the useful takeaway is narrower: the experiment now has a programmable end. Treat that end as the start of a reconciliation and decision workflow, not as automatic authorization for the next customer or commercial action.
Original source
This DailyRevOps article is written in our own words from the source signal and adds RevOps context, workflow analysis, and operator interpretation.
- Original source: Amplitude
- Original publication date:
- Source link: Read the original article