[Question] Is there a valid and recognized approach to stopping rules that considers stability over time?
Reddit r/statistics2d5 min read
When you look at data over time, it's fairly common to see weird blips, even in cumulative mean effect calculations, that look like this: /\ | \ ________| \___________________ Or, while waiting for regression to the mean, it's very normal to see your cumulative mean effect, plotted over time, look like this: \ \ /\ \ / \ /\/\___________ \ / \/ \/ In a recent role advising a company on how to run experiments, I showed them examples of this type of data to demonstrate how stopping a test early could lead to misleading results. We had a great example of data where a bizarre blip on a single day,