9 months of daily A/B-testing of Google AdSense banner ads on Gwern.net indicates banner ads decrease total traffic substantially, possibly due to spillover effects in reader engagement and resharing.
Using the historical traffic data, how easy would it be to detect a total traffic reduction of ~3%, the critical boundary for the ads/no-ads decision? Standard non-time-series methods are unable to detect it at any reasonable sample size, but using more complex time-series-oriented methods like ARIMA models (either NHST or Bayesian), it can be detected given several months of data.