Given observational data from two disparate product campaigns, how would you mathematically structure a quasi-experiment or A/B test if we observe a 3% baseline increase for one product?

Question

Anonymous · Accepted Answer

First, I'd clarify what business question are we trying to answer? Are we
estimating a campaign’s incremental impact, comparing the two campaigns, or
deciding how to allocate future spending? Are we measuring unit sales, revenue,
or conversion?

I'll assume two different products, concurrent campaigns, and that the 3% means
one product was already ahead before launch.

I'd structure it as a regression: Y_it = β0 + β1·Treat_i + β2·Post_t +
β3·(Treat_i × Post_t) + error. Y is unit sales for product i in period t. Treat
flags the campaigned product, Post flags the campaign window.

Each coefficient has a job: β0 is the control product's pre-campaign average. β1
is the pre-existing gap between the products, so if the 3% is a level
difference, it lands there and stays out of the effect estimate. β2 is anything
that lifted both products at the same time, like seasonality. β3 is the answer:
the treated product's change minus the control's change, which is the campaign's
incremental lift.

That subtraction is only valid if the products would have moved in parallel
without the campaign, so I'd check pre-period data first. If the 3% is actually
a growing trend rather than a fixed gap, β1 can't absorb it, so I'd add a
treated-product trend term, β4·(Treat_i × t), or adjust each Y by θ times its
pre-period value to strip the drift out.

Then I'd convert β3 into incremental units per dollar to compare the campaigns
for budget allocation.

Given observational data from two disparate product campaigns, how would you mathematically structure a quasi-experiment or A/B test if we observe a 3% baseline increase for one product?

Practice this question with AI

Go Premium

Community Answers

Unlock Full Access

Practice More Questions