Metrics and ExperimentationProduct InterpretationStatistical

You A/B test a new home feed ranking model and see a 3% lift in 7-day retention, but the sample ratio is 52% treatment and 48% control even though you ramped 50/50. What do you do next, and how do you decide whether to trust the retention lift?

Was asked at

Practice this question with AI

First session is free - no credit card required.

Go Premium

More interviews, more skills, more success.

No answers yet

Be the first to share your approach to this question

Practice More Questions

Interview question asked to Data Scientists interviewing at Workday, UberEats, PayPal and other companies. Original question asked: You A/B test a new home feed ranking model and see a 3% lift in 7-day retention, but the sample ratio is 52% treatment and 48% control even though you ramped 50/50. What do you do next, and how do you decide whether to trust the retention lift?.