StatisticalProduct InterpretationMetrics and Experimentation

If a ranking experiment yields a 3% retention win but suffers from a severe SRM, what is your diagnostic checklist before reporting the results?

Was asked at

Practice this question with AI

First session is free - no credit card required.

Go Premium

More interviews, more skills, more success.

Practice More Questions

1 answer published. Last candidate submission on Jul 30 2026, 4:56pm PDT. Interview question asked to Data Scientists interviewing at Deliveroo, Duolingo, Yahoo and other companies. Original question asked: If a ranking experiment yields a 3% retention win but suffers from a severe SRM, what is your diagnostic checklist before reporting the results?. Question last reported by a candidate interviewing at Pinterest for a Data Scientist interview on Jun 6 2026, 2:24pm PDT.