<p>I wouldn’t immediately stop or proceed with the launch. First, I’d validate whether the decline is real and isolate where it’s happening in the funnel. Then I’d diagnose root causes using cohort and behavioral analysis. If the issue is minor or understood, I’d proceed with a controlled rollout. But if core metrics like activation or retention are significantly impacted, I’d delay the launch and fix the underlying problem before scaling.</p>

<div>This would depend on the usage metrics that are in decline and the product launching.</div>
<div>I would start with investigating why there is a drop in usage, as there are several scenarios -</div>
<div>1. if low usage is overall and due to seasonality, I would generally hold off on product launches.</div>
<div>2. if some of the usage metrics are down, and some are up, then it depends on the product itself -</div>
<div>a. If the new product is more relevant to the ones that are up, no reason not to go forward.</div>
<div>b. If the new product is more relevant to the declined metrics, then we need to understand if it's a steady decilne (trend seen overtime) and the new product might assist with it. Or, perhaps it's a sudden decline, and I would hold off with product launch until I have an understanding of what is causing this sudden decline.</div>
<div>Of course, if any publich commitments were made to the launch of this product, it's also something we need to take into consideration, since changing a public publish date is problematic for user trust.</div>

<div>Uber's mission: reinvent how the world moves</div>
<div>Uber rides mission: get people from point A to B</div>
<div><br></div>
<div>I have some questions first about the decline in usage metrics.</div>
<div>What usage metrics specifically are we seeing decline?</div>
<div>When did the dropoff start? was it sudden or gradual?</div>
<div>How big was the change relative to what we've seen in the past?</div>
<div>How important is the change? Is it endangering the core function of the product?</div>
<div><br></div>
<div>If the viability of the core product is jeopardized, we should probably focus on fixing the core product rather than launching a new product (unless we think the new product could fix the usage decline). These questions will help us understand the dropoff and address it.</div>
<div><br></div>
<div>Next I have questions about motivation for the new product.</div>
<div>Do we have a mission for our new product? Do we know how it interacts with the existing core product? That will help us understand if the viability is jeopardized. If it remains viable, likely we want to continue with the launch, but we want to make sure we can evaluate it. It may be hard to evaluate the launch if our metrics are in a free-fall. So we should want to see our metrics have stabilized before launching the new product.</div>
<div><br></div>
<div>Do we know what KPIs we'd want to use to evaluate the new product? Perhaps % of our users who used the product. Given potential for network effects in the Uber marketplace, we may want to run this as a market-level A/B. We can either pick a few markets by hand where we'd like to implement this and then use a synthetic control to create a synthetic comparison group of other markets as a control. Or we can do a stratified random sample and implement the program in the areas selected for treatment. I would recommend the former so we can maximize the success of the new program.</div>
<div><br></div>
<div>We should also have success criteria, e.g., we will expand the experiment if utilization exceeds 20%. &nbsp;Once the experiment has run, we can determine whether it met the success criteria, and if so, roll it out to a larger treatment aea for additional study.</div>

<div>Clarifying questions -&nbsp;</div>
<ul>
  <li>Which usage metrics are declining? Are these for the overall Meta site or only for the product about to launch? Are all metrics on decline or only this specific metric?&nbsp;</li>
  <li>What does launch mean? Does it mean to start A/B testing, or post testing? Is the Launch going to be ramped or big bang?</li>
  <li>Is there a similar product currently existing? Internally or competitor? What is Meta's goal with launching this new product?</li>
  <li>What is the time frame of the observed decline? Is it sudden or gradual?</li>
</ul>
<div>Let us say,&nbsp;</div>
<ul>
  <li>avg time spent by a user in a day on this product is declining.&nbsp;</li>
  <li>A/B testing is complete, a phased launch is planned with upcoming phase being all users in US.</li>
  <li>Let us say there is a mature existing product in the market but no Meta product.</li>
</ul>
<div>I would trend the avg time spent by user over time, with the x axis being a calendar day. Since this is still pre-launch the volume of users is not much and time frame is also small. So I would take all users who were in the experiment, from the start of the experiment. Say 2 months.</div>
<div><br></div>
<div>I observe that at the start of the experiment, there was high activity - avg time spent grew for the first 1 week, flattened for a week after and then started having a slow decline. The time plots seems to indicate novelty effect. The decline seems to have stabilized post novelty.&nbsp;</div>
<div><br></div>
<div>If the goal of Meta is to take market share from competitor, this new product might still be a viable product to launch. Since the usage is stable and sustained, the product has a potential to grow from network effect post launch.&nbsp;</div>

<p>We need to reschedule the launch date and recheck the planning stage. Collaborate with the product team on how you can make sure that the usage metrics will increase.</p>

In the event of a product launch, but a noticeable decrease in user engagement metrics, what course of action would you suggest taking?

Practice this question with AI

Go Premium

Community Answers

Unlock Full Access

Unlock Full Access

Unlock Full Access

Unlock Full Access

Unlock Full Access

Practice More Questions