Clarify
All notifications from Facebook or related to specific feature or product?
Notifications sent via app or sent via email have reduced?
What time frame was the 15% drop for?
Are there any other metrics with unexpected drops?
Why is it important?
Let us say email notifications from FB to users about all FB products & features have reduced over a month by 15%. Notifications are important because they cause people to log in more frequently.
Question - Has there been a change in the algorithm used to generate notifications?
Let us assume no, there has not been.
Validation -
I would firstly trend total notifications, DAU, Total daily number of users to whom notifications are sent for the past 3 months. I would check if I see the decline mentioned and if I understand the metric definition correctly.
Let us assume that I do see a 15% decline in total notifications sent. the decline is not sudden but gradual. This indicates it is less likely to be a bug or data drop.
I'd like to investigate further by slicing the total Notifications trend created above by country, users with less than 50 friends, 50-200, 200+ and check if some group behaves abnormal.
Hypothesis -
Let us consider why the decline could be? Notifications come from activity of other users or groups in a users network. Maybe the overall content creation has gone down, there by feeding the notification algorithm less input.
Am I thinking about this the right way?
Adding to the above trends I started exploring, I'd like to trend total number of posts generated over the same time period of 3 months.
Another segment to consider is the number of notifications broken out by origination - were they notifications about friend's activity, posts on groups, marketplace activity, reels, friend requests, videos.
Looking at all these trends, I might get a pointer on what is going on.
Say, I see notifications have dipped in the US but steady in other major geos. I may also see a dip in marketplace notifications.
When I filter all above created visuals to just US, I may see that marketplace notifications were the ones that were the contributor.
At this point, I would talk to engineering & PMs to check if they know of anything going on with marketplace. Maybe there is a new feature being tested or launched which is interfering with triggering notifications from there.
