[ad_1]
- We’re sharing how Meta applies statistics and machine learning (ML) to improve personalization and management on Instagram – especially for daily push ads.
- By using decision making and ML to identify the most active users who are likely to see more content, we have been able to reduce the number of notifications sent and improve overall user experience.
On Instagram, notifications play an important role in providing a good way of communication between Instagram and our users. With the proliferation of notifications, there has been a need to provide people with personalized notification experiences that help them avoid receiving too many notifications, which they may not find important.
At Meta, we’ve applied statistics and machine learning (ML) for insights and management on Instagram. Today, we want to share an example of how we use inference and ML to manage the delivery of daily push notifications.
Moving beyond click-through rate models
A daily news push notification is a type of notification that lists a digest of the news being shared and ready for anyone to view. When such an ad is sent to someone’s device, they click on the ad to see the details. Traditionally, an ML model called a click-through rate (CTR) model is used to predict how people will click on an ad. CTR models perform well in many applications across the industry. Predictive click probability is used as a proxy to indicate the quality of the ad to the user. If the predicted click probability is too low, the notification will be dropped in the middle of its delivery flow and the user will not receive the notification because it is considered to be of low quality.
The CTR pattern analysis worked well for the daily game notification in the sense that the average click rate was higher when using the CTR pattern than without the pattern. However, we also noticed that using the CTR model meant that a large proportion of daily game notifications were sent to users who were active on Instagram. For many of the most active Instagram users, even if they don’t send these game notifications every day, they can view the same information in a normal way. This opens up the opportunity to provide a better user experience by sending small notifications to active users who view the information listed in the notifications.
The challenge is how to identify these users. If we limit the notifications sent to users who are active in receiving such notifications, those users may become less active. In other words, if the right users are not properly selected for the discount offer, there is a risk that the user engagement will decrease.
Reasoning with ML
Basically, it’s a user choice problem. We want to maximize the effectiveness of the notifications sent by selecting the right user groups. The solution we came up with to address this problem is a combination of inference and ML.
To formulate the problem, let’s assume that there is an accounting fee for posting each game ad each day, and a total budget for these notifications to be spent. Now it has become a budget allocation problem. The key to solving this problem is to consider the increased value of posting game ads every day versus not posting them. For example, the added value for the user i It can be defined according to the user’s ability, that is, ui= Pri(active|do(post)) – Pri(active|action(drop notification)). For some user groups, they tend not to receive daily game notifications, so the added values are limited; selecting these groups to send digest notifications is ineffective and may offend these users. For a better product experience and efficiency, we can sort the ads by value added in descending order and select the top notifications with the highest value increase to send and promote more than the total inflation value and less budget (the supply book).
The next question is how to estimate the increase in value before you decide whether to offer or downsize. It’s a challenging question because for that announcement, we can give or give; there is no way to observe both situations. Basically, this is a causal inference problem and uplift modeling techniques can be used. To request a moving samplewe designed a randomized trial in which each notification was randomly assigned or dropped, as illustrated in Figure 1 .
Based on the data collected from this randomized trial, we developed a neural network-based optimization model to predict the increase between non-dispensing and dispensing daily game notifications. for information at the user level. Given the estimates of the inflation effect for all media, the solution to the budget allocation problem above is limited. However, in practice notifications are generated and scored online, so we cannot prepare impact estimates for all candidate notifications in advance.
As a result, there must be an online way to decide which ads are posted or dropped. A simple but effective solution is to compare the account created online with a fixed threshold – if the account is higher than the threshold we can send it. By doing so, we want to maintain the ad delivery rate r where 0 < r < 1.
When we applied this method in online tests we observed that the transmission rate values because the inflation (inflation effect) produced by the ML models may change from time to time due to the number of of issues. In order to guarantee a delivery rate, we use an online measurement calculation service to convert estimates to a uniform distribution when keeping orders. To keep the transmission rate at rwe simply compare the improved estimate to r to determine the supply, as the estimate has changed Z~U(0,1), Pr( Z >= 1 – r ) = r. This process is shown in Figure 2.
Better insights and reasoning with ML
By using this model and focusing on users + notifications and high impact, the delivery volume was significantly reduced compared to using the CTR model, and did not see a drop in engagement. users. This has two benefits: improved user experience and reduced resource usage.
Within the Instagram Notification System team, ML and statistics have been used in various areas to improve the user notification experience. If you are interested in learning more about this role or would like to join one of our engineering teams, please contact us work pagefollowed us on Facebook.
[ad_2]
Source link