Table of Contents<\/strong><\/h2>\n\n\n- YouTube Video Recommendation Systems<\/a><\/li>\n
\n- Overview<\/a><\/li>\n
- Candidate Generation Network<\/a><\/li>\n
\n- Architecture and Feature Representation<\/a><\/li>\n
- Training<\/a><\/li>\n<\/ul>\n
- Ranking Network<\/a><\/li>\n
\n- Feature Selection<\/a><\/li>\n
- Architecture<\/a><\/li>\n<\/ul>\n
- Toward Multi-Task Ranking System<\/a><\/li>\n
\n- Multi-Task Learning<\/a><\/li>\n
- Multi-Gate Mixture of Experts<\/a><\/li>\n
- Learning Selection Bias<\/a><\/li>\n<\/ul>\n<\/ul>\n
- Summary<\/a><\/li>\n
\n- Citation Information<\/a><\/li>\n<\/ul>\n<\/ul>\n<\/div>\n
\nYouTube Video Recommendation Systems<\/strong><\/a><\/h2>\nIn this tutorial, you will learn about YouTube video recommendation systems.<\/p>\n
\n![\"\"](\"https:\/\/b2633864.smushcdn.com\/2633864\/wp-content\/uploads\/2023\/09\/youtube-featured.png?lossy=2&strip=1&webp=1\")
<\/a><\/figure>\n<\/div>\nYouTube is the world\u2019s largest platform to create, consume, and share video content. Their recommendations (Figure 1<\/strong>) help billions of users discover personalized content from an ever-growing corpus of videos. According to Cristos Goodrow (VP of Engineering at YouTube), recommendations drive a significant amount of the overall viewership on YouTube, even more than channel subscriptions or searches. <\/p>\n\n![\"\"](\"https:\/\/lh5.googleusercontent.com\/EO-MunfIo7bByhD35RQVhCkzms8DLpc_-U8tYhTLCKKwI4bSXQH34p17m90zb4jkRCdpM3ujVyaionki0GW_Jvd7iniDZBN8sy7ckbOxNiG3N-kHCPzOFTy3UUFxvjUoe8mIMmBRfZwBUUFGH07Z-K0\")
<\/a> Figure 1:<\/strong> YouTube video recommendations on mobile app (source: Covington et al., RecSys\u201916<\/a>).<\/figcaption><\/figure>\n<\/div>\nSo, how do their recommendations actually work? In this lesson, we will answer this question by explaining the machine learning behind YouTube video recommendations.<\/p>\n
This lesson is the 2nd in a 3-part series on Deep Dive into Popular Recommendation Engines 102<\/strong>:<\/p>\n\n- Amazon Product Recommendation Systems<\/em><\/a><\/li>\n
- YouTube Video Recommendation Systems<\/em><\/strong><\/a> <\/em>(this tutorial)<\/strong><\/li>\n
- Spotify Music Recommendation Systems<\/em><\/li>\n<\/ol>\n
To learn how YouTube recommender systems work, <\/strong>just keep reading.<\/em><\/strong><\/p>\n
\nYouTube Video Recommendation Systems<\/strong><\/a><\/h2>\nWe will start with a system overview of the YouTube recommendation algorithm and then dive into individual components later.<\/p>\n
\nOverview<\/strong><\/a><\/h3>\nThe YouTube recommendation algorithm is extremely challenging because of three main reasons:<\/p>\n
\n- Scale:<\/strong> The platform serves billions of users with billions of videos. Highly specialized distributed learning algorithms and efficient serving mechanisms are required to process and serve such massive information in the user base and video corpus.<\/li>\n<\/ul>\n
\n- Freshness:<\/strong> Videos are uploaded every second on the YouTube platform, and user behavior can also change frequently. Hence, the recommendation algorithm should be responsive enough to provide new updates based on the latest videos and user actions. The algorithm should also maintain a balance between new content and well-established videos.<\/li>\n<\/ul>\n
\n- Noise:<\/strong> The metadata associated with the content doesn\u2019t have a well-defined ontology. Further, the user satisfaction signals are implicit most of the time (e.g., watch time), which can also be affected due to external factors. For example, during a cricket season (external factor), users might feel more engaged with match-related videos. <\/li>\n<\/ul>\n
To address all these challenges, YouTube employs a two-stage deep learning-based recommendation strategy that trains large-scale models (with approximately one billion parameters) on hundreds of billions of examples. The overall system (Figure 2<\/strong>) consists of two neural networks for candidate generation and ranking.<\/p>\n\n![\"\"](\"https:\/\/lh4.googleusercontent.com\/ynNiJVoWEo9IqZG2RMhgpPzgGLXyMjMX2cAXYWHjC4J5_pIOtXKw77PE_KqxQzfP1HYSCFi0uUoim1_E5WEtK8ds38UhHN01wAJCjJ_aKxRwblTI5Jby5laXd4rixKnynV47Ygeib5T_-hPZsz-AhZU\")
<\/a> Figure 2:<\/strong> Overview of YouTube recommendation algorithm (source: Covington et al., RecSys\u201916<\/a>).<\/figcaption><\/figure>\n<\/div>\nThe candidate generation network aims to filter out a few thousand video candidates from the vast corpus based on user watch history and context. These candidates are generally relevant to the user with high precision. This network uses coarse features like IDs of video watches, demographics, and search query tokens to generate relevant candidates.<\/p>\n
On the other hand, the ranking network distinguishes between candidates and assigns them a rank according to their relevance to the user with a high recall. This model uses a rich set of features to assign the relevance score between a candidate video and the user.<\/p>\n
We will now describe candidate generation and ranking in further detail.<\/p>\n
\nCandidate Generation Network<\/strong><\/a><\/h3>\nGiven a user 
and context 
, the candidate generation network solves a multi-classification problem where the prediction problem accurately classifies a video watch 
(at time 
) from millions of videos in corpus 
. In other words,<\/p>\n
<\/p>\n
where 
is a high-dimensional embedding representing user history and context. 
represents the embedding of video 
. The candidate generation network aims to learn the dense user embeddings 
as a function of user history and context so that it helps discriminate videos via a softmax classifier. Note that the video embeddings are pre-learned in a fixed vocabulary via continuous bag-of-words language models.<\/p>\n
\nArchitecture and Feature Representation<\/strong><\/a><\/h4>\nAs shown in Figure 3<\/strong>, the candidate generation network comprises several features (e.g., user watch history, search query, demographics, user gender, example age, etc.). All these features are concatenated and fed to a feed-forward network comprising fully connected layers and ReLU activation.<\/p>\n\n![\"\"](\"https:\/\/lh6.googleusercontent.com\/Zz0-JEsQGGzbrzs8JkLQRugmubfh-HZZVSWbE-HI7KSgd9GXauNv9h_VtRjeZCDSFFz5-R8xYIOLkHyYVKgwin5w8Sq_MgwWPi-n1d4wGTTlcG7wb0FTa0Lxy6wIxAdHlSV6ZAI5owtDsE_YpMiw4ok\")
<\/a> Figure 3:<\/strong> Architecture of candidate generation network (source: Covington et al., RecSys\u201916<\/a>).<\/figcaption><\/figure>\n<\/div>\nThe user watch history is represented as a variable-length sequence of sparse video IDs (watched by the user recently) mapped to their dense embeddings . These dense embeddings (of videos watched by the user) are then averaged to represent the user watch vector fed into a feed-forward network. Note that the video embeddings are learned jointly with other model parameters.<\/p>\n
Like the user watch vector, the search query is also tokenized into unigrams and bigrams, and each token is mapped to its respective token embedding. The embeddings of all tokens are then averaged to obtain a user search vector.<\/p>\n
The user\u2019s geographic location and device ID are embedded and concatenated to represent demographics. Further, the binary and continuous features (e.g., user gender, logged-in state, and age) are fed directly after being normalized to ![[0, 1]](https:\/\/b2633864.smushcdn.com\/2633864\/wp-content\/latex\/264\/264884439b70ab09a86bc848421c6de6-ffffff-000000-0.png?lossy=2&strip=1&webp=1 "[0, 1]")
.<\/p>\n
The distribution of video popularity is highly non-stationary (i.e., it lasts for a few days, slows down, and then converges). Still, the above candidate recommendation network will only recommend based on the average watch likelihood of a video (but not its popularity). To address this, a unique \u201cExample Age\u201d feature is also given as input during training.<\/p>\n
\u201cExample Age\u201d represents the age of the training example used. During inference, it is set to zero or negative to ensure that the model predictions don\u2019t get biased toward the past, and it assigns more likelihood to a new video. Figure 4<\/strong> shows that adding \u201cExample Age\u201d as a feature increases the probability of a recently uploaded video (as opposed to a baseline model) and is close to its empirical value.<\/p>\n\n![\"\"\/](\"https:\/\/lh6.googleusercontent.com\/8wH9Hz1tE4qTXtTphktvkQjC-aHNkdQVp-8W-DKaYMNBxN1V4lS1UldyCMcy9WyXVlMhaFHVB4to6YkxutsMnMLpuQx1utmYEvp0_k1cNCzKYFb3HMF0kcHlV8xRk_Ukg9EmSrq1J_Hl2u99AtCwi1I\")
<\/a> Figure 4:<\/strong> Effect on class probabilities with \u201cExample Age\u201d as input feature (source: Covington et al., RecSys\u201916<\/a>).<\/figcaption><\/figure>\n<\/div>\n
\nTraining<\/strong><\/a><\/h4>\nTo train the network, YouTube leverages implicit feedback signals where a user completing a video is considered a positive example. This is because explicit feedback signals like thumbs up and down in-product surveys are quite rare. <\/p>\n
Further, training examples are generated from all YouTube watches (even those embedded on other sites) rather than watches from the recommendations. This ensures that feedback signals contain information about users discovering videos through means other than recommendations. This way, the recommender would not be biased toward exploitation. <\/p>\n
Now, given a watch history of a user 
and corresponding search history 
, there are two ways of training the candidate generation network:<\/p>\n
\n- Predicting Held-Out Watch<\/strong> (Figure 5<\/strong>): In this setting, you predict a randomly held-out look, e.g.,
given past and future search queries 
and watch history 
as inputs to network. However, this leaks future information and ignores asymmetric consumption patterns (e.g., you will likely watch episode 2 of the series only after episode 1 but not vice versa). <\/li>\n<\/ul>\n\n![\"\"](\"https:\/\/lh6.googleusercontent.com\/3waH4oCu6TSNH0ksB0izag5LbY2Sm5s5Ro9lAFlm-80IpXg5n4uRf9bLHKayR4bG2_ce44evr7RxY5I987mR7IEvQ54943ocpT8gxwaoG3yDzF0EAPnvKj0735OpEGMHg7we9XfxaPALAnQsRecr3Gw\")
Figure 5:<\/strong> Predicting held-out watch setting (source: Covington et al., RecSys\u201916<\/a>).<\/figcaption><\/figure>\n<\/div>\n\n- Predicting Future Watch<\/strong> (Figure 6<\/strong>): This setting involves predicting future watches given past search queries and watch history. Since the YouTube recommendation problem is posed as predicting the next watched video, this setting is more appropriate for training neural networks. <\/li>\n<\/ul>\n\n
![\"\"\/](\"https:\/\/lh3.googleusercontent.com\/4MKlmWQusvL1BG3IVMGiV9yNWo3S-9t0l7Lp4PrXQ99DRfUj613etzTW4XIiR4e5O4cllYh5UioOclPx2pXc4K3wcWQSs0PAIMTouG6eFOBfaQuOWr07IeBBXYdAE27AbmzhJ4v670LxIuhIBtRanFA\")
<\/a> Figure 6:<\/strong> Predicting future watch setting (source: Covington et al., RecSys\u201916<\/a>).<\/figcaption><\/figure>\n<\/div>\nFigure 7<\/strong> shows the performance of the candidate generation network when different sets of features are given input to the network and network depth is varied. The performance is measured via holdout mean average precision (MAP), which measures how accurately the models predict held-out watches. The network achieves the best performance using all features (watch history, search history, example age, demographics). Further, the performance improves monotonically with an increase in the network depth.<\/p>\n\n![\"\"](\"https:\/\/lh4.googleusercontent.com\/gpHfgu5Kju45yMKvlc2oWoZwhSj-9ICgdQsr26az099ruq4DSORinb8sSJTEEJKOC_O9uUI70SFxrs-6pP_8PrXYOhra0Grsuvcj7xerlxlXMnEG6OWa5Z-9dlgrMXziAN2Q7NnLNxlV1VCZgfNu-gg\")
<\/a> Figure 7:<\/strong> Performance of candidate generation network with different sets of features as input (source: Covington et al., RecSys\u201916<\/a>).<\/figcaption><\/figure>\n<\/div>\n
\nRanking Network<\/strong><\/a><\/h3>\nThe goal of the ranking network is to take the video candidates from the candidate generation network and rank them in decreasing order of their relevance to the user. A user might watch a particular video with a high probability, but because the thumbnail image is not appealing, the user never clicks it. <\/p>\n
Hence, the ranking network takes in many more features describing the video (e.g., video thumbnail, video content, etc.) and the user’s relationship to the video to assign a relevance score. The network uses both categorical and continuous features. Categorial features can be binary (e.g., whether the user is logged in?) or have millions of values (e.g., the user’s last search query).<\/p>\n
\nFeature Selection<\/strong><\/a><\/h4>\nContinuous features describe a user\u2019s past interaction with similar videos. For example, while scoring a video uploaded by channel X, it is important to consider how many videos a user has watched from channel X? Or when was the last time the user watched a similar video. <\/p>\n
Other continuous features include signals from the candidate generation network. For example, what score was assigned by the candidate generation network? Or which sources are nominating the video candidate? <\/p>\n
Further, features describing the frequency of past video impressions are also critical so that successive requests return different recommendations. If a user recently recommended a video but did not watch it, the model should reduce its relevance score.<\/p>\n
\nArchitecture <\/strong><\/a><\/h4>\nFigure 8 <\/strong>describes the overall architecture of the ranking network.<\/p>\n\n![\"\"](\"https:\/\/lh4.googleusercontent.com\/bMX6KOf_wOcrkLS40H6ly6uZzUTY0UYZ7MQ6t9eLiyKUMboFvFEK1bFsD8LTvwHAw_I6A0Up041jfBW6HOpfQV_O1gDLomGn2Bly3gHI6_cPVgDemvfSl3ue5uYo5lKKL_7p83L9giIML3gGEN7u9CA\")
<\/a> Figure 8:<\/strong> Architecture of ranking network (source: Covington et al., RecSys\u201916<\/a>).<\/figcaption><\/figure>\n<\/div>\nLike the candidate generation network, the video IDs are mapped to their respective fixed-sized embeddings, which are learned along with the network parameters. The network takes the embedding of candidate videos, the average embedding of watched videos, and user and video language embeddings as inputs. <\/p>\n
For continuous features like \u201ctime since last watch\u201d and \u201c# previous impressions,\u201d we first normalize the values such that the feature is equally distributed in the range 
. In Figure 9<\/strong>, 
represents the normalized feature. We also provide powers of normalized features such as 
, 
as inputs to give the network more expressive power by allowing it to form super- and sub-linear functions of the feature easily.<\/p>\nThe networks take all these features (in reality, more than we described) and pass them through a fully connected network to predict the expected watch time via weighted logistic regression. In weighted logistic regression, the positive (clicked) impressions are weighted by the observed watch time on the video. Negative (unclicked) impressions all receive unit weight.<\/p>\n
\nToward Multi-Task Ranking System<\/strong><\/a><\/h3>\n
\nMulti-Task Learning<\/strong><\/a><\/h4>\nIn the approach described above, the main objective of the ranking system was to improve the likelihood of a video being watched or clicked. However, in reality, the task of recommending videos is multi-objective. For example, we may want to recommend videos that users rate highly and share with their friends in addition to watching.<\/p>\n
Further, the system often has implicit bias where the user might have clicked or watched a video simply because it was ranked high and not because the user liked it. This will further get amplified via feedback loops as such videos will be considered positive examples while updating the network.<\/p>\n
Objectives can be categorized into two categories: <\/p>\n
\n- Engagement level objectives include clicks, watches, intensity of watch, etc.<\/li>\n
- Satisfaction objectives include liking a video, leaving a rating, sharing, etc.<\/li>\n<\/ol>\n
\nMulti-Gate Mixture of Experts<\/strong><\/a><\/h4>\nTo model these multiple objectives, YouTube leverages a Multi-Gate Mixture of Experts (MMoE) to learn parameters to share across potentially conflicting objectives. The mixture of experts is a popular machine-learning technique that uses multiple expert networks. <\/p>\n
It modularizes the input layer into several experts, each focusing on different input aspects. This improves the representation of learning. Then, by utilizing multiple gating networks, each objective can choose experts to share or not share with others. <\/p>\n
YouTube models engagement level objectives via two types of tasks:<\/p>\n
\n- binary classification task for predicting the user click probability<\/li>\n
- regression task to predict time spent on a video<\/li>\n<\/ul>\n
Similarly, satisfaction level objectives can be modeled via binary classification tasks (e.g., predicting user likes and shares) and regression tasks (predicting ratings).<\/p>\n
As shown in Figure 9<\/strong>, for each video candidate and query (can be a video or search query), the model takes their embeddings as inputs along with user and context features. Then, scores from these multiple objective tasks (engagement and satisfaction) are obtained using MMoE. These scores are then combined into a single score using weighted multiplication (weights are manually tuned to achieve the best performance) during serving. <\/p>\n\n![\"\"](\"https:\/\/lh3.googleusercontent.com\/srMvHUT3vxROj1BvZGFdmxKdAb4dsNcZxicVjXN_YC79rROVlvESc6BZNFhahKaAJt07SQHTbr-QxTpbsLhUe7O9gk_bvz2CeC1H5xCPWgkkbat_x91YvKZd4sT1UfNibS4mIjP0ytCpNqvO-5lW9zw\")
<\/a> Figure 9:<\/strong> Overview of the multi-task objective ranking system using a multi-gate mixture of experts (source: Zhao et al., RecSys\u201919<\/a>).<\/figcaption><\/figure>\n<\/div>\nMMoE is a soft parameter-sharing model structure with expert networks shared across all tasks but gating networks trained for each task. As shown in Figure 10<\/strong>, multiple experts are added on top of a shared bottom layer. This way, MoE can learn modularized information from the input. The implementation of expert networks is similar to multi-layer perceptrons with ReLU activations.<\/p>\n\n![\"\"](\"https:\/\/lh6.googleusercontent.com\/QOBdBeniGsacdTaMm1EHsUgvKaiMnitogrNH8va4hFmTQyxAsigaokOmSPq3GkL8clbGB9tKUdj8UQ3BF7PDonfH6r3mbOH2e7lycwTnVxiXIg1ojSxgbDhCOIwAsyLWFTSVp8ewgXvmJykT4ecoJpo\")
<\/a> Figure 10:<\/strong> Comparison of a conventional shared-bottom architecture with a multi-gate mixture of experts (MMoE) (source: Zhao et al., RecSys\u201919<\/a>).<\/figcaption><\/figure>\n<\/div>\nGiven a task 
, the prediction 
of the MMoE (with 
expert networks) is given as follows:<\/p>\n
<\/p>\n
<\/p>\n
where 
is the 
th expert network, 
is the gating network for task , and 
is the final prediction layer for task .<\/p>\n
The gating networks are simple linear transformations of the input with the softmax layer<\/p>\n
<\/p>\n
where 
are the parameters of the gating network.<\/p>\n
Table 1<\/strong> reports the improvement in engagement and satisfaction level metrics when the number of experts in MMoE architecture varies.<\/p>\n\n![\"\"](\"https:\/\/lh3.googleusercontent.com\/eq3a-EORw4sOWr4KozSt_J3b9wKiC2-C9X0IOovD9qHeZ6CzsXUvZSscmhYbnQyzP3qyQS-vdO9zlNXoC1T92x2hOZVouDHJTh8eKNdGk-nYZ-Q8VU_S8ucVv6ku0lFzuRuFDz_MyNoPzQYIUSvl_Wg\")
<\/a> Table 1:<\/strong> Performance of MMoE compared to shared-bottom architecture (source: Zhao et al., RecSys\u201919<\/a>).<\/figcaption><\/figure>\n<\/div>\n
\nLearning Selection Bias<\/strong><\/a><\/h4>\nIn case you have not noticed, Figure 9<\/strong> shows another component called a \u201cshallow tower.\u201d The role of this shallow tower is to mitigate positional bias in the ranked items. In a ranking system, it is common for the users to click the videos that are on the top, even if they are irrelevant to the user and the given context (search query or video being watched currently). This is referred to as position bias.<\/p>\nFigure 11<\/strong> illustrates the distribution of CTR (click-through rate) for positions 1 to 9. The CTR represents the probability of an item being clicked by the user (i.e., an engagement level metric). As can be seen, the CTR is significantly higher for top positions because of position bias.<\/p>\n\n![\"\"](\"https:\/\/lh4.googleusercontent.com\/inhth2iQLpFqYGnsUdpCXSCqXpm9N1-LcVFVerTSocPwjfOStEFRABUo_E-hcWldqpZaXm-X1MNsixPO52iA2_jfKd9c-fhx6CXTjDyOzOcCpOyFoApQr3NlubZRZtUzGJ1uLDkDdrsXFaNjuLBqujE\")
<\/a> Figure 11:<\/strong> Distribution of CTR (click-through rate) for positions 1 to 9 (source: Zhao et al., RecSys\u201919<\/a>).<\/figcaption><\/figure>\n<\/div>\nTo remove this position bias, YouTube factorizes the model prediction into two components: the user utility component from the main ranking network (that uses MMoE) and the position bias components. This is illustrated in Figure 12<\/strong>.<\/p>\n\n![\"\"](\"https:\/\/lh5.googleusercontent.com\/wa7lA3MzrWoaLokY6raR1Pw36VZUv-SQpFod0aKojji2con3oQmH1b09ON4N-TFeSFqNtL9Zzz3tZK-tH1aDz_5HOu35iepYCovs1mXfqER4WvRBI1OtLrgdnpqNO6yIk6ok1NN6wUXmkCi7EZtEpn8\")
<\/a> Figure 12:<\/strong> Shallow tower network for incorporating position bias (source: Zhao et al., RecSys\u201919<\/a>).<\/figcaption><\/figure>\n<\/div>\nMore specifically, a shallow tower or network is trained with features contributing to position bias to output a position bias logit. This position bias logit is then added with the logit from the main ranking network. During training, the positions of all candidate videos are used with a 10% feature drop to prevent the model from over-relying on the position bias of the video.<\/p>\n
This feature is treated as missing during serving time, and the ranking is done based on the scores from the main ranking network.<\/p>\n
\n\nWhat’s next? I recommend PyImageSearch University<\/a>.<\/h3>\n\t
- \n
- YouTube Video Recommendation Systems<\/a><\/li>\n
- \n
- Overview<\/a><\/li>\n
- Candidate Generation Network<\/a><\/li>\n
- \n
- Architecture and Feature Representation<\/a><\/li>\n
- Training<\/a><\/li>\n<\/ul>\n
- Ranking Network<\/a><\/li>\n
- \n
- Feature Selection<\/a><\/li>\n
- Architecture<\/a><\/li>\n<\/ul>\n
- Toward Multi-Task Ranking System<\/a><\/li>\n
- \n
- Multi-Task Learning<\/a><\/li>\n
- Multi-Gate Mixture of Experts<\/a><\/li>\n
- Learning Selection Bias<\/a><\/li>\n<\/ul>\n<\/ul>\n
- Summary<\/a><\/li>\n
- \n
- Citation Information<\/a><\/li>\n<\/ul>\n<\/ul>\n<\/div>\n
\nYouTube Video Recommendation Systems<\/strong><\/a><\/h2>\n
In this tutorial, you will learn about YouTube video recommendation systems.<\/p>\n
\n<\/a><\/figure>\n<\/div>\nYouTube is the world\u2019s largest platform to create, consume, and share video content. Their recommendations (Figure 1<\/strong>) help billions of users discover personalized content from an ever-growing corpus of videos. According to Cristos Goodrow (VP of Engineering at YouTube), recommendations drive a significant amount of the overall viewership on YouTube, even more than channel subscriptions or searches. <\/p>\n
\nFigure 1:<\/strong> YouTube video recommendations on mobile app (source: Covington et al., RecSys\u201916<\/a>).<\/figcaption><\/figure>\n<\/div>\n So, how do their recommendations actually work? In this lesson, we will answer this question by explaining the machine learning behind YouTube video recommendations.<\/p>\n
This lesson is the 2nd in a 3-part series on Deep Dive into Popular Recommendation Engines 102<\/strong>:<\/p>\n
- \n
- Amazon Product Recommendation Systems<\/em><\/a><\/li>\n
- YouTube Video Recommendation Systems<\/em><\/strong><\/a> <\/em>(this tutorial)<\/strong><\/li>\n
- Spotify Music Recommendation Systems<\/em><\/li>\n<\/ol>\n
To learn how YouTube recommender systems work, <\/strong>just keep reading.<\/em><\/strong><\/p>\n
\nYouTube Video Recommendation Systems<\/strong><\/a><\/h2>\n
We will start with a system overview of the YouTube recommendation algorithm and then dive into individual components later.<\/p>\n
\nOverview<\/strong><\/a><\/h3>\n
The YouTube recommendation algorithm is extremely challenging because of three main reasons:<\/p>\n
- \n
- Scale:<\/strong> The platform serves billions of users with billions of videos. Highly specialized distributed learning algorithms and efficient serving mechanisms are required to process and serve such massive information in the user base and video corpus.<\/li>\n<\/ul>\n
- \n
- Freshness:<\/strong> Videos are uploaded every second on the YouTube platform, and user behavior can also change frequently. Hence, the recommendation algorithm should be responsive enough to provide new updates based on the latest videos and user actions. The algorithm should also maintain a balance between new content and well-established videos.<\/li>\n<\/ul>\n
- \n
- Noise:<\/strong> The metadata associated with the content doesn\u2019t have a well-defined ontology. Further, the user satisfaction signals are implicit most of the time (e.g., watch time), which can also be affected due to external factors. For example, during a cricket season (external factor), users might feel more engaged with match-related videos. <\/li>\n<\/ul>\n
To address all these challenges, YouTube employs a two-stage deep learning-based recommendation strategy that trains large-scale models (with approximately one billion parameters) on hundreds of billions of examples. The overall system (Figure 2<\/strong>) consists of two neural networks for candidate generation and ranking.<\/p>\n
\nFigure 2:<\/strong> Overview of YouTube recommendation algorithm (source: Covington et al., RecSys\u201916<\/a>).<\/figcaption><\/figure>\n<\/div>\n The candidate generation network aims to filter out a few thousand video candidates from the vast corpus based on user watch history and context. These candidates are generally relevant to the user with high precision. This network uses coarse features like IDs of video watches, demographics, and search query tokens to generate relevant candidates.<\/p>\n
On the other hand, the ranking network distinguishes between candidates and assigns them a rank according to their relevance to the user with a high recall. This model uses a rich set of features to assign the relevance score between a candidate video and the user.<\/p>\n
We will now describe candidate generation and ranking in further detail.<\/p>\n
\nCandidate Generation Network<\/strong><\/a><\/h3>\n
Given a user
and context , the candidate generation network solves a multi-classification problem where the prediction problem accurately classifies a video watch (at time ) from millions of videos in corpus . In other words,<\/p>\n<\/p>\nwhere
is a high-dimensional embedding representing user history and context. represents the embedding of video . The candidate generation network aims to learn the dense user embeddings as a function of user history and context so that it helps discriminate videos via a softmax classifier. Note that the video embeddings are pre-learned in a fixed vocabulary via continuous bag-of-words language models.<\/p>\n
\nArchitecture and Feature Representation<\/strong><\/a><\/h4>\n
As shown in Figure 3<\/strong>, the candidate generation network comprises several features (e.g., user watch history, search query, demographics, user gender, example age, etc.). All these features are concatenated and fed to a feed-forward network comprising fully connected layers and ReLU activation.<\/p>\n
\nFigure 3:<\/strong> Architecture of candidate generation network (source: Covington et al., RecSys\u201916<\/a>).<\/figcaption><\/figure>\n<\/div>\n The user watch history is represented as a variable-length sequence of sparse video IDs (watched by the user recently) mapped to their dense embeddings
. These dense embeddings (of videos watched by the user) are then averaged to represent the user watch vector fed into a feed-forward network. Note that the video embeddings are learned jointly with other model parameters.<\/p>\nLike the user watch vector, the search query is also tokenized into unigrams and bigrams, and each token is mapped to its respective token embedding. The embeddings of all tokens are then averaged to obtain a user search vector.<\/p>\n
The user\u2019s geographic location and device ID are embedded and concatenated to represent demographics. Further, the binary and continuous features (e.g., user gender, logged-in state, and age) are fed directly after being normalized to
.<\/p>\nThe distribution of video popularity is highly non-stationary (i.e., it lasts for a few days, slows down, and then converges). Still, the above candidate recommendation network will only recommend based on the average watch likelihood of a video (but not its popularity). To address this, a unique \u201cExample Age\u201d feature is also given as input during training.<\/p>\n
\u201cExample Age\u201d represents the age of the training example used. During inference, it is set to zero or negative to ensure that the model predictions don\u2019t get biased toward the past, and it assigns more likelihood to a new video. Figure 4<\/strong> shows that adding \u201cExample Age\u201d as a feature increases the probability of a recently uploaded video (as opposed to a baseline model) and is close to its empirical value.<\/p>\n
\nFigure 4:<\/strong> Effect on class probabilities with \u201cExample Age\u201d as input feature (source: Covington et al., RecSys\u201916<\/a>).<\/figcaption><\/figure>\n<\/div>\n
\nTraining<\/strong><\/a><\/h4>\n
To train the network, YouTube leverages implicit feedback signals where a user completing a video is considered a positive example. This is because explicit feedback signals like thumbs up and down in-product surveys are quite rare. <\/p>\n
Further, training examples are generated from all YouTube watches (even those embedded on other sites) rather than watches from the recommendations. This ensures that feedback signals contain information about users discovering videos through means other than recommendations. This way, the recommender would not be biased toward exploitation. <\/p>\n
Now, given a watch history of a user
and corresponding search history , there are two ways of training the candidate generation network:<\/p>\n- \n
- Predicting Held-Out Watch<\/strong> (Figure 5<\/strong>): In this setting, you predict a randomly held-out look, e.g., given past and future search queries and watch history as inputs to network. However, this leaks future information and ignores asymmetric consumption patterns (e.g., you will likely watch episode 2 of the series only after episode 1 but not vice versa). <\/li>\n<\/ul>\n\n
Figure 5:<\/strong> Predicting held-out watch setting (source: Covington et al., RecSys\u201916<\/a>).<\/figcaption><\/figure>\n<\/div>\n - \n
- Predicting Future Watch<\/strong> (Figure 6<\/strong>): This setting involves predicting future watches given past search queries and watch history. Since the YouTube recommendation problem is posed as predicting the next watched video, this setting is more appropriate for training neural networks. <\/li>\n<\/ul>\n\n
Figure 6:<\/strong> Predicting future watch setting (source: Covington et al., RecSys\u201916<\/a>).<\/figcaption><\/figure>\n<\/div>\n Figure 7<\/strong> shows the performance of the candidate generation network when different sets of features are given input to the network and network depth is varied. The performance is measured via holdout mean average precision (MAP), which measures how accurately the models predict held-out watches. The network achieves the best performance using all features (watch history, search history, example age, demographics). Further, the performance improves monotonically with an increase in the network depth.<\/p>\n
\nFigure 7:<\/strong> Performance of candidate generation network with different sets of features as input (source: Covington et al., RecSys\u201916<\/a>).<\/figcaption><\/figure>\n<\/div>\n
\nRanking Network<\/strong><\/a><\/h3>\n
The goal of the ranking network is to take the video candidates from the candidate generation network and rank them in decreasing order of their relevance to the user. A user might watch a particular video with a high probability, but because the thumbnail image is not appealing, the user never clicks it. <\/p>\n
Hence, the ranking network takes in many more features describing the video (e.g., video thumbnail, video content, etc.) and the user’s relationship to the video to assign a relevance score. The network uses both categorical and continuous features. Categorial features can be binary (e.g., whether the user is logged in?) or have millions of values (e.g., the user’s last search query).<\/p>\n
\nFeature Selection<\/strong><\/a><\/h4>\n
Continuous features describe a user\u2019s past interaction with similar videos. For example, while scoring a video uploaded by channel X, it is important to consider how many videos a user has watched from channel X? Or when was the last time the user watched a similar video. <\/p>\n
Other continuous features include signals from the candidate generation network. For example, what score was assigned by the candidate generation network? Or which sources are nominating the video candidate? <\/p>\n
Further, features describing the frequency of past video impressions are also critical so that successive requests return different recommendations. If a user recently recommended a video but did not watch it, the model should reduce its relevance score.<\/p>\n
\nArchitecture <\/strong><\/a><\/h4>\n
Figure 8 <\/strong>describes the overall architecture of the ranking network.<\/p>\n
\nFigure 8:<\/strong> Architecture of ranking network (source: Covington et al., RecSys\u201916<\/a>).<\/figcaption><\/figure>\n<\/div>\n Like the candidate generation network, the video IDs are mapped to their respective fixed-sized embeddings, which are learned along with the network parameters. The network takes the embedding of candidate videos, the average embedding of watched videos, and user and video language embeddings as inputs. <\/p>\n
For continuous features like \u201ctime since last watch\u201d and \u201c# previous impressions,\u201d we first normalize the values such that the feature is equally distributed in the range
. In Figure 9<\/strong>, represents the normalized feature. We also provide powers of normalized features such as , as inputs to give the network more expressive power by allowing it to form super- and sub-linear functions of the feature easily.<\/p>\nThe networks take all these features (in reality, more than we described) and pass them through a fully connected network to predict the expected watch time via weighted logistic regression. In weighted logistic regression, the positive (clicked) impressions are weighted by the observed watch time on the video. Negative (unclicked) impressions all receive unit weight.<\/p>\n
\nToward Multi-Task Ranking System<\/strong><\/a><\/h3>\n
\nMulti-Task Learning<\/strong><\/a><\/h4>\n
In the approach described above, the main objective of the ranking system was to improve the likelihood of a video being watched or clicked. However, in reality, the task of recommending videos is multi-objective. For example, we may want to recommend videos that users rate highly and share with their friends in addition to watching.<\/p>\n
Further, the system often has implicit bias where the user might have clicked or watched a video simply because it was ranked high and not because the user liked it. This will further get amplified via feedback loops as such videos will be considered positive examples while updating the network.<\/p>\n
Objectives can be categorized into two categories: <\/p>\n
- \n
- Engagement level objectives include clicks, watches, intensity of watch, etc.<\/li>\n
- Satisfaction objectives include liking a video, leaving a rating, sharing, etc.<\/li>\n<\/ol>\n
\nMulti-Gate Mixture of Experts<\/strong><\/a><\/h4>\n
To model these multiple objectives, YouTube leverages a Multi-Gate Mixture of Experts (MMoE) to learn parameters to share across potentially conflicting objectives. The mixture of experts is a popular machine-learning technique that uses multiple expert networks. <\/p>\n
It modularizes the input layer into several experts, each focusing on different input aspects. This improves the representation of learning. Then, by utilizing multiple gating networks, each objective can choose experts to share or not share with others. <\/p>\n
YouTube models engagement level objectives via two types of tasks:<\/p>\n
- \n
- binary classification task for predicting the user click probability<\/li>\n
- regression task to predict time spent on a video<\/li>\n<\/ul>\n
Similarly, satisfaction level objectives can be modeled via binary classification tasks (e.g., predicting user likes and shares) and regression tasks (predicting ratings).<\/p>\n
As shown in Figure 9<\/strong>, for each video candidate and query (can be a video or search query), the model takes their embeddings as inputs along with user and context features. Then, scores from these multiple objective tasks (engagement and satisfaction) are obtained using MMoE. These scores are then combined into a single score using weighted multiplication (weights are manually tuned to achieve the best performance) during serving. <\/p>\n
\nFigure 9:<\/strong> Overview of the multi-task objective ranking system using a multi-gate mixture of experts (source: Zhao et al., RecSys\u201919<\/a>).<\/figcaption><\/figure>\n<\/div>\n MMoE is a soft parameter-sharing model structure with expert networks shared across all tasks but gating networks trained for each task. As shown in Figure 10<\/strong>, multiple experts are added on top of a shared bottom layer. This way, MoE can learn modularized information from the input. The implementation of expert networks is similar to multi-layer perceptrons with ReLU activations.<\/p>\n
\nFigure 10:<\/strong> Comparison of a conventional shared-bottom architecture with a multi-gate mixture of experts (MMoE) (source: Zhao et al., RecSys\u201919<\/a>).<\/figcaption><\/figure>\n<\/div>\n Given a task
, the prediction of the MMoE (with expert networks) is given as follows:<\/p>\n<\/p>\n<\/p>\nwhere
is the th expert network, is the gating network for task , and is the final prediction layer for task .<\/p>\nThe gating networks are simple linear transformations of the input with the softmax layer<\/p>\n
<\/p>\nwhere
are the parameters of the gating network.<\/p>\nTable 1<\/strong> reports the improvement in engagement and satisfaction level metrics when the number of experts in MMoE architecture varies.<\/p>\n
\nTable 1:<\/strong> Performance of MMoE compared to shared-bottom architecture (source: Zhao et al., RecSys\u201919<\/a>).<\/figcaption><\/figure>\n<\/div>\n
\nLearning Selection Bias<\/strong><\/a><\/h4>\n
In case you have not noticed, Figure 9<\/strong> shows another component called a \u201cshallow tower.\u201d The role of this shallow tower is to mitigate positional bias in the ranked items. In a ranking system, it is common for the users to click the videos that are on the top, even if they are irrelevant to the user and the given context (search query or video being watched currently). This is referred to as position bias.<\/p>\n
Figure 11<\/strong> illustrates the distribution of CTR (click-through rate) for positions 1 to 9. The CTR represents the probability of an item being clicked by the user (i.e., an engagement level metric). As can be seen, the CTR is significantly higher for top positions because of position bias.<\/p>\n
\nFigure 11:<\/strong> Distribution of CTR (click-through rate) for positions 1 to 9 (source: Zhao et al., RecSys\u201919<\/a>).<\/figcaption><\/figure>\n<\/div>\n To remove this position bias, YouTube factorizes the model prediction into two components: the user utility component from the main ranking network (that uses MMoE) and the position bias components. This is illustrated in Figure 12<\/strong>.<\/p>\n
\nFigure 12:<\/strong> Shallow tower network for incorporating position bias (source: Zhao et al., RecSys\u201919<\/a>).<\/figcaption><\/figure>\n<\/div>\n More specifically, a shallow tower or network is trained with features contributing to position bias to output a position bias logit. This position bias logit is then added with the logit from the main ranking network. During training, the positions of all candidate videos are used with a 10% feature drop to prevent the model from over-relying on the position bias of the video.<\/p>\n
This feature is treated as missing during serving time, and the ranking is done based on the scores from the main ranking network.<\/p>\n
\n\nWhat’s next? I recommend PyImageSearch University<\/a>.<\/h3>\n
\t
- Predicting Future Watch<\/strong> (Figure 6<\/strong>): This setting involves predicting future watches given past search queries and watch history. Since the YouTube recommendation problem is posed as predicting the next watched video, this setting is more appropriate for training neural networks. <\/li>\n<\/ul>\n
- Predicting Held-Out Watch<\/strong> (Figure 5<\/strong>): In this setting, you predict a randomly held-out look, e.g.,
- Noise:<\/strong> The metadata associated with the content doesn\u2019t have a well-defined ontology. Further, the user satisfaction signals are implicit most of the time (e.g., watch time), which can also be affected due to external factors. For example, during a cricket season (external factor), users might feel more engaged with match-related videos. <\/li>\n<\/ul>\n
- Freshness:<\/strong> Videos are uploaded every second on the YouTube platform, and user behavior can also change frequently. Hence, the recommendation algorithm should be responsive enough to provide new updates based on the latest videos and user actions. The algorithm should also maintain a balance between new content and well-established videos.<\/li>\n<\/ul>\n
- YouTube Video Recommendation Systems<\/em><\/strong><\/a> <\/em>(this tutorial)<\/strong><\/li>\n
- Amazon Product Recommendation Systems<\/em><\/a><\/li>\n
- Multi-Gate Mixture of Experts<\/a><\/li>\n
- Architecture<\/a><\/li>\n<\/ul>\n
- Training<\/a><\/li>\n<\/ul>\n
- Candidate Generation Network<\/a><\/li>\n
- Overview<\/a><\/li>\n