Evolving beyond 100 billion recommendations a dayBy Jovan Sardinha, Yue Jin, Alexander Trimm and Garrett Hoffman https://preview.redd.it/bn8kcrldhpz61.png?width=1200&format=png&auto=webp&s=81a1a72e8399abdd1379411e4751d93d19f9f4bc Over the years, Reddit has evolved to become a vast and diverse place. At its core, Reddit is a network of communities. From the content in your feeds to the culture you find in discussions across the site, communities are the lifeblood that makes Reddit what it is today. Reddit’s growth over the years has put extreme pressure on the data processing and serving systems that have served us in the past. This is the journey of how we are building systems that adapt to Reddit and what this has to do with a search for better guides. The Quest Getting comfortable navigating a new place is never easy. Whether it’s learning a new subject or exploring a different environment, we’ve all experienced that overwhelming feeling at some point. This feeling holds us back until we have good guides that help us navigate the new terrain. The sheer scale and diversity that Reddit embodies can be challenging to maneuver at first. If Reddit were a city, ther/popular page would be the town hall, where you can see what is drawing the most discussion. This is where new users get their first taste of Reddit and our core users stumble upon new communities to add to their vast catalogue. The home feed atreddit.com would be the equivalent to a neighborhood park and where each user gets personalized content based on what they subscribed to. For our users, these feeds act as important guides that help them navigate Reddit and discover content that is relevant to their interests. Challenges In 2016, our machine learning models promoted discussion and content that was fresh and liked by people similar to you. This promoted new content and communities that showcased what Reddit had to offer at a point in time. With more diversity of content being published to the platform, our original approach started breaking down. Today, content on Reddit completely changes in minutes; while content that would be relevant to a user could change depending on what they recently visited. The users that make up Reddit are more diverse than ever before. People with a variety of backgrounds, beliefs and situations visit Reddit everyday. In addition, our user interests and attitudes change over time and expect their Reddit experience to reflect this change. Our traditional approaches did not personalize the Reddit experience to accommodate this dynamic environment. Given the amount of change that was taking place, we knew we were quickly approaching a breaking point. The Rebuild To build something our users would love: * Our feeds needed content that was tailored to each individual user when they loaded their feed. * Our systems needed to adapt to changes in user interests, attitudes and consumption patterns. * We had to quickly incorporate feedback from our users and evolve the underlying systems. To do this, we broke down user personalization into a collection of supervised learning subtasks. These subtasks enable our systems to learn a general personalization policy. To help us iteratively learn this policy, we set up a closed loop system (as illustrated below) where each experiment builds on previous learnings: https://preview.redd.it/bjr3zpkshpz61.png?width=960&format=png&auto=webp&s=00185ac513b97b81c8db6db55a99c95b60726c21 This system is made up of four key components. These components work together to generate a personalized feed experience for each Reddit user. A further breakdown of each component: User Activity Library: This component helps us clean and build datasets. These datasets are used to train multi-task deep neural network models which learn a collection of subtasks necessary for personalization https://preview.redd.it/dcu9eoaxhpz61.png?width=960&format=png&auto=webp&s=dbe954e99997be5867c75a56e58b17be5a325824 These datasets contain features that are aggregated on a per user, per post basis across a bounded time horizon (as shown in the image above). Models that train on these datasets simultaneously embed users, subreddits, posts, and user contexts which allow them to predict user actions for a specific situation. For example, for each Reddit user, the model is able to assign a probability the user will upvote any new post, while also assigning a probability the user will subscribe to that subreddit, and if they will comment on the post. These probabilities can be used to estimate long term measures such as retention. Multi-task models have become particularly important at Reddit. Users engage with content in many ways, with many content types, and their engagement tells us what content and communities they value. This type of training also implicitly captures negative feedback - content the user chose not to engage with, downvotes, or communities they unsubscribe from. We train our multi-task neural network models (example architecture shown below) using simple gradient descent-style optimization - like that provided by TensorFlow. At Reddit, we layer sequential Monte Carlo algorithms on top to search for model topology given a collection of subtasks. This allows us to start simple and systematically explore the search space in order to demonstrate the relative value of deep and multi-task structures. https://preview.redd.it/w0iwm521ipz61.png?width=960&format=png&auto=webp&s=508943b1699c20fb63ed5f43dc29b4409978423e Gazette: Feature Stores and Model Prediction Engine: Given the time constraints and the size of the data needed to make a prediction, our feature stores and models live in the same microservice. This microservice is responsible for orchestrating the various steps involved in making predictions during each GET request. We have a system that allows anyone at Reddit to easily create new machine learning features. Once these features are created, this system takes care of updating, storing and making these features available to our models in a performant manner. For real-time features, an event processing system that is built on Kafka pipelines and Flink stream processing directly consumes every key event in real-time to compute features. Similar to the batch features, our systems take care of making these features available to the model in a performant manner. This component maintains a 99.9% uptime and constructs a feed with p99 in the low hundred milliseconds. Which means that this design should hold as we scale to handle trillions of recommendations per day. Model Evaluation and Monitoring: When you predict billions of times a day something is bound to go wrong. Given Reddit’s scale, obvious things (logging every prediction, analyzing model behavior in real time and identifying drifts) become quite challenging. Scaling this component of the system is something we think about a lot and are actively working on. Planning: On every experimentation cycle, we look for ways to improve so that each iteration is better than the past. This discussion will involve looking at data from our models so we can more effectively answer questions such as: * What new tasks can we add to our models so that we can better learn the user policy? * What new components can we add or remove from the current system so that we make the current system more mature? * What new experiments and experiments can we launch so we can learn more about our users? What’s next ? As the world around has changed, we’ve evolved Reddit’s platform: * To incorporate content that is more relevant to each user. * To incorporate real-time changes that might enhance the user experience. * To improve our iteration speed in which we improve our underlying systems. ‘Evolve’ is a core value for all of us at Reddit. This system not only gives us the ability to deal with an ever growing platform, but to try different approaches at a much faster rate. Our next steps will involve experimentation at a new scale as we better understand what makes this place special for our users. https://preview.redd.it/oqaxybz3ipz61.png?width=834&format=png&auto=webp&s=cf8084681fa458edb85b830c0cf1560b7382a7a7 We believe we are just taking the first steps in our journey and our most important changes are yet to come. If this is something that interests you and you would like to join our machine learning teams,check out our careers page for a list of open positions. Team: Jenny Jin, Alex Trimm, Garrett Hoffman, Kevin Loftis, Courtney Wang, Emily Fay, Shafi Bashar, Aishwarya Natesh, Elliott Park, Ugan Yasavur, Jesjit Birak, Jonathan Gifford, Stella Yang, Kevin Zhang, Charlie Curry, Jack Hanlon, Matt Magsombol, Artem Yankov, Jovan Sardinha, Jamie Williams, Jessica Ashoosh, JK Ogungbadero, Susie Vass, Jennifer Gil, Jack Hanlon, Yee Chen, Savannah Forood, Kevin Carbone.
1