IDEAS: Incremental Dense Representations

Modern content-based search and recommendation systems take a piece of text as an input, such as a web page or a job posting, and create a representation of this entire text. This is used to generate results and recommendations based on the user’s query. This approach works well, but it lacks control over how parts of the input text influence the resulting representation. Yates proposes a paradigm where information is incrementally added into a representation piece-by-piece. This control can improve result quality and enable analysis of the content inside a representation, allowing us to look for harmful content like biases.