Motivation:

The vast majority of machine learning (ML) systems are designed to be self-contained, with both knowledge and reasoning encoded in model parameters. They suffer from a number of major shortcomings that can be (fully or partially) addressed, if machine learning models have access to efficient and effective retrieval models:

• Knowledge grounding: A number of important real-world problems, often called knowledge-intensive tasks, require access to external knowledge. They include (among others) open-domain question answering, task-oriented dialogues, and fact checking. Therefore, ML systems that make predictions solely based on the data observed during training fall short when dealing with knowledge-intensive tasks. In addition, ML models related to non-stationary domains, such as news or social media, can significantly benefit from accessing fresh data. An information retrieval (IR) system can decouple reasoning from knowledge, allowing it to be maintained and updated independent of model parameters at a cadence aligned with the corpus.
• Generalization: Recent work has shown that many ML models can significantly benefit from retrieval augmentation. For example, enhancing generative ML models, such as language models and dialogue systems, using retrieval will have a large impact on their generalization.
• Significant growth in model parameters: Since all the required information for making predictions is often encoded in the ML models' parameters, increasing their capacity by increasing the number of parameters generally leads to higher accuracy. For example, the number of parameters used in LMs has increased from 94 million in ELMo to 1.6 trillion in Switch Transformers, an over 16x increase in just three years (2018 -- 2021). Despite these successes, improving performance by increasing the number of model parameters can incur significant cost and limit access to a handful of organizations that have the resources to train them. As such, this approach is neither scalable nor sustainable in the long run, and providing access to a scalable large collection (or memory) can potentially mitigate this issue.
• Interpretability and explainability: Because the knowledge in training data is encoded in learned model parameters, explanations of model predictions often appeal to abstract and difficult-to-interpret distributed representations. By grounding inference on retrieved information, predictions can more easily be traced to specific data, often stored in a human-readable format such as text.

Recent research has demonstrated that errors in many REML models are mostly due to the failure of retrieval model as opposed to the augmented machine learning model, confirming the well-known "garbage in, garbage out" phenomenon. We believe that the expertise of the IR research community is pivotal for further progress in REML models. Therefore, we propose to organize a workshop with a fresh perspective on retrieval-enhanced machine learning through an information retrieval lens. For more information, refer to the following perspective paper:

H. Zamani, F. Diaz, M. Dehghani, D. Metzler, M. Bendersky. "Retrieval-Enhanced Machine Learning". In Proc. of SIGIR 2022.

Theme and Scope:

The workshop will focus on models, techniques, data collections, and evaluation methodologies for various retrieval-enhanced machine learning problems. These include but are not limited to:

• Effectiveness and/or efficiency of retrieval models for knowledge grounding, e.g., for open-domain question answering, dialogue systems, fact verification, and information extraction.
• Effectiveness and/or efficiency of retrieval models for generalization through memorization, e.g., nearest neighbor language models.
• Effectiveness and/or efficiency of retrieval models for memory networks.
• Effectiveness and/or efficiency of retrieval models for retrieval-augmented representation learning.
• Retrieval-enhanced optimization.
• Retrieval-enhanced domain adaptation.
• Retrieval-enhanced models for multi-media and multi-modal learning.
• Query generation for retrieval-enhanced models.
• Retrieval result utilization by machine learning models.
• Interactive retrieval-enhanced machine learning models.
• Retrieval-enhanced models for non-stationary data, such as news, social media, etc.

Program Committee: