Most machine learning models are designed to be self-contained and encode both “knowledge” and “reasoning” in their parameters. However, such models cannot perform effectively for tasks that require knowledge grounding and tasks that deal with non-stationary data, such as news and social media. Besides, these models often require huge number of parameters to encode all the required knowledge. These issues can be addressed via augmentation with a retrieval model. This category of machine learning models, which is called Retrieval-enhanced machine learning (REML), has recently attracted considerable attention in multiple research communities. For instance, REML models have been studied in the context of open-domain question answering, fact verification, and dialogue systems and also in the context of generalization through memorization in language models and memory networks. We believe that the information retrieval community can significantly contribute to this growing research area by designing, implementing, analyzing, and evaluating various aspects of retrieval models with applications to REML tasks. The goal of this full-day hybrid workshop is to bring together researchers from industry and academia to discuss various aspects of retrieval-enhanced machine learning, including effectiveness, efficiency, and robustness of these models in addition to their impact on real-world applications.

Submissions must be in English, in PDF format, and in the current ACM two-column conference format. Suitable LaTeX, Word, and Overleaf templates are available from the ACM Website (use “sigconf” proceedings template for LaTeX and the Interim Template for Word). Submissions may consist of up to 6 pages of content, plus unrestricted space for appendices and references. The papers can represent reports of original research or preliminary research results. The review process is double-blind. Authors are required to take all reasonable steps to preserve the anonymity of their submission. The submission must not include author information and must not include citations or discussion of related work that would make the authorship apparent. However, it is acceptable to refer to companies or organizations that provided datasets, hosted experiments or deployed solutions if there is no implication that the authors are currently affiliated with these organizations. Papers will be evaluated according to their significance, originality, technical content, style, clarity, relevance to the workshop, and likelihood of generating discussion. Authors should note that changes to the author list after the submission deadline are not allowed. At least one author of each accepted paper is required to register for, attend , and present the work at the workshop (either in person or remotely). All papers are to be submitted via EasyChair.

Papers presented at the workshop will be required to be uploaded to arXiv.org after the acceptance notification, but will be considered non-archival, and may be submitted elsewhere (modified or not), although the workshop site will maintain a link to the arXiv versions. Therefore, articles being under consideration by other venues as long as they are within the submission policy of the other venues can be submitted to REML. This makes the workshop a forum for the presentation and discussion of current work, without preventing the work from being published elsewhere.

How can retrieval aid long-form text generation?

Mohit Iyyer

Large language models (LLMs) encode huge amounts of knowledge about the world into their parameters. However, this parametric knowledge is often insufficient to generate accurate and specific information about arbitrary user-selected topics. The emerging area of retrieval-augmented language models thus aims to give LLMs access to external data, usually in the form of a vector database. There are many ways to integrate retrieved data into an LLM for text generation, from late-fusion inference-only methods such as kNN-LM to end-to-end trainable systems such as RETRO. How well do these methods improve long-form text generation, in which a model must generate paragraph-length responses to user queries? In this talk, I first provide an overview of the modeling and evaluation challenges associated with retrieval-augmented long-form question answering. Next, I show that methods such as the kNN-LM may not actually improve the generation quality of language models, again highlighting the challenge of evaluation for future research. Finally, I switch gears and discuss another usage of retrieval entirely: detecting whether a piece of long-form text has been generated by an LLM.