Chain-of-query prompting pipeline for improving small-scale language models in multi-hop open-domain question answering
No Thumbnail Available
Authors
Meeting name
Sponsors
Date
Journal Title
Format
Thesis
Subject
Abstract
Large Language Models (LLMs) have shown to exhibit robust performance in multi-hop open-domain question-answering (ODQA), which is often attributed to the large number of parameters and extensive training. While smaller-scale language models (LMs) offer a more cost-effective approach for real-world applications, these LMs are often challenged with maintaining factual responses in multi-hop ODQA settings. In this thesis, we introduce a novel prompting approach viz., Chain-of-Query (CoQ), that is designed to enhance smaller-scale LMs by decomposing complex queries into context-based subqueries for robust ODQA in multi-hop settings. Our CoQ prompting approach creates an efficient pipeline that integrates with Retrieval-Augmented Generation (RAG) LMs for optimizing the retrieval process through multiple query generation, thereby adding external knowledge to the LM with small amount of context. We show how our CoQ approach can substantially boost the performance of small-scale LMs against state-of-the-art LLMs using metrics like Exact Match (EM) and F1 score, making it a valuable advancement for complex QA tasks. Lastly, we discuss some future research directions and extensions of our work to generative models and conclude by discussing some applications of our pipeline to improve open-source availability of powerful small-scale Language Models.
Table of Contents
DOI
PubMed ID
Degree
M.S.