Boosting LLM performance with generative question-answer Pairs via Wh-transformation

This study explores methods to enhance the performance of offline Large Language Models (LLMs) using generative question-answer (QA) pairs. Existing research highlights the effectiveness of example-based prompts and QA pairs in improving LLM robustness and contextual understanding (Takahashi et al. 2023Takahashi, Kosuke, Takahiro Omi, Kosuke Arima, and Tatsuya Ishigaki. 2023. Training Generative Question-answering on Synthetic Data Obtained from an Instruct-tuned Model. Retrieved September 27, 2024, from [URL], Chowdhury & Chadha 2024Chowdhury, Arijit Ghosh, and Aman Chadha. 2024. Generative data augmentation using LLMs improves distributional robustness in question answering. Proceedings of the 18^th Conference of the European Chapter of the Association for Computational Linguistics: Student Research Workshop, ed. by Neele Falk, Sara Papi and Mike Zhang, 258–265. Stroudsburg, PA: Association for Computational Linguistics. ). However, generating domain-specific QA pairs remains challenging due to the scarcity of datasets across diverse industrial sectors. To address this issue, we advance an innovative and adaptive approach that employs Generative Grammar (Chomsky 1957Chomsky, Noam. 1957. Syntactic Structures. The Hague, Netherlands: Mouton. et seq.) to convert industry-specific statements into questions, thereby facilitating QA pair creation. We compare the efficacy of this method with that of LLM-generated QA pairs. Our proposed approach not only reduces the labor-intensive process typically associated with prompt engineering but also provides a transparent and systematic framework for question generation through controlled wh-movement transformations. Initial findings indicate that QA pairs generated via these transformational rules substantially enhance LLM performance in industrial chatbot applications by enriching contextual information and highlighting promising directions for future LLM research and downstream applications.