Retrieval Augmented Generation for Intelligent Querying of Databases and Documents

Abstract

The rise in conversational AI systems, resulted in a change in the way data is being retrieved and queried. In the conventional systems, various data retrieval techniques were used but users had to be proficient with them. With the evolution in Large Language Models, a strong performance in natural language generation was observed. Large Language Models such as Gemini 2.0 and LLaMA showed the exceptional capabilities while interacting with humans but their limited access to external, dynamic and real time data made them unable to be used in enterprise applications. Retrieval-Augmented Generation was adopted to deal with these issues. RAG when combined with LLMs efficiently retrieves the external data and allows users to communicate through human prompts. The objective was to achieve intelligence for a system capable of changing user prompts to the executable queries that helps fetching data from structured and non-structured data sources. For structured data, Gemini 2.0 was used that generated the queries through tool calling and provides the automated exploration of table schemas and field relations. Unstructured data sources were processed by using Sentence Transformer model for chunking and creating embeddings. Multiple chunking techniques were explored to achieve the best context aware embeddings. These embeddings then stored into vector databases allow the cosine similarity search with the query embeddings and returns the nearest candidates. Performance was evaluated using both automated metrics and human judgment. Structured queries achieved a 94.2% precision rate in SQL translation and execution, with schema awareness reaching 89.5%. Unstructured document retrieval showed a semantic relevance score of 91.2%, while textual faithfulness of generated answers was measured at 83.5%. It was concluded that RAG can effectively enhance the LLM performance in the data driven environments, especially in enterprise applications. This approach of integrating RAG and LLM proved to be scalable, adaptable and implementable in various realworld problems.