Today, it has become widely accepted that large language models (LLMs), on their own, are insufficient to enable intelligent AI agents that can deliver real value within an enterprise context. Rather, it is only by integrating and augmenting LLMs with enterprise data that we can deliver truly robust, accurate, and valuable AI agents and applications.
Retrieval augmented generation (RAG) currently offers the most practical and scalable approach for grounding LLMs with enterprise data. By leveraging vector database and embedding technologies, RAG enables LLMs to process requests and deliver responses based on information it was not trained on and has never seen before. This capability is particularly crucial when applying AI in dynamic enterprise environments, in which it’s essential to access real-time, accurate data. With that said, it has proven to be challenging to implement RAG and deploy generative AI (GenAI) applications within an enterprise environment. One recent study by Gartner predicts that 30% of GenAI projects will be abandoned by the end of 2025 due to poor data quality and other factors.
This blog post delves into how Query RAG, a variation of the RAG pattern with a different set of steps and requirements, can be applied to structured data sources to deploy powerful AI agents and how the Denodo Platform, along with the newly released Denodo AI SDK offering, can help organizations to quickly transition from RAG concepts to the deployment of highly relevant and valuable AI applications.
Query RAG and Structured Data
Most RAG contents and examples today focus on augmenting LLMs with unstructured data. As a result, the techniques and eco-systems around implementing RAG using unstructured data such as websites and PDFs have matured rapidly and are proving to be robust and scalable. Unfortunately, the same level of technological maturation has not extended to the world of structured or tabular data. It turns out that augmenting LLMs with structured data presents a different set of challenges that are poorly understood today.
At Denodo, we have been working closely with many customers to implement the RAG pattern using structured data sources, and we came across a number of unique challenges that do not apply in the world of unstructured data:
- Structured or tabular data cannot simply be retrieved; it must be accessed and queried via correct SQL or API calls
- Access mechanisms and optimal data retrieval techniques can differ from vendor to vendor and platform to platform (These include file parsing, SQL, GraphQL, and proprietary APIs, to name a few)
- Vector embedding strategies must cover both metadata and actual data, and bridge the gap between natural-language-based queries and often highly technical metadata.
- Structured data (and its associated metadata) changes all the time. So, accessing current, real-time data presents unique changes.
- Existing data security and permission controls need to be respected and integrated
These unique challenges highlight the need for a distinct approach to handling structured data. We have been studying these challenges closely, and have identified Query RAG as a variation of the RAG pattern with a different set of steps and requirements. At a high level, Query RAG should provide:
- A robust, scalable way to create vector embeddings across all metadata and data sources and platforms
- A unified SQL access engine that abstracts away the complexities of the underlying data sources and systems
- A powerful query optimizer that enables the SQL generated by LLMs to be highly performant across multiple data architectures and use cases
- A flexible, robust semantic layer that enables you to develop new data views that encapsulate the relevant business context and language
- A simple process for enforcing user permissions and data security at every step
One of the more unique aspects of Query RAG is that the goal of the embedding search is not to find the text to augment the prompt but to find the right data view to query. Once the right data view and correct SQL have been identified, the tabular data is then retrieved from multiple data sources and is used to construct the relevant response back to the user.
Query RAG provides the ability to utilize enterprise data repositories in a robust, scalable way. By leveraging embedding and exposing real-time data to AI applications, this approach can help organizations to extend their GenAI capabilities beyond unstructured data, delivering real-time insights and improving decision-making in ways that we have never seen before.
The Denodo Platform and Denodo AI SDK
As a robust and mature logical data management solution, we believe that the Denodo Platform is uniquely suited to support Query RAG implementation across large, complex enterprise environments. The Denodo Platform’s highly optimized data virtualization engine tackles the challenges of integrating data silos and provides a unified, secure access point. By leveraging orchestration frameworks such as LangChain and vector database technologies, developers can already start building powerful new AI chatbots and applications using Query RAG and the Denodo Platform.
With the recently released Denodo AI SDK we want to further simplify the process of building these new types of AI applications. The Denodo AI SDK includes the popular LangChain open-source library as well as pre-built integrations with all popular LLMs and vector databases. By offering multiple integration options, Denodo customers are not locked into specific LLM providers and are free to evolve and innovate in a rapidly changing AI ecosystem.
Via customized LangChain flows, the AI SDK automatically handles the data embedding and orchestration process. It also includes pre-built custom prompts, to facilitate SQL generation and the data query process without additional custom development.
More importantly: by offering a set of streamlined APIs purpose-built for Query RAG, the Denodo Platform and AI SDK empower organizations to easily construct conversational experiences atop any data repositories without being constrained by traditional, centralized data access methods. This means supporting analytical-driven questions as well as operational requests that leverage live production data in real time. All the while, the Denodo Platform enforces the necessary data security and governance policies, aligning data access and usage with organizational control, without additional manual intervention.
Finally, to drive ongoing adoption and innovations, the Denodo AI SDK and the included example chatbot application have been released for free under the Apache open-source license. It is also bundled in and distributed as part of the free Denodo Express offering. This means that anyone can download, view, and modify the AI SDK Python source code and quickly experiment and evaluate an end-to-end GenAI application today without any significant upfront cost.
Data Agents and AI Applications
While a Denodo-enabled AI agent can be extremely powerful, organizations will need more. This can be said for just about any AI agent or application on the market today. We believe the future of enterprise AI will involve multiple AI agents working together, each with its own unique skills and capabilities.
We see a future in which data agents will form a foundational pillar of this agentic world. The data agent powered by Denodo and the SDK will be a specialist agent, configured and optimized to retrieve data and interact with all structured data repositories in a scalable, optimized, and secure way. The Denodo AI SDK and the API endpoint, was built from the ground up to support the development of these data agents. Not only does it support the OpenAPI specifications today, but it also supports different interaction methods and modes.
A Denodo-powered data agent can already deliver powerful insights and interactions via platforms such as Microsoft Teams and Slack. These types of data agents can support open-ended questions rather than users relying on hundreds of pre-built PowerBI dashboards. By embracing open standards such as REST and OpenAPI, we want to make it easy for customers to deploy a Denodo data agent that can collaborate with other AI agents and systems, both now and in the future.
From Data Silos to Intelligent Data Agents
Gartner recently talked about the gap between AI-ready data requirements and the capabilities of traditional data management. The issues of data delivery and access associated with data silos in enterprise environments have long been significant challenges. These challenges have become even more pronounced with the advent of GenAI, acting as major inhibitors to the development and deployment of next-generation AI agents.
However, these traditional barriers should now be viewed as opportunities for innovation. With tools like the Denodo Platform and AI SDK, and new approaches like Query RAG, businesses can unlock transformational capabilities, enabling AI-driven insights that are both immediate and impactful.
Hear from organizations such as Perkins Coie and Festo on how they are embracing this future and using the Denodo Platform to build the needed foundation for AI-driven innovations. Better still, you can get started right now and build your very own data agent and chatbot today by downloading Denodo Express for free. Or you can explore and customize the Denodo AI SDK codebase by checking them out from the GitHub AI SDK repository.
Whether you’re embarking on your first AI journey or enhancing your current capabilities, now is the perfect time to seize these transformative tools. Transform your data strategy today and pave the way for cutting-edge solutions that will define the future of AI in your enterprise.
- Query RAG – A New Way to Ground LLMs with Facts and create Powerful Data Agents - December 16, 2024
- Welcome to the Era of Denodo Assistant - November 20, 2024
- Unlocking the Power of Generative AI: Integrating Large Language Models and Organizational Knowledge - February 22, 2024
- Query RAG – A New Way to Ground LLMs with Facts and create Powerful Data Agents - December 16, 2024
- Managing Misuse, in Dual-Use Foundation AI Models - October 31, 2024