Dify.AI x TiDB: Build Scalable AI Agent with Knowledge Base

Introduction

Building AI Agents can be an intricate affair, but it doesn’t have to be with the right tools. The combination of TiDB’s new Serverless Vector Search, now welcoming users in its Public Beta phase, alongside Dify’s intuitive low-code platform, simplifies the creation of scalable, efficient AI solutions. This guide offers a clear pathway to seamlessly integrating TiDB and Dify, enabling you to harness robust vector search capabilities to power your AI agents.

Why This Matters

Accelerated Development: Bypass the complexities of traditional coding and fast-track the creation of AI agents with tools designed for speed and simplicity.
Optimized Retrieval: Leverage TiDB Serverless Vector Search for quick, precise information retrieval that significantly boosts your AI’s responsiveness and relevancy.
Tailored Scalability: Adjust effortlessly to fluctuating data volumes and computational demands thanks to TiDB’s flexible architecture.

What You’ll Discover

Effortless Setup: Learn how to establish a TiDB Serverless cluster swiftly.
Seamless Deployment: Explore the steps to deploy Dify using Docker Compose for a smooth start.
Knowledge Base Mastery: Master the art of constructing and managing an effective knowledge base.
AI Agent Innovation: Unleash the potential of AI agents using Dify’s streamlined toolset.

Prerequisites

Create a TiDB Serverless Cluster

First, let’s create a TiDB Serverless cluster by following the steps below:

Log in to TiDB Cloud console.
Enter Create Cluster page and configure your new cluster:
- Choose Serverless type of cluster
- Select the region closest to where your application is deployed, for example: Tokyo (ap-northest-1)
- Fill in your cluster name
Click Create button to complete cluster creation.

After the cluster is created, you need to create a database schema and execute the following SQL statement on the SQL Editor of TiDB Cloud:

CREATE SCHEMA dify;

Finally, we can click on the upper right corner of the cluster Overview page Connect button to obtain the database connection parameters and save them for subsequent steps.

Deploy Dify Using Docker Compose

If individuals want to use Dify, there are two deployment methods: cloud service and Self-hosted Community Edition.

Dify Cloud currently does not support the configuration of vector databases, so if you want to use TiDB Serverless as its underlying vector database, you can use the community open source version Self-hosted. However, the good news is that the TiDB team is in close communication with the Dify team to discuss possible cooperation options.

We recommend using Docker Compose for Dify deployment. Please refer to the specific operation steps. The following is a brief step:

First, clone Dify’s open source repository locally:

git clone https://github.com/langgenius/dify.git
cd dify/docker
cp .env.example .env
vi .env

In order to store TiDB Serverless as a vector for Dify, we need to modify .env in file Environment variables:

# Will VECTOR_STORE change into tidb_vector, the default value is weaviate
VECTOR_STORE: tidb_vector

# Modify the following environment variables to the previously saved connection TiDB parameter
TIDB_VECTOR_HOST: gateway01.***.prod.aws.tidbcloud.com
TIDB_VECTOR_PORT: 4000
TIDB_VECTOR_USER: ********.root
TIDB_VECTOR_PASSWORD: ********
TIDB_VECTOR_DATABASE: dify

After the modification is completed, execute the following startup command:

docker compose up -d

After successful deployment, enter http://localhost in the browser. You can access Dify now.

Build a Knowledge Base Based on TiDB

In the above preparation work, we have configured TiDB Serverless, so how to use Dify to upload documents and enter them into the vector database, and how to configure the Embedding and Reranker models?

Configuration Model Provider

First, in Dify’s “Settings ->Mode Provider”, add a new model and configure Token authorization, for example: use OpenAI as the dialogue model and Embedding model, and use Cohere as the Reranker model.

Upload Documents

Then, visit http://localhost/datasets/create, create a new blank knowledge base, and upload your documents to the knowledge base.

Dify will help you automatically build vector indexes for uploaded documents and store them in TiDB Serverless vector storage.

Configure Retrieval Model

After the document is uploaded, you can further set the Embedding and Reranker models on the settings page of the knowledge base:

On the knowledge base configuration page, we can see that Dify provides three search modes for the knowledge base:

Vector Search: Retrieval based on ANN query, Reranker model is optional.
Full Text Search: Based on BM25 retrieval, the Reranker model is optional (currently not supported by TiDB Vector type data sources)
Hybrid (vector + full text): ANN + BM25 search, Reranker model is required.

In terms of function implementation, Dify has enabled TiDB Serverless Vector Search Index function by default, allowing us to complete the search within hundreds of thousands of documents in millisecond level.

Create AI with Dify Chatflow

After configuring the knowledge base settings and uploading the documents, we can start creating our AI chat assistant.

Select to create a blank application in “Studio” and select “Chatflow” orchestrate method, then set app icon, name and description information:

After entering the chat assistant Chatflow design page, we can find that Dify has initialized the basic chat workflow for us.

Click the “+” button between the “Start” node and the “LLM” node to add a “Knowledge Retrieval” node, and add the knowledge base we just created to the “Knowledge” option of the node.

Click the “LLM” node and set its “Context” option to the running result of “Knowledge Retrieval” (result array). Write the “SYSTEM” prompt, insert the “Context” variable into the prompt and add the following requirements for response to it:

Consider contextual information and answer questions without relying on prior knowledge.

Click the “Debug and Preview” button in the upper right corner, we can open the preview dialog box, in the preview dialog box we ask a question to trigger the AI to execute the Chatflow you just designed to answer questions.

After completing the above operations, we have used Dify to create a chat assistant that answers questions based on the knowledge base you upload.

If you want to use it on other platforms or websites, you can click “Publish” in the upper right corner. Dify supports by <script> or <iframe> tag mode, as well as API calling method to embed the chat on your site.

Why Choose TiDB Vector Search?

Dynamic scalability: TiDB provides elastic scalability to help application developers easily deal with the dynamic and unpredictable data growth or workload growth needs of AI applications.
All-in-one database: In addition to supporting Hybrid Transaction/Analytical Processing (HTAP), TiDB can now also be used as a vector storage, which can meet the complex and diverse data storage and query needs of GenAI applications such as document storage, graph data storage, and conversation message storage.
SQL compatibility: Developers can use familiar SQL CLI tools, application drivers, and GUI tools to manage vector data and perform semantic search queries.

Why Choose Dify?

Low code, high efficiency: Dify provides a visual workflow orchestration system that can help users quickly implement ideas and build customized AI workflows.
Powerful ecology, ready to use out of the box: Dify integrates many model providers, data sources and tools, and users only need simple configuration to complete the docking of different third-party platforms.
Easy to use, collaborative and co-creative: The emergence of Dify lowers the threshold for users to build enterprise-level AI applications, providing enterprises with capabilities from Agent construction to AI workflow orchestration, RAG retrieval, model management, etc., to easily build and operate generative AI native applications.
Model neutral, embrace open source: Dify emphasizes the importance of model neutrality and open source ecosystems, focusing on solving the cost, security and observability challenges that enterprises face when deploying multi-modal and multi-model applications.

Learn More

There are some examples to show how to use the pingcap/tidb-vector-python to interact with TiDB Vector in different scenarios.

OpenAI Embedding: use the OpenAI embedding model to generate vectors for text data, store them in TiDB Vector, and search for similar text.
Image Search: use the OpenAI CLIP model to generate vectors for image and text, store them in TiDB Vector, and search for similar images.
LlamaIndex RAG with UI: use the LlamaIndex to build an RAG(Retrieval-Augmented Generation) application.
Chat with URL: use LlamaIndex to build an RAG(Retrieval-Augmented Generation) application that can chat with a URL.
GraphRAG: 20 lines code of using TiDB Serverless to build a Knowledge Graph based RAG application.
GraphRAG Step by Step Tutorial: Step by step tutorial to build a Knowledge Graph based RAG application with Colab notebook. In this tutorial, you will learn how to extract knowledge from a text corpus, build a Knowledge Graph, store the Knowledge Graph in TiDB Serverless, and search from the Knowledge Graph.
Vector Search Notebook with SQLAlchemy: use SQLAlchemy to interact with TiDB Serverless: connect db, index&store data and then search vectors.
Build RAG with Jina AI Embeddings: use Jina AI to generate embeddings for text data, store the embeddings in TiDB Vector Storage, and search for similar embeddings.
Semantic Cache: build a semantic cache with Jina AI and TiDB Vector.