DB-GPT

What is DB-GPT?

As large models are released and iterated upon, they are becoming increasingly intelligent. However, in the process of using large models, we face significant challenges in data security and privacy. We need to ensure that our sensitive data and environments remain completely controlled and avoid any data privacy leaks or security risks. Based on this, we have launched the DB-GPT project to build a complete private large model solution for all database-based scenarios. This solution supports local deployment, allowing it to be applied not only in independent private environments but also to be independently deployed and isolated according to business modules, ensuring that the ability of large models is absolutely private, secure, and controllable.

DB-GPT is an experimental open-source project that uses localized GPT large models to interact with your data and environment. With this solution, you can be assured that there is no risk of data leakage, and your data is 100% private and secure.

Features

Currently, we have released multiple key features, which are listed below to demonstrate our current capabilities:

SQL language capabilities
- SQL generation
- SQL diagnosis
Private domain Q&A and data processing
Database knowledge Q&A
Data processing
Plugins
- Support custom plugin execution tasks and natively support the Auto-GPT plugin, such as:
- Automatic execution of SQL and retrieval of query results
- Automatic crawling and learning of knowledge
Unified vector storage/indexing of knowledge base
- Support for unstructured data such as PDF, Markdown, CSV, and WebURL
Milti LLMs Support
- Supports multiple large language models, currently supporting Vicuna (7b, 13b), ChatGLM-6b (int4, int8)
- TODO: codegen2, codet5p

Demo

Run on an RTX 4090 GPU. YouTube

Run

SQL Generation

Generate Create Table SQL

Generating executable SQL:To generate executable SQL, first select the corresponding database and then the model can generate SQL based on the corresponding database schema information. The successful result of running it would be demonstrated as follows:

Q&A

Based on the default built-in knowledge base, question and answer.

Add your own knowledge base.

Learning from crawling data from the Internet

Introduction

DB-GPT creates a vast model operating system using FastChat and offers a large language model powered by Vicuna. In addition, we provide private domain knowledge base question-answering capability through LangChain. Furthermore, we also provide support for additional plugins, and our design natively supports the Auto-GPT plugin.

Is the architecture of the entire DB-GPT shown in the following figure:

The core capabilities mainly consist of the following parts:

Knowledge base capability: Supports private domain knowledge base question-answering capability.
Large-scale model management capability: Provides a large model operating environment based on FastChat.
Unified data vector storage and indexing: Provides a uniform way to store and index various data types.
Connection module: Used to connect different modules and data sources to achieve data flow and interaction.
Agent and plugins: Provides Agent and plugin mechanisms, allowing users to customize and enhance the system's behavior.
Prompt generation and optimization: Automatically generates high-quality prompts and optimizes them to improve system response efficiency.
Multi-platform product interface: Supports various client products, such as web, mobile applications, and desktop applications.

Below is a brief introduction to each module:

Knowledge base capability

As the knowledge base is currently the most significant user demand scenario, we natively support the construction and processing of knowledge bases. At the same time, we also provide multiple knowledge base management strategies in this project, such as:

Default built-in knowledge base
Custom addition of knowledge bases
Various usage scenarios such as constructing knowledge bases through plugin capabilities and web crawling. Users only need to organize the knowledge documents, and they can use our existing capabilities to build the knowledge base required for the large model.

LLMs Management

In the underlying large model integration, we have designed an open interface that supports integration with various large models. At the same time, we have a very strict control and evaluation mechanism for the effectiveness of the integrated models. In terms of accuracy, the integrated models need to align with the capability of ChatGPT at a level of 85% or higher. We use higher standards to select models, hoping to save users the cumbersome testing and evaluation process in the process of use.

Vector storage and indexing

In order to facilitate the management of knowledge after vectorization, we have built-in multiple vector storage engines, from memory-based Chroma to distributed Milvus. Users can choose different storage engines according to their own scenario needs. The storage of knowledge vectors is the cornerstone of AI capability enhancement. As the intermediate language for interaction between humans and large language models, vectors play a very important role in this project.

Connections

In order to interact more conveniently with users' private environments, the project has designed a connection module, which can support connection to databases, Excel, knowledge bases, and other environments to achieve information and data exchange.

Agent and Plugin

The ability of Agent and Plugin is the core of whether large models can be automated. In this project, we natively support the plugin mode, and large models can automatically achieve their goals. At the same time, in order to give full play to the advantages of the community, the plugins used in this project natively support the Auto-GPT plugin ecology, that is, Auto-GPT plugins can directly run in our project.

Prompt Automatic Generation and Optimization

Prompt is a very important part of the interaction between the large model and the user, and to a certain extent, it determines the quality and accuracy of the answer generated by the large model. In this project, we will automatically optimize the corresponding prompt according to user input and usage scenarios, making it easier and more efficient for users to use large language models.

Multi-Platform Product Interface

TODO: In terms of terminal display, we will provide a multi-platform product interface, including PC, mobile phone, command line, Slack and other platforms.

Deployment

1. Hardware Requirements

As our project has the ability to achieve ChatGPT performance of over 85%, there are certain hardware requirements. However, overall, the project can be deployed and used on consumer-grade graphics cards. The specific hardware requirements for deployment are as follows:

GPU	VRAM Size	Performance
RTX 4090	24 GB	Smooth conversation inference
RTX 3090	24 GB	Smooth conversation inference, better than V100
V100	16 GB	Conversation inference possible, noticeable stutter

2. Install

This project relies on a local MySQL database service, which you need to install locally. We recommend using Docker for installation.

$ docker run --name=mysql -p 3306:3306 -e MYSQL_ROOT_PASSWORD=aa12345678 -dit mysql:latest

We use Chroma embedding database as the default for our vector database, so there is no need for special installation. If you choose to connect to other databases, you can follow our tutorial for installation and configuration. For the entire installation process of DB-GPT, we use the miniconda3 virtual environment. Create a virtual environment and install the Python dependencies.

python>=3.10
conda create -n dbgpt_env python=3.10
conda activate dbgpt_env
pip install -r requirements.txt

3. Run

You can refer to this document to obtain the Vicuna weights: Vicuna .

If you have difficulty with this step, you can also directly use the model from this link as a replacement.

Run server

$ python pilot/server/llmserver.py

Run gradio webui

$ python pilot/server/webserver.py

Notice: the webserver need to connect llmserver, so you need change the .env file. change the MODEL_SERVER = "http://127.0.0.1:8000" to your address. It's very important.

Usage Instructions

We provide a user interface for Gradio, which allows you to use DB-GPT through our user interface. Additionally, we have prepared several reference articles (written in Chinese) that introduce the code and principles related to our project.

Multi LLMs Usage

To use multiple models, modify the LLM_MODEL parameter in the .env configuration file to switch between the models.

Acknowledgement

The achievements of this project are thanks to the technical community, especially the following projects:

Contributors

This project follows the git-contributor spec, auto updated at Fri May 19 2023 00:24:18 GMT+0800.

Licence

The MIT License (MIT)

Contact Information

We are working on building a community, if you have any ideas about building the community, feel free to contact us. Discord

from Hacker News https://ift.tt/PvmNSn6

SymmetricalDataSecurity

Monday, May 22, 2023

Show HN: DB-GPT, an LLM tool for database