---
title: "LocalGPT API: Build Powerful Doc Chat Apps"
source_url: https://youtu.be/KBrT0czrtAs
captured_at: 2025-05-17
author: "Prompt Engineering"
contributor: "ชิดชนก"
platform: youtube
---
# LocalGPT API: Build Powerful Doc Chat Apps

youtube video by **Prompt Engineering**, 14m42s, uploaded 2023-08-13. URL: https://youtu.be/KBrT0czrtAs
Tags: prompt engineering, Prompt Engineer, natural language processing, GPT-4, chatgpt for pdf files, ChatGPT for PDF, langchain openai, langchain in python, embeddings stable diffusion, Text Embeddings, langchain demo, langchain, langchain javascript, gpt-3, openai, vectorstorage, chroma, train gpt on your data, train openai model, embeddings, localGPT, Llama-2, localGPT and Llama2, llama-2

## Shared in LINE Note+
- 2025-05-17 08:12 by ชิดชนก

## Description
In this video, I will show you how to use the localGPT API. With localGPT API, you can build Applications with localGPT to talk to your documents from anywhere you want. In this example application, we will be using Llama-2 LLM. 

💻 Pre-configured localGPT VM:  (use Code: PromptEngineering for 50% off).  


#llama #localgpt #llm 
▬▬▬▬▬▬▬▬▬▬▬▬▬▬ CONNECT ▬▬▬▬▬▬▬▬▬▬▬
☕ Buy me a Coffee: 
|🔴 Support my work on Patreon: Patreon.com/PromptEngineering
🦾 Discord: 
▶️️ Subscribe: 
📧 Business Contact: engineerprompt@gmail.com
💼Consulting: 
▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬
LINKS:
LocalGPT Github: 
LocalGPT Video: 
▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬
Timestamps:
Intro: [00:00]
Setting up localGPT Repo: [01:33]
Llama-2 and other LLMs: [03:18]
Running the API: [04:25]
Running the GUI with API: [05:00]
The GUI Tutorial: [06:15]
What exactly is happening: [08:37]
Add more docs to the Vector Store: [09:05]
Code Walkthrough: [10:30]
Outro: [14:25]
▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬
All Interesting Videos:
Everything LangChain: 

Everything LLM: 

Everything Midjourney: 

AI Image Generation:

## Transcript (captions:auto:en-orig)
How to serve local GPT to create powerful application using local GPT API? That's what we're going to be exploring in this video. The idea is very simple. Just like you can build applications using chat GPT API, we can replace chat GPT API with local GPT API. That way you are not sharing your information or your data with anyone. In this case, you're going to be putting your local GPT instance somewhere in the cloud on a powerful machine, and then making API calls to it in order to build applications on top of it. So, that way your client does not really need to have a powerful GPU in order to run applications. And you can also scale it to multiple computers or machines. So, in this video, I'm going to walk you through a step-by-step process of how the local GPT API is is implemented, and then I'm going to show you example application that is built on top of the local GPT API. For those of you who are not familiar, local GPT is my own project that lets you chat with your documents locally using the power of open-source large language models. Now, thanks to the amazing contributors, we have already passed 14,000 stars, and the project has been featured on trending GitHub repos a couple of times. Local GPT API was developed by Jim, who is an amazing developer. Do check out his projects. In this video, I'm going to show you how to set up the local GPT API on your system, and then we will run uh an example application that's built on top of it. Now, if you're new to the local GPT project, I would actually recommend you to watch this video to get a better understanding of how the underlying code works. So, we'll start off by cloning the repo. So, for that, you need to go click on this green button, then copy the link to the repo. Now, I have already cloned this repo, but I'm going to simply walk you through the process of how to do this again. So, you need to have Git installed on your computer. Now, in order to clone the repo, you need to use the Git clone command, then provide the link to the repo, and then you can optionally provide the name of the folder where you want to store the cloned repo. I have already done this, so I'm not going to do this again. Once you're done with this, next we need to create a virtual environment. So, again, we're going to use conda create, and then {dash}n, then the name of the virtual environment, so I'm going to call this local GPT {underscore} API, and then the Python version that you want to use. So, I like to use Python 3.10.0. I have already created this virtual environment. So, it simply tells me that do I want to uh remove the existing environment? So, I'm going to say no. Right? But, once you create the virtual environment, then you need to activate it. So, we're going to use conda activate, and then the name of the virtual environment that we just created. So, and you see that we're now within this new virtual environment. After that, you need to install all the required packages, and for that, you can use the pip install {dash}r requirements.txt file. And that will install all the required packages. I have already done so, so I'm not going to do this again. So, we are all set for running the API as well as the GUI. But, in order to run this, I'm going to be going back and referring to the GitHub tutorial. So, in this case, we're going to be looking at this section, run the UI. Now, before that, I want to highlight a couple of changes that has been made. Now, the model definition has been moved to constant.py. Now, if you're not familiar uh with local GPT project, and you're not sure what I'm talking about, I would recommend to check out the earlier videos. But, for this video, you don't really need to know all of that. Uh the only thing is that the more the LLM that you are going to be using has to be defined within the constant.pi file, right? So, here I'm using the llama.cpp 7B chart model the GGML format. If you have an Nvidia GPU, I would actually recommend you to use the GPTQ format for quantized model rather than GGML. There are some issues that we need to figure out and fix. But, that would be my recommendation for Nvidia users. I just want to reiterate that this project is under active development. So, things will change. Now, let me show you how to run the API and then I'll walk you through the code. In order to run the API, need to be in the root folder of the local GPT project and then you need to simply type python run score local GPT and then we're going to use the API file and this will start running the API. Now, this will take a little bit of time but then it will load and you will see something like this. So, it's actually running on this specific address which is the local host address with a port 5110. After that, I'm going to open another terminal. So, here I need to again activate the virtual environment and that was local GPT and the score API. Now, in this case we need to actually move to the second folder. So, this is local GPT API. I'm going to type in Let me just do this. Okay. Let's type in CD and then local GPT UI folder. Right? And within this folder if you see, there is one python file and then some other folders and this is the python file that we need to run. This is basically a UI for local GPT that is using the local GPT API. I'm going to walk you through the code in a little bit, but just want to show you a quick demo of what is going on. Oh, okay. I I'm actually already running this in another terminal. So, I'm going to go and close that and let's go back and run this again. So, once you run this, you're going to see this message, which again, it's running on my local machine, but now the port number is different than where the API is running. Now, we simply need to copy this link and go back to our browser. So, simply put this in your browser and it will show you this user interface. Now, the goal of this UI is to show you how you can use the local GPT API to build your own applications. Uh so, we put together a very simple interface, but you can uh add a lot more to it if you want. Now, there is a pretty nice and detailed tutorial on how to use this application. Uh so, you can basically download the constitution.pdf file or there are some news articles that Jim has put together. So, if you click on this, this will also download some news articles for you that you can use. Now, in terms of the basic functionality, this is what you're going to do. So, you can simply click on the upload button and select the file that you want to upload. So, for example, for the first document, we are going to be simply uploading the constitution.pdf file. I'm going to just upload that. Now, you can simply click click on add. That will add your document to the uh knowledge base. Now, there is a small issue right now, which uh we need to fix, but uh every time you add a new document, it will simply recreate the whole knowledge base. You can also reset it, so that will delete the existing knowledge base or you can cancel it. So, in this case, I'm going to just add it and then it will start the ingestion process. And you can actually see uh the ingestion process on the Visual Code Studio. So, right now it says the directory does not exist, so it will create a persistent uh directory where it's going to put the vector store. So, you can see that it says that it will put the vector DB in the DB folder, right? So, it's going to go through this process. I'm going to walk you through the code in a little bit. So, once that is done, you can start asking questions. For example, I said, "What is this document about?" Click on search, and it will start creating the data. And here is the response that we got. So, here's our query. Then the response, which says that that this document appears to be the …