---
title: "LocalGPT: Convert Your Chats with Docs to Fine-tuning Datasets"
source_url: https://youtu.be/2_o6epQToVY
captured_at: 2025-05-17
author: "Prompt Engineering"
contributor: "ชิดชนก"
platform: youtube
---
# LocalGPT: Convert Your Chats with Docs to Fine-tuning Datasets

youtube video by **Prompt Engineering**, 15m02s, uploaded 2024-01-22. URL: https://youtu.be/2_o6epQToVY
Tags: prompt engineering, Prompt Engineer

## Shared in LINE Note+
- 2025-05-17 08:01 by ชิดชนก

## Description
You can use localGPT to create custom training datasets by logging RAG pipeline. You can use pre-configure Virtual Machine to run localGPT here:

💻  (use Code: PromptEngineering for 50% off).  

🦾 Discord: 
☕ Buy me a Coffee: 
|🔴 Patreon: 
💼Consulting: 
📧 Business Contact: engineerprompt@gmail.com
▶️ Become Member: 
💻 Pre-configured localGPT VM:  (use Code: PromptEngineering for 50% off).  

LINKS:
localGPT repo: 
LocalGPT Playlist: 


TIMESTAMPS:
[00:00] Introduction 
[01:29] Pre-configure Virtual Machine for LocalGPT
[03:13] Setting Up the Virtual Machine
[07:28] Creating a Vector Store with LocalGPT
[09:23] Logging User Interactions for training


All Interesting Videos:
Everything LangChain: 

Everything LLM: 

Everything Midjourney: 

AI Image Generation:

## Transcript (captions:auto:en-orig)
did you know that you can use local GPT to create training data sets for fine-tuning your local llms if you're not familiar with local GPT it's my own project that lets you chat with your documents on your local machine it already has more than 18,000 stars in GitHub with over 2,000 Forks local GPT enables you to chat with your documents in different formats but there is a little known feature that you can use used to create training data sets if you're new to the local GPD project I have a playlist of all the videos related to the project link is going to be in the description the idea behind this is very simple local GPT lets you create a vector database of your knowledge base then when the user asks a question local GPT will generate a response based on the llm that you have chosen however you can also store both the query and response pairs that means that if you have enough interaction with your files you can use this data to fine-tune a model that is specifically trained on your own documents I think this is going to be extremely helpful in cases where you have a lot of users who are using local GPT and you uh storing the question response pairs in a local storage let me show you how to do this within local GPT using a single command line argument but before that if you want to run local GPT you will need access to a GPU now depending on the model that you want to run you will need different amount of vram in case if you don't have access to a computer with a GPU you can use different Cloud providers I have partnered with a company called M compute and you can rent one of their virtual machines which comes preconfigured with local GPT if if you use Link in the description below you will get access to this preconfigured virtual machine that has an a6000 GPU which has 48 GB of vram this is enough to run 70 billion model in 4bit quantization it comes with local GPT visual code Studio and cond pre-installed now the best part is that you can use this virtual machine for any of your own projects it doesn't have to be restricted to Lo GPT currently you're going to be paying $4 for 3 hours dedicated usage however if you use my code prompt engineering without space you're going to get 50% off of this price I will get a small commission if you decide to use this virtual machine if you decide to use this you will need to download the thin link client in order to access the virtual machine tending client also works on the M1 and M2 series T client also works on the M series of Max but you will need to install Rosetta now if you decide to use this virtual machine using thin link you can set it up by providing the server ID as well as the password that MK computer is going to send you once you reserve your viral machine you can also en able the transfer files between your local machine as well as this virtual machine so let me show you how to do that so first click on options then click on local devices then click on drives then click on details here you can actually add the path of the folder that you want to share between both your local machine as well as the uh remote virtual machine so for example if I I click on prompt this is my uh local machine within my documents I created a folder called thin link this is going to be shared between my local device and the virtual machine so just click okay and now I need to provide my server IP address as well as the password that mask compute has sent me once you log into the virtual machine here is the interface that you're going to see so on a desktop you have the local GPD folder which is cloned from the GitHub repo and there is a welcome doc that I would highly recommend to read before start using the uh VM this provides all the information that you need to know in order to start using local GPT on this virtual machine as I said it comes preconfigured with a local GPT code base from GitHub there is also cond preconfigured with visual code Studio the best thing is that you can use this virtual machine for anything you want it doesn't just have to be related to local GPT now here's a stepbystep process of how to get started with local GPT we are going to look at this in a little bit however you want to look at the frequently asked questions there is just one small issue with the copy paste functionality if you're using this on Linux you can use contrl C and contrl V on Mac I believe it's command C and command V the only other thing you need to keep in mind is that if you're copying anything to the terminal on the virtual machine then you need to use control shift and V they highly recommend to use the thin link client in order to access the virtual machine but you can also use your browser as well although that's uh not recommended by MK compute visual code studio is preconfigured on this virtual machine and it will open the local GPT folder they have also included a virtual environment uh called local GPT which you can directly activate here so we're going to use cond activate local GPT and that will activate the local GPT virtual environment you will need to do one extra step before you start using the local GPT you'll need to manually install all the requirements so we're going to just copy this go back to the terminal and we'll paste that you can run this this will start installing all the requirements I'm working with the mass compute team to have a virtual environment preconfigured with these requirements so you don't have to do this step next let me show you how to use local GPT to log user interactions and create a data set that you can use to fine tune your llms in this case we're going to be using the r paper as our source document this comes preconfigured with local GPT project a side trick by default you will not be able to see a PDF preview but what you can do is you can go to extensions look for PDF and just install the PDF viewer extension that will enable you to open PDF files within visual code Studio I personally find this very helpful before we start interacting with this file we need to create a vector store and for that we're going to be using the injust dopy so in order to ingest our document we're going to use the command python injust dopy when we run this command it will create a vector store that has the um embeddings of different text chunks uh from this document we have a total of 193 chunks the paper itself is 51 pages long and it will create a DB folder for us that has the chroma DB Vector store next we're going to use the Run local gbt file to start interacting with this Vector DB within the Run local gbd file we have this new Option save QA this will enable you to save your query as well as the model response before running the uh local GPT file let's look at what model we are using so by default this uh virtual machine is going to be using the LMA 2 7 billion parameter model and this is the unquantized version however if you decide to use a quantise model I'll highly recommend to use the gptq file format then rather than the GG UF file format and the reason is that the gptq quantise model are well optimized for NVIDIA gpus so you will get better performance if you use the gptq models on this a6000 GPU and as I said in the beginning you will be able to run up to 70 billion parameter model if you decide to go with 4 bit quantization on this virtual machine in order to enable logging of the user question and answers we're going to use this extra command with run local GPT and that is going to be D- savecore QA and let's run this I already downloaded the model that's why it's simply loading the checkpoints however in you in your case it will first download the model the download speed is pretty good I think it has a link of 1 megabits per second so that …