---
title: "Run ALL Your AI Locally in Minutes (LLMs, RAG, and more)"
source_url: https://youtu.be/V_0dNE-H2gw
captured_at: 2025-05-18
author: "Cole Medin"
contributor: "ชิดชนก"
platform: youtube
---
# Run ALL Your AI Locally in Minutes (LLMs, RAG, and more)

youtube video by **Cole Medin**, 20m19s, uploaded 2024-09-16. URL: https://youtu.be/V_0dNE-H2gw
Tags: ai, artificial intelligence, ai agents, software engineering, software development, coding, automation, saas, development, rag, rag ai agent, llm rag, ai rag, ai agent chatbot, n8n, n8n rag, supabase rag, n8n ai agent, n8n ai, production ready ai, n8n workflow, vector db, local ai, local ai agent, local rag, docker, ollama, ollama rag, qdrant, qdrant rag, llama 3.1, llama 3, llama 3 rag

## Shared in LINE Note+
- 2025-05-18 23:15 by ชิดชนก

## Description
Update! Follow up video for deploying this app to the cloud! 



Artificial Intelligence is no doubt the future of not just software development but the whole world. And I'm on a mission to master it - focusing first on mastering AI Agents.

In this video I show you an INCREDIBLE packaged local AI solution with Ollama for the LLMs, Qdrant for RAG, Postgres for the SQL database, and n8n for the no code workflow automations. This package is SO easy to set up and use! I walk you through the setup quick and then even show you how to use it to build a RAG AI Agent that runs entirely locally!

One really important thing to keep in mind is that while I created a no code AI agent with n8n in this video, this whole local AI setup could be used to actually code your AI agents! It doesn't have to be "no code" n8n for the actual agent. A custom coded RAG AI agent with n8n for the accompanying workflow automations (agent tools!) is another fantastic use of this local AI package.

Hardware: I am actually using a laptop for this video - with a 3070 laptop GPU and an i7-12700H CPU with 14 cores and 20 threads. So nothing crazy at all! It takes up to 5 minutes for me to get a response from Llama 3.

## Transcript (captions:auto:en-orig)
have you ever wished for a single package that you could easily install that has everything you need for local AI well I have good news for you today because I have exactly what you are looking for I have actually never been so excited to make a video on something before today I'm going to show you an incredible package for local AI developed by the n8n team and this thing has it all it's got old llama for the llms quadrant for the vector database postgress for the SQL database and then n8n to tie it Al together with workflow automations this thing is absolutely incredible and I'm going to show you how to set it up in just minutes then I'll even show you how to extend it to make it better and use it to create a full rag AI agent in n8n so stick around because I have a lot of value for you today running your own AI infrastructure is the way of the future especially because of how accessible is becoming and because open-source models like llama are getting to the point where they're so powerful that they're actually able to compete with close Source models like GPT and clad so now is the time to jump on this and what I'm about to show you is an excellent start to doing so and at the end of this video I'll even talk about how I'm going to extend this package in the near future just for you to make it even better all right so here we are in the GitHub repository for the self-hosted AI starter kit by n8n now this repo is really really basic and I love it there are basically just two files that we have to care about here we have our environment variable file where we'll set credentials for things like postgress and then we have Docker compose the caml file here where we'll basically be bringing in everything together like postgress quadrant and olama to have a single package for our local AI now the first thing that I want to mention here is that this read me has instructions for how to install everything yourself but honestly it's quite lacking and there's a couple of holes that I want to fill in here with ways to extend it to really make it what I think that you need and so I'll go through that a little bit and we'll actually get this installed on our computer now there are a couple of dependencies before you start basically you just need git and Docker so I'd recommend installing GitHub desktop and then Docker desktop as well because this also has Docker compose with it which is what we need to bring everything together for one package so with that we can go ahead and get started downloading this on our computer so the first thing you want to do to download this code is copy the get clone command here with the URL of the repository you'll go into a terminal then and then paste in this command for me I've already cloned this that's why I get this error message but you're going to get this code downloaded on your computer and then you can change your directory into this new repository that you've pulled and so with this we can now go and edit the files in any editor of our choice I like using VSS code and so if you have VSS code as well you can just type in code Dot and this is going to pull up everything in Visual Studio code now the official instructions in the readme that we just saw would tell you at this point to run everything with the docker compos post command now that is not actually the right Next Step I'm not really sure why they say that cuz we have to actually go and edit a couple of things in the code to make it customized for us and that starts with the EnV file so you're going to want to go into your EnV file I've just made a env. example file in this case because I already have my credentials set up so you'll go into your EnV and then set up your postgress username and password the database name and then also a couple of n8n Secrets these can be whatever you want just make sure that they are very very here and basically just a long alpha numeric string and then with that we can go into our Docker compose file and here's where I want to make a couple of extensions to really fill in the gaps so the couple of things that were missing in the original Docker compose file first of all for some reason the postgress container doesn't have the port exposed by default so you can't actually go and use postgress as your database in an NN workflow I think n uses postgress internally which is why it's set up like that initially but we want to actually be a to use postgress for our chat memory for our agents and so I'm going to show you how to do that basically all you have to do is go down to the postgress service here and then just add these two lines of code right here ports and then just a single item where we have 5432 map to the port 5432 inside the container and that way we can go Local Host 5432 and access postgress so that is super super important otherwise we won't actually be able to access it within an NA end workflow we're going to be doing that later when we build the rag AI agent now the other thing that we want to do is we want to use olama for our embeddings for our Vector database as well now the base command when we initialize olama is just this part right here so we sleep for 3 seconds and then we pull llama 3.1 with oama so that's why we have llama 3.1 Available To Us by default but what I've added here is another line to pull one of the olama embedding models and we need this if we want to be able to use AMA for our Rag and so I've added this line as well that is very very key so that is literally everything that you have to change in the code to get this to work and I'll even have a link in the description of this video to my version of this you can pull that directly if you want to have all the customizations that we just went over here and with that we can go ahead and actually start it with Docker compose and so the installation instructions in the readme are actually kind of useful here because there's a slightly different Docker compose command that you want to run based on your architecture so if you have a Nvidia G you can follow these instructions which are a bit more complicated but if you want to you can and then you can run with a GPU Nvidia profile and then if you are a Mac User you follow this Command right here and then for everyone else like what I'm going to use in this case even though I have a Nvidia GPU I'll just keep it simple with Docker compose d-profile CPU up and so we'll copy this command go into our terminal here and paste it in and in my case I have already created all these containers and so it's going to run really really fast for me but in your case it's going to have to pull each of the images for olama postgress n8n and quadrant and then start them all up and it'll take a little bit because I also have to do things like pulling llama 3.1 for the old llama container and so in my case it's going to blast through this pretty quick because it's already done a lot of this I did that on purpose so it can be a quicker walkthrough for you here um but you can see all the different containers the different colors here that are running everything to set me up for each of the different services and so like right here for example it pulled llama 3.1 and then right here it pulled the embedding model that I chose from AMA as well um and so at this point it's basically done so I'm going to pause here and come back when everything is ready all right so everything is good to go and now I'm going to actually take you in a Docker so we can see all of this running live so you're going to want to open up your Docker desktop and then you'll see one record here …