---
title: "How to Build a 3D Chatbot with ChatGPT & ElevenLabs"
source_url: https://youtu.be/EzzcEL_1o9o
captured_at: 2025-09-15
author: "Wawa Sensei"
contributor: "ป้าเกษตร"
platform: youtube
---
# How to Build a 3D Chatbot with ChatGPT & ElevenLabs

youtube video by **Wawa Sensei**, 25m26s, uploaded 2023-10-13. URL: https://youtu.be/EzzcEL_1o9o
Tags: threejs, three js tutorial, threejs tutorial, react three fiber, react three fiber tutorial, three js react, three js, three.js, react, threejs journey, threejs course, react three fiber ultimate guide, threejs for beginners, how to build a chatbot, chatgpt chatbot javascript, elevenlabs chatbot, 3D chatbot tutorial, chatgpt api, elevenlabs api tutorial, elevelenlabs nodejs, chatgpt nodejs, oculus

## Shared in LINE Note+
- 2025-09-15 01:47 by ป้าเกษตร — note: บ้านบ้าน

## Description
Let's learn how to build a 3D Chatbot powered by AI with ChatGPT, ElevenLabs, and ReadyPlayerMe Avatars.

In this project we will create a virtual girlfriend chatbot, but feel free to use what you'll learn to build your own support assistant, language teacher, or any ideas you have in mind!

To understand how the lipsync system work, watch my previous video 👉 

Final code 🔗
Frontend 👉 
Backend 👉 

#threejs #chatgpt #chatbot 

▬▬▬▬▬▬ Useful L I N K S  🔗  ▬▬▬▬▬▬ 

🚀 Learn React Three Fiber with my complete course ✨


ElevenLabs


ElevenLabs node package


OpenAI API Key


OpenAI node package


ReadyPlayerMe morphTarget documentation


Download RhubarbLibrary


▬▬▬▬▬▬ T I M E S T A M P S ⏰  ▬▬▬▬▬▬

00:00 - Intro
00:54 - Project presentation
02:01 - ReadyPlayerMe Avatar
04:18 - Packing multiple Mixamo animations
08:49 - MorphTargets facial expression
13:18 - Backend
17:04 - ChatGPT
19:57 - ElevenLabs
24:56 - Thank you!

▬▬▬▬▬▬ Support the channel 🙏   ▬▬▬▬▬▬ 

Become a member to get access to awesome perks:


▬▬▬▬▬▬ Connect with me 👋   ▬▬▬▬▬▬ 

💻 The Discord Community


📸 Instagram : 


🎎 Facebook :


🐦Twitter : 


🐦TikTok : 


💻 My website :

## Transcript (captions:auto:en-orig)
hello I'm Emma nice to meet you I'm your new AI powered girlfriend I'm a very funny person I love coding and dancing what about you you are a lot to ask me questions and improvements about the lip sync video that has been recorded months ago so I decided to go one step further will make it more natural at awesome facial expressions but best of all I will show you how to connect it to J GPT and 11 Labs API you can use the final code to start your billion dooll AI chatbot company but please don't forget me when it happens and consider hitting the join button on my YouTube memberships are you ready to learn how to build an AI powered full stack application the code for this project is splitted into two repository one for the front end and one for the back back end I will put both Link in the description let's start by cloning the front end install the dependencies with the yarn and run yarn Dev this is what you will get on the screen you will have that Avatar here you can see it blinks you have multiple settings to change for example the animation played but you can also change the fal expression to one that I configured and I will teach you how to create your owns if we go into app.jsx you can see we have UI hidden and Leva that is visible and when you save you are not anymore in the setup mode but more in the chatbot production ready mode so you can zoom in zoom out you have the chat here we will try it later and you can switch the camera so it's a green screen that you can use if you want to make video like I did for the intro let's go back to development setup mode for now and let's start diving into this project first we will talk about the Avatar if you are familiar with my YouTube videos you know I've been using Ready Player me a lot because it has out of the box avatars ready go to your account enter Hub or log in and create your account then when you have your avatars you can click on the download button but just copy Avatar link on our Avatar we need something called morph Target this is what we will use to animate the facial expression and when we will simulate with lip sync when our character talks the one starting with with vism are coming from Oculus LIF Sy and all the other ones are from AR kit and they are more for facial emotions and sentiments than just speaking out of the box if we use the URL IOP head we have only the mouse smile morph Target but we need more so if we go to the documentation we have get 3D avatar it's the same URL we are using right now you can see its models Ready Player me and the Avatar URL there are a lot of query parameters that we can use to alter the file that we will be downloading from Ready Player me you can see that we have more targets available we need to scroll down to see the different values we can use let's search for morph targets and we have the different options and the one we will need is morph targets AR kit and Oculus VM so copy it and paste it at the end of the URL now hit enter and it downloaded the file contain the morph Target that we will require now we have our file we go and drag and drop it into public models then we create a component named avatar. jsx containing all the different skin mesh to be able to interact with that component usually I show you how to do it with the CLI doing npx gtf jsx but I had a lot of comment that people have issues when they do it so the alternative solution is to use the online version of it so you can drag and drop your file here and it will generate the component for you you just have to copy paste it rename model into Avatar add models in the path if you follow the path that I'm using and you're ready to go I won code in real time so if there are thing that you are missing you can watch my previous video about lip sync now to finish preparing our Avatar we will need also to get access to the different animations we'll be using mix Amo go to mix Amo hit login or create your account for free then we will need to upload our character but it only support fbx OBG or zip so we need to convert our Gelb to fbx to do it go to blender create new file General hit a to select everything X to delete okay then you're good you can just import the file so go here glb select your glb file double click and file export fbx and you can keep the default setting so minus D forward and Y up should be fine then export fbx then drag and drop your file into the upload character you should see your character animated it has been Auto rigged with the arat that is already present from Ready Player me hit next and select the different animations that you want once you have them hit download and just be sure to not select with skin without skin so we have only the animation data we don't need to redownload the mesh hit download do it for all the animations that you want to have in your project again be sure it is without skin and hit download usually what I did was to have all the animation separated and using the fbx file but this time we will import all of them into one single file and a glb file so let's see how to do it hit file new general don't save you reselect everything delete and now we will import our first fbx animation we have old man Idol we don't need to do anything just import fbx then on the bottom expanded you should see the Armature if we play we have the animation playing only with the bones we go on the bottom left here dub cheet and action editor now we have our animation that is loaded here but the name is mix.com layer zero so name it with the name you want to have it inside your project we have old man law thing here so this is good for one animation but let's see how to add a second animation we need to import the second one file import and fbx sitting Lo thing so it's a different one but what it did is it added another Armature we can just get rid of it right click and delete hierarchy now if we go on the first Armature we go back into dub sheet and we select the new animation we can play we know it's this one the name is incorrect so it was sitting man I don't remember exactly and hit push down now we have only one Armature in our file if we go into nonlinear animation we have our two animations inside that single Armature if we go here animation nla track we have two tracks with the good animation name once you are good with all your animations or you can just save it and add other animations later hit file export and this time we will export it as a glb go to public models and you can name it animations. glb the format it will be glb go to Geometry hit compression and all the default settings should be good then hit export gltf now that you have your animations. glb file you you just need to use jtf to save it into animations then use animations with the animation that you loaded do it on the group that is containing your whole Avatar where is it here the main group and have the code that is running our animation based on the animation we are currently own so what I did is a use State on all the animation if it find an idol one it will start with the idol if not it will play the first animation available in our file so it's a generic so now our Avatar is ready it has all the different animation that I added in my file automatically if you're using my code and different Avatar file it will be able to show that drop down in Lev so you will have access to all animations now let's have a look at the facial expression if you look here you can see that …