In this article I will show how to install a fully local version of the PrivateGPT on Ubuntu 20.04 machine.
PrivateGPT is a project developed by Iván Martínez, which allows you to run your own GPT model trained on your data, local files, documents and etc. It is free and can run without internet access in local setup mode. Project is open-sourced and available at https://github.com/imartinez/privateGPT
My local setup includes the following components of the PrivateGPT:
Prerequisites:
Before installation ensure that you have satisfied these prerequisites:
Python version 3.11. To install and set the 3.11 version you can use pyenv tool:
💡To install pyenv on your machine checkout this article: How to install 'pyenv' Python version manager on Ubuntu 20.04pyenv install 3.11 pyenv global 3.11
For dependency management install Poetry:
curl -sSL https://install.python-poetry.org | python3 -
Ensure that you have
make
tool installed:make --version
Install PrivateGPT:
Clone PrivateGPT repository, and navigate to it:
git clone https://github.com/imartinez/privateGPT cd privateGPT
Install Ollama for LLM and Embeddings:
curl -fsSL https://ollama.com/install.sh | sh
Ensure that your
ollama
service is running:ollama serve
sudo systemctl status ollama.service
Now in a separate terminal install the default models and settings from
settings-ollama.yaml
file:sudo ollama pull mistral sudo ollama pull nomic-embed-text
If you want to run
ollama
withoutsudo
, then add your user account to therender
group:sudo usermod -aG render <your_username>
after adding the group log out and log back in to make the changes take effect.
To list the groups your user belongs to use:
groups
Install PrivateGPT with the following command:
poetry install --extras "ui llms-ollama embeddings-ollama vector-stores-qdrant"
Once installed, you can run PrivateGPT. Before that ensure that you have a working Ollama running locally.
PGPT_PROFILES=ollama make run
Now you can access your PrivateGPT UI http://localhost:8001/