Fine-tune models like MistralAI using Koyeb's Serverless GPUs.

Fine-Tune LLMs with Serverless GPUs

A step-by-step guide to learn how to fine-tune MistralAI on domain-specific knowledge and evaluate the fine-tuned model's performance.

Fine-Tune MistralAI and Evaluate the Fine-Tuned Model on Koyeb Serverless GPUs


Large Language Models (LLMs) are fantastic tools for getting quick answers on programming questions. However, their knowledge is not always up to date and they may not know about your favourite framework or library.

In this guide, we'll walk you through how to fine-tune an LLM on your favourite project's documentation. This will let the model answer questions with (hopefully) correct, and up-to-date information. We'll be using [Llama 3.1 8B Instruct](https://huggingface.co/meta-llama/Llama-3.1-8B-Instruct), an open-weights model from Meta and teach it about Apple's new deep learning framework: [MLX](https://github.com/ml-explore/mlx).

We will first generate a custom LLM training dataset from Apple's documentation and publish it on the HuggingFace Hub. Then, we'll fine-tune Llama 3.1 8B using QLORA, a training method which significantly reduces GPU memory usage and training time. Finally, we'll deploy the model on Koyeb's serverless GPUs, enabling you to get answers to your questions in real-time.

Quick disclaimer: This guide is intended as an introductory overview. Fine-tuning a language model involves careful consideration of data distribution, hyperparameters, and continual pre-training. For production-level models, a more rigorous approach is required.

## Requirements

To successfully follow this tutorial, you will need the following:

- [Python 3.8](https://www.python.org/) or later.
- An [OpenAI API key](https://platform.openai.com/api-keys).
- A [HuggingFace access token](https://huggingface.co/settings/tokens) with write permissions and access to [Llama 3.1 8B Instruct](https://huggingface.co/meta-llama/Llama-3.1-8B-Instruct).
- A [Weights & Biases access token](https://wandb.ai/) (Optional).

## Steps

1. Configure the local environment.
2. Build the Apple MLX documentation from source (Optional).
3. Generate the training dataset with Python and the OpenAI API.
4. Fine-tune the model using Jupyter Notebook on Koyeb.
5. Deploy and use the fine-tuned model.

## Configure the local environment

First, we'll clone the repository for this project, create a Python virtual environment, and install the required dependencies.

```bash
# Clone the repository
git clone https://github.com/koyeb/example-finetune-llama-3-1.git
cd example-finetune-llama-3-1

# Create a virtual environment
python3 -m venv venv

# Activate the virtual environment (Windows)
.\venv\Scripts\activate.bat
#  Activate the virtual environment (Linux & macOS)
source ./venv/bin/activate
```

Now let's install the required dependencies:

```bash
pip install datasets==2.16.1 openai==1.42.0 tqdm
```

The `datasets` library is used to push our dataset to the HuggingFace Hub and the openai library lets us interact with the OpenAI API.

Next, we'll login to the HuggingFace Hub.

```
huggingface-cli login
```

Follow the instructions in the terminal and paste your access token when prompted.

## Build the Apple MLX documentation from source

The repository for this tutorial already contains the Apple MLX documentation in text format. However, if you want to build the documentation from source, you can follow the instructions below. Otherwise, you can skip to the next step.

You'll need to install doxygen to build the Apple MLX documentation from source.

```bash
# Install doxygen (macOS)
brew install doxygen
# Install doxygen (Linux)
sudo apt-get install doxygen
```

Now, you can clone the MLX repository and build the documentation using Doxygen.

```bash
# Clone the MLX repository
rm -r mlx ; git clone https://github.com/ml-explore/mlx.git

# Install the required dependencies and build the documentation in text format
cd mlx/docs
pip install -r requirements.txt
doxygen && make text

# Move back to the project directory
cd ../..
```

If everything went well, the `mlx/docs/build/text' directory should now contain the documentation in text format. If you encounter any issues, you can fall back to using the [pre-built documentation](https://github.com/koyeb/example-finetune-llama-3-1/tree/main/mlx/docs/build/text) from the repository.

## Generate the training dataset

To generate the training dataset, we'll use the OpenAI API. This step should use less than $10 in OpenAI credits and take an hour or so. If you don't have an OpenAI API key or don't want to use it, you can skip to the next step and use [the dataset we published on HuggingFace](https://huggingface.co/datasets/koyeb/Apple-MLX-QA).

In order to generate conversation samples from the MLX documentation, we'll use the `generate_dataset.py` script in this tutorial's repository. There's a lot going on in the script, so let's break it down:

- At the top of the file, we define the prompts used to generate questions and answers.
- After parsing the command-line arguments, we read all the documentation files in an array.
- For each chunk of documentation, we generate N questions using the chat endpoint of the OpenAI API.
  - We use OpenAI's structured output feature to ensure the model generates a list of questions. This is done by specifying a JSON schema in the `response_format` parameter. For more information on structured outputs, see the [OpenAI API documentation](https://platform.openai.com/docs/guides/structured-outputs).
- For each question, we generate an answer using the same chat endpoint.
- Finally, we write the question-answer pairs to a JSONL file and push it to the HuggingFace Hub.

Let's run the script, specifying the input directory, the OpenAI model to use and the HuggingFace repository to push the dataset to. This should be the name of your organization (or HuggingFace account) and the name of the dataset (for example koyeb/Apple-MLX-QA).

```bash
export OPENAI_API_KEY='your-openai-api-key'
export HF_ORG='your-huggingface-organization'
python generate_dataset.py --input mlx/docs/build/text --model gpt-4o --repo $HF_ORG/Apple-MLX-QA
```

Once done, you should see your dataset on the [HuggingFace Hub](https://huggingface.co/).

## Fine-tune the model using Jupyter Notebook on Koyeb

Now that we have our dataset, we can proceed with fine-tuning. The next step involves deploying a Jupyter Notebook server on a Koyeb GPU instance. To do this, you can visit the [One-Click App page for Jupyter Notebook on Koyeb](https://www.koyeb.com/deploy/pytorch-jupyter) and follow the instructions on the page.

Once your notebook is started, visit the URL of your service and connect to Jupyter using the password you set during the deployment process. Once you're logged into Jupyter, import the [notebook.ipynb](https://github.com/koyeb/example-finetune-llama-3-1/blob/main/notebook.ipynb) file that's in this tutorial's repository by clicking on the "Upload" button in the Jupyter interface as shown below:

![Upload the notebook](/static/images/tutorials/fine-tune-llama-3-1/upload-notebook.png)

The rest of the instructions for this step are in the notebook. Once you're done, you can come back here to deploy and use the fine-tuned model on Koyeb.

## Deploy and use the fine-tuned model on Koyeb

This section teaches you how to use the model in your Python code as well as how to deploy it for production use on Koyeb's serverless GPUs.

You can use your LORA adapter in Python code using torch, transformers, and peft. Here's an example:

```
pip install torch transformers peft
```

```python
import torch
from peft import PeftModel
from transformers import AutoModelForCausalLM, AutoTokenizer

# Load the base model and tokenizer
model = AutoModelForCausalLM.from_pretrained(
    "meta-llama/Llama-3.1-8B-Instruct",
    torch_dtype=torch.bfloat16,
    device_map="cuda",
)
tokenizer = AutoTokenizer.from_pretrained("meta-llama/Llama-3.1-8B-Instruct")

# Load the LORA adapter
model = PeftModel.from_pretrained(
    model,
    "koyeb/Meta-Llama-3.1-8B-Instruct-Apple-MLX-Adapter",
)

# Utility function for chat completions
def complete_chat(model, tokenizer, messages, **kwargs):
    inputs = tokenizer.apply_chat_template(messages, return_tensors="pt", return_dict=True, add_generation_prompt=True).to(model.device)
    num_input_tokens = len(inputs["input_ids"][0])
    model.eval()
    with torch.inference_mode():
        return tokenizer.decode(model.generate(**inputs, **kwargs)[0][num_input_tokens:], skip_special_tokens=True)

# Generate and print the response
print(complete_chat(model, tokenizer, [
    {"role": "system", "content": "You are a helpful AI coding assistant with expert knowledge of Apple latest machine learning framework: MLX. You can help answer questions about MLX, provide code snippets, and help debug code."},
    {"role": "user", "content": "How do you transpose a matrix in MLX?"}
], max_new_tokens=256))
```

For production, you can deploy your fine-tune on Koyeb's serverless GPUs using vLLM with One-Click Apps.

1. Visit the [One-Click App page for vLLM](/deploy/vllm) and click the "Deploy" button.
2. Override the command args and specify the HuggingFace repository for your merged model: `["--model", "YOUR-ORG/Meta-Llama-3.1-8B-Instruct-Apple-MLX", "--max-model-len", "8192"]`.
3. Set your HuggingFace access token in the `HF_TOKEN` environment variable. Optionally, set `VLLM_DO_NOT_TRACK` to `1` to disable telemetry.

Once deployed, you can interact with the model using the OpenAI API format. Here's an example using curl:

```
export SERVICE_URL='your-service.koyeb.app'
curl https://$SERVICE_URL/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
     "messages": [
        {"role": "system", "content": "You are a helpful AI coding assistant with expert knowledge of Apple latest machine learning framework: MLX. You can help answer questions about MLX, provide code snippets, and help debug code."},
        {"role": "user", "content": "How do you transpose a matrix in MLX?"}
    ],
     "model": "koyeb/Meta-Llama-3.1-8B-Instruct-Apple-MLX",
     "temperature": 0.3
   }'
```

## Conclusion

Congratulations, you've successfully fine-tuned Llama 3.1 8B using QLORA!

Remember, fine-tuning is an iterative process. Feel free to experiment with different hyperparameters and training methods to get the best results. You can also work on increasing the size or improving the quality of your training dataset using additional data sources or data augmentation techniques. Hack on!


Fine-tune custom models using Koyeb's serverless GPUs.

Learn how to fine-tune an Llama 3.1 8B using QLORA on your favourite project's documentation. This will enable the model to answer questions with (hopefully) correct, and up-to-date information.

Fine-Tune Llama 3.1 8B using QLORA on Koyeb Serverless GPUs


## Introduction

[Celery](https://github.com/celery/celery) is an open-source distributed task queue that focuses on real-time processing and task scheduling. It coordinates tasks through a backing message broker, typically [Redis](https://redis.io/) or [RabbitMQ](https://www.rabbitmq.com/), making it straightforward to scale as demand increases. While Celery is written in Python, its messaging protocols are not language specific, allowing for clients in any language.

In this guide, we will go over how to deploy a Celery worker on the [Koyeb](/) serverless platform using git-driven deployment. We will create a repository with Celery and its dependencies installed, deploy RabbitMQ to act as our backing message broker, and then connect our GitHub account to Koyeb to automatically deploy the Celery worker to the platform.

At the end of this guide, you should have a Celery worker deployed to Koyeb that you can automatically update by pushing changes to your GitHub account.

## Requirements

To follow along with this guide, you will need the following:

- Python 3 installed on your local machine
- A [Koyeb account](https://app.koyeb.com/) to deploy and run your Celery worker and RabbitMQ message broker
- A [GitHub account](https://github.com/) to store your application code and trigger deployments via repository changes

## Steps

This guide will cover how to deploy a Celery worker to Koyeb through the following steps:

1. [Create a Python Virtual Environment](#create-a-python-virtual-environment)
2. [Install Celery](#install-celery)
3. [Create a Basic Celery Application](#create-a-basic-celery-application)
4. [Push the Celery Project to GitHub](#push-the-celery-project-to-git-hub)
5. [Deploy RabbitMQ on Koyeb](#deploy-rabbit-mq-on-koyeb)
6. [Deploy the Celery Worker on Koyeb](#deploy-the-celery-worker-on-koyeb)
7. [Testing the Celery Worker](#testing-the-celery-worker)

## Create a Python Virtual Environment

To get started, we will create a Python virtual environment on our local machine to isolate our project's dependencies from the rest of the system packages. Python provides this functionality natively through the standard [`venv` module](https://docs.python.org/3/library/venv.html).

Start by creating a new directory to store our project files.

```shell
mkdir ~/celery-koyeb
```

Next, create the virtual environment within the project directory by typing:

```shell
python3 -m venv ~/celery-koyeb/venv
```

This will create a directory called `venv` within the `~/celery-koyeb` containing initialization scripts, copies of essential tools like `pip`, and links to the local Python interpreters.

To tell Python to use the virtual environment, activate it by sourcing the initialization script:

```shell
source ~/celery-koyeb/venv/bin/activate
```

Your shell prompt will change to reflect that the virtual environment is active for this session. Python will now use this directory structure in place of the system's standard Python file locations, allowing us to isolate dependencies.

## Install Celery

Next, we need to install Celery within the virtual environment using `pip`, the Python package manager:

```shell
pip install celery
```

Python will download Celery and its dependencies and install them within the virtual environment filesystem.

Along with the Python modules, the installation also includes a small wrapper script called `celery` that lets you interact with it from the command line. Use the command now to verify that the installation was successful:

```shell
celery --version
```

The command should show the version as expected. The exact version returned does not matter, so long as the command returns successfully:

```
5.2.7 (dawn-chorus)
```

With Celery installed, record the project's dependencies by typing:

```shell
pip freeze > ~/celery-koyeb/requirements.txt
```

## Create a Basic Celery Application

Next, we'll create a basic Celery application in our project directory.

Create a file called `tasks.py` within the `~/celery-koyeb` directory with the following content inside:

```python
import os
from celery import Celery


# Fail if the `BROKER_HOST` environment variable is not defined
BROKER_HOST = os.environ.get('BROKER_HOST', None)
if BROKER_HOST is None:
    raise Exception("BROKER_HOST environmental variable is not defined")
broker_url = 'pyamqp://guest@' + BROKER_HOST + '//'

app = Celery('tasks', broker=broker_url, backend='rpc://')

@app.task
def add(x, y):
    return x + y
```

Let's talk a bit about what is happening in the code above.

After importing the `os` module and the `Celery` class, the code above sets the location of the backing RabbitMQ message broker based on the value of the `BROKER_HOST` environment variable. If this variable is not set, we will error out to avoid accidentally deploying misconfigured instances.

Next, we create an instance of the `Celery` class called `app`. We pass `tasks` as the name of the main module and include the broker and backend configuration to allow Celery to connect to the RabbitMQ instance indicated by the environment variable mentioned earlier.

Finally, we create a function with the `@app.task` decorator to add it to Celery's task registry. Our function adds two numbers together and returns the result.

Save and close the file when you are finished.

## Push the Celery Project to GitHub

Next, we will commit our project to `git` and push our changes to our GitHub account.

In your project directory, initialize a `git` repository to get started:

```shell
cd ~/celery-koyeb
git init
```

Add the `tasks.py` application and the `requirements.txt` file to the staging area. You can also create a minimal `.gitignore` file to make sure your virtual environment does not accidentally get committed. Commit the changes when you are done:

```shell
echo 'venv' >> .gitignore
git add tasks.py requirements.txt .gitignore
git commit -m 'Initial commit'
```

Create a [new repository](https://github.com/new) within your GitHub account. This can be either public or private. Add the GitHub repository address as the `origin` of your project and push your changes to the repository:

```shell
git remote add origin git@github.com:<YOUR_GITHUB_USERNAME>/<YOUR_GITHUB_REPOSITORY>.git
git branch -M main
git push -u origin main
```

If you refresh the GitHub repository page in your browser, your project files will appear.

## Deploy RabbitMQ on Koyeb

Before we deploy our Celery worker, we need to deploy a RabbitMQ instance for Celery to use as a message broker. We will use the [official RabbitMQ image on DockerHub](https://hub.docker.com/_/rabbitmq) for this purpose.

Go to the **Overview** tab of the [Koyeb control panel](https://app.koyeb.com/) and click **Create Web Service** to begin:

1. Select **Docker** as the deployment method.
2. In the **Docker image** field, enter `docker.io/rabbitmq`.
3. In the **Exposed ports** section, change the port number to 5672 and deselect the **Public** option. This will allow other services within this application (like our Celery application) to access the message broker using its service name without exposing the service to the internet.
4. Click the **Deploy** button.

Your new application will be created and the RabbitMQ service will begin to deploy within it. In a few minutes, the RabbitMQ container will finish deploying to Koyeb and the service will be marked as healthy.

Before moving on, copy the **Private domain** of your RabbitMQ deployment. We will need this later as we configure the Celery worker service.

## Deploy the Celery Worker on Koyeb

Now that the RabbitMQ service is deployed, we can deploy the Celery application from our GitHub repository.

In the **Apps** tab of the [Koyeb control panel](https://app.koyeb.com/), click the app that contains your RabbitMQ service. Click the **Create Service** button to deploy a new service to the app alongside the RabbitMQ container:

1. Choose **GitHub** as the deployment method.
2. Select the repository for your Celery application from the list.
3. In the **Service type** section, select **Worker**.
4. In the **Builder** section, click the **Override** toggle associated with the **Run command** and enter `celery --app=tasks.app worker` in the field.
5. In the **Environment variables** section, create a new variable called `BROKER_HOST` to tell the Celery worker where to find the RabbitMQ instance. Set the value to the **Private domain** you copied from the RabbitMQ service. It will follow this format: `<YOUR_SERVICE_NAME>.<YOUR_APP_NAME>.koyeb`.
6. Click the **Deploy** button.

The Celery application will be pulled from your GitHub repository and deployed. In a few minutes, the Celery worker will begin running within the service and be available for work.

If you want to learn about how Koyeb automatically builds your Python applications from git, make sure to read the [how we build from git](/docs/apps/build-from-git) documentation.

## Testing the Celery Worker

Unlike a web application, Celery does not have a web interface to easily show you that the deployment succeeded. We can, however, use Koyeb's built-in Console to test that the worker is running, able to connect with RabbitMQ, and can process tasks.

In the [Koyeb control panel](https://app.koyeb.com/), click the Celery service and then select the **Console** tab.

In the console window, start a `bash` session by typing:

```shell
/bin/bash
```

Next, source the environment configuration file so that Python can find the modules associated with our application:

```shell
source .profile.d/python.sh
```

Start a Python session so that we can interact with the Celery application:

```shell
python
```

First, import our `tasks` module:

```python
import tasks
```

Next, we can run the `add` function using [Celery's `delay` method](https://docs.celeryq.dev/en/stable/reference/celery.app.task.html#celery.app.task.Task.delay):

```python
results = tasks.add.delay(93, 80)
```

We can then use the `results` object to check on the task's progress. Call the [`ready` method](https://docs.celeryq.dev/en/stable/reference/celery.result.html#celery.result.AsyncResult.ready) to see if it has been processed yet:

```python
results.ready()
True
```

You can check the answer returned by the function using the [`get` method](https://docs.celeryq.dev/en/stable/reference/celery.result.html#celery.result.AsyncResult.get). It's usually a good idea to set a timeout so that it won't hang waiting if the result is not available yet:

```python
results.get(timeout=1)
173
```

While this test uses trivial calculations, the results validate that the system is working as intended and able to communicate with RabbitMQ using Koyeb's [service mesh](/docs/reference/service-mesh-and-discovery).

When you are finished, exit the Python session by typing:

```python
exit()
```

You can then close the bash session by typing:

```shell
exit
```

## Conclusion

In this guide, we covered how to deploy a Celery worker on Koyeb using git-driven deployment. Because Celery requires a message broker for coordination, we also discussed how to deploy a RabbitMQ container on Koyeb using Docker-based deployment.

By deploying both of these services within a single application, we were able to configure Celery to connect with RabbitMQ over Koyeb's service mesh with minimal configuration. Our Celery deployment is backed by a GitHub repository, so it will automatically redeploy whenever new changes are pushed to the branch it is tracking. This makes it easy to change your running Celery worker configuration by modifying your project's code.

Questions or suggestions to improve this guide? Join us on the [community platform](https://community.koyeb.com/) to chat!


Dynamically scale your Celery workers with ease on Koyeb. Run on high-performance infrastructure. No credit card required.

Easy-to-scale Celery workers

This guide explains how to deploy Python Celery Workers using git-driven deployment on Koyeb Serverless Platform.

Deploy a Python Celery Worker

This tutorial showcases how to build a REST API with Flask and a Fauna database and deploy it on Koyeb serverless platform. It also explains how to add authentication to a REST API.

How to Deploy a REST API with Flask, Fauna, and Authentication on Koyeb


## Introduction

[Flask](/docs/deploy/flask) is a lightweight web framework written in Python that offers developers powerful tools and features to create web applications with ease and flexibility.

This guide demonstrates how to deploy a Flask application using Gunicorn on the Koyeb Severless Platform. You will create a "Hello World" Flask application, push your code to a GitHub repository, and then deploy your application on the Koyeb platform using its git-driven deployment.

With Koyeb's git-driven deployment, your application benefits from a built-in continuous deployment pipeline because each git push you make to your repository automatically triggers a new build and deployment for your application on the Koyeb Serverless Platform.

For those who prefer deploying Docker containers, we also published a guide explaining how to dockerize and deploy a [Python Flask application on the Koyeb Serverless Platform](/tutorials/python-flask-application-deployment-on-koyeb).

## Requirements

To successfully follow and complete this guide, you need:

- A [Koyeb account](//app.koyeb.com/auth/signin) to deploy and run your Python Flask application
- [Python](//www.python.org/downloads/) installed on your computer
- A [GitHub account](//github.com/)

## Steps

To successfully create and deploy Flask application on Koyeb using git-driven deployment, you will need to follow these steps:

1. [**Create a Flask application**](#create-a-flask-application)
2. [**Create a git directory for the Flask application**](#continuously-deploy-a-flask-app-using-gunicorn-on-koyeb#create-a-git-directory-for-the-flask-application)
3. [**Deploy the Flask application on Koyeb**](#deploy-the-flask-application-on-koyeb)

## Create a Flask application

Begin by creating a directory where our application will live and moving into it by running the following commands:

```
mkdir flask-on-koyeb
cd flask-on-koyeb
```

### Create a virtual environment

Next, we will create a virtual environment, also known as virtualenv. Virtual environments allow you to create an isolated Python environment to avoid interfering with Python's system packages or other virtual environments.

To create a virtual environment, execute:

```
python3 -m venv ~/.venv/flask-app
```

This command creates a new directory `flask-app` in the ~/.venv directory. The virtualenv contains a copy of the Python interpreter, pip, the standard library, and various supporting files.

To activate the virtualenv we just created, run:

```
source ~/.venv/flask-app/bin/activate
```

The virtualenv python and pip executables are now added into your shell’s PATH.

### Install Dependencies

Next, install the dependencies required to run the application. In this case, we need Flask and Gunicorn. Flask is the lightweight Python framework we are using for this application and Gunicorn is a web server gateway interface (WSGI) server implementation used to run Python web apps. Gunicorn is a high-performing server better suited for production traffic.

```
pip install Flask Gunicorn # Install Flask and Gunicorn
pip freeze > requirements.txt # Create the requirements.txt to store the dependencies and version of each package required to run our application.
```

### Create the Flask application

In your `flask-on-koyeb` directory, create a new `app.py` file with the following:

```
from flask import Flask
app = Flask(__name__)

@app.route('/')
def hello_world():
    return 'Hello, World!'

if __name__ == "__main__":
    app.run()
```

This file is storing our Flask application code. In the snipper above, we define a route to handle `/` requests and return `Hello, World!` as a response.

Check to see if the application is working locally by running:

```
python3 app.py
```

In your browser, go to `http://localhost:5000` and you will see `Hello, World!`.

## Create a git directory for the Flask application

Now that we have created our Flask application, we will create a git directory to store it.

Initialize a new git directory for your Flask project by running the following in your terminal:

```
git init
```

To keep only the necessary files in the repository, add this .gitignore file to exclude unnecessary files.

```
curl https://raw.githubusercontent.com/github/gitignore/master/Python.gitignore > .gitignore
```

Next, execute the following command to add your files and commit the changes to your repository:

```
git add .
git commit -m "Flask app initial commit"
```

Open a new tab in your browser and go to GitHub. Create a new repository named `flask-on-koyeb` and click the _Create repository_ button. Then, go back to your terminal and add GitHub as a remote repository by running:

```
git remote add origin git@github.com:YOUR_GITHUB_USERNAME/flask-on-koyeb.git
```

Rename the repository default branch to main executing:

```
git branch -M main
```

Next, push your changes to the GitHub repository by running:

```
git push -u origin main
```

Now, you are ready to deploy your application on Koyeb.

## Deploy the Flask application on Koyeb

On the **Overview** tab of the [Koyeb control panel](https://app.koyeb.com), click **Create Web Service** to begin:

1. Choose **GitHub** as the deployment method.
2. Select the repository for your Flask application.
3. In the **Builder** section, click the **Override** toggle associated with the **Run command** and enter `gunicorn app:app` in the field.
4. Choose a name for your App and Service, for example `koyeb-flask-demo`, and click **Deploy**.

**Note:** You can add more regions, change the instance size, set environment variables, and define horizontal scaling according to your needs.

You land on the deployment page of your service where you can follow the progress of your Flask application's deployment. Koyeb will take care to build and deploy your application. Once these steps are completed, you can access your application by clicking your App URL ending with koyeb.app in the Koyeb control panel.

If you want to learn about how Koyeb automatically builds your applications from git, make sure to read our [how we build from git](/docs/apps/build-from-git) documentation.

Your Flask application is now running on Koyeb and benefits from native autoscaling, automatic HTTPS (SSL), auto-healing, and global load-balancing across our edge network.

## Conclusion

In this guide, you learned how to deploy a Flask application on Koyeb and benefit from its built-in continuous deployment pipeline. Each change you push to your repository will automatically trigger a new build and deployment on the Koyeb Serverless Platform. Your changes then go live as soon as the deployment passes all necessary health checks. In case of a failure during one of your deployments, we ensure to keep the latest working deployment active so your application is always up and running.

Other great benefits to deploying your Python Flask application on Koyeb include the platform's built-in autoscaling, automatic HTTPS (SSL), high availability, global load-balancing and edge network.

All of these features and more work out-of-the-box and from the getgo. You focus on your code while Koyeb handles all the necessary underlying infrastructure to keep your services scalable, high performing and available.

Want more Flask on Koyeb? Check out our [Flask documentation](/docs/deploy/flask) and [one-click Flask app](/deploy/flask) to learn how to deploy a Flask application on Koyeb.

Give Koyeb a try with your application! If you would like to read more Koyeb tutorials, checkout out our [tutorials collection](/tutorials). Have an idea for a tutorial you'd like us to cover? Let us know by joining the conversation over on the **[Koyeb community platform](https://community.koyeb.com/)**!


This guide shows how to deploy a Python Flask application using Gunicorn on the Koyeb Serverless Platform. Using Koyeb's git-driven deployment lets you benefit from the platform's built-in continuous deployment pipeline.

Continuously Deploy a Flask App using Gunicorn on Koyeb


## Introduction

Django is a popular and powerful framework for Python application developers. Django comes with many features like authentication, content administration, an Object-Relational Mapper (ORM), and much more.
Django supports various relational databases including MySQL which makes it possible to run a Django application using the PlanetScale database serverless platform without having to perform any changes in your application.

In this guide, we will create a Django application, configure our project to use a PlanetScale database, and deploy the application on the Koyeb Serverless Platform using git-driven deployment.

Using Koyeb git-driven deployment, your Koyeb application is linked to your GitHub repository and each time changes are pushed to your repository, your application is being redeployed automatically by Koyeb.

At the end of this guide, you will have a working Django application running on Koyeb.

## Requirements

To successfully deploy a Django application on Koyeb, you need:

- Python installed on your machine
- A [Koyeb account](https://app.koyeb.com) to deploy and run the Django application
- A [PlanetScale account](https://planetscale.com) to run the database our application will use
- A [GitHub account](https://github.com/) to store your application code and deploy your application on Koyeb via GitHub

## Steps

To complete this guide to have a Django application running on Koyeb, you need to follow these steps:

1. [**Create a virtual environment**](#create-a-virtual-environment)
2. [**Create a new Django application**](#create-a-new-django-application)
3. [**Configure Django to use a PlanetScale database**](#configure-django-to-use-a-planetscale-database)
4. [**Deploy the Django application on Koyeb**](#deploy-the-django-application-on-koyeb)

## Create a virtual environment

Before creating the Django application, we will create a virtual environment also known as a virtualenv. Virtual environments allow you
to create an isolated Python environment to avoid interfering with the Python's system packages or other virtual environments.

To create a new virtual environment that we will use for our Django application, in your terminal run:

```bash
python3 -m venv ~/.venv/django-app
```

This command creates a new folder `django-app` in the `~/.venv` directory. The virtualenv contains a copy of the Python interpreter, pip, the standard library, and various supporting files.

We can now activate the virtualenv we previously created by running:

```bash
source ~/.venv/django-app/bin/activate
```

Once the virtualenv is activated, the virtualenv python and pip executables are added into your shell’s PATH.

## Create a new Django application

Let's start by creating a dedicated folder for our app and moving into it:

```bash
mkdir django-on-koyeb
cd django-on-koyeb
```

Before creating our Django application, we need to install:

- django, to install the Django framework and libraries
- dj-database-url, a Django utility that allows to parse DATABASE_URL.
- gunicorn, a pure-Python HTTP server for WSGI applications that can run multiple Python concurrent processes
- mysqlclient, the librairies to use MySQL or in our case Planetscale

> mysqlclient requires system librairies, please check
> https://pypi.org/project/mysqlclient/ for detailled instructions

In your terminal, execute the following commands to install the packages locally and save
these dependencies.

```bash
pip install django gunicorn dj_database_url mysqlclient
pip freeze > requirements.txt
```

Once the previous step is completed, you can create the Django application in
our local directory by running:

```bash
django-admin startproject django_on_koyeb ./
```

This command creates a `django_on_koyeb` directory in your current directory containing the following files:

```bash
├── django_on_koyeb
│   ├── __init__.py
│   ├── asgi.py
│   ├── settings.py
│   ├── urls.py
│   └── wsgi.py
└── manage.py
```

### Prepare the Django application for production

Next, we will edit the `settings.py` file as we need to customize settings
related to security and performance for
our development and production environment.

Open the `settings.py` using your favorite editor, here nano:

```bash
nano django_on_koyeb/settings.py
```

We will edit the `settings.py` file to load some important settings of our application using environment variables.
To access and read environment variables, we need to import the `os` module.

```python
PRISM_INSERTED import os
from pathlib import Path
```

Django also requires a `SECRET_KEY`. This key is used to provide cryptographic signing and should be set to a unique, unpredictable value.
By default, we will want to generate a random secret key when nothing is passed as an environment variable, i.e. when you are running in development mode.

To generate a random secret key, we will need to import `from django.core.management.utils import get_random_secret_key`:

```python
import os
PRISM_INSERTED from django.core.management.utils import get_random_secret_key
from pathlib import Path
```

Search for the `SECRET_KEY = ...` line in the `settings.py` file and replace it as below:

```python
SECRET_KEY = os.environ.get('DJANGO_SECRET_KEY', get_random_secret_key())
```

Now, when the `DJANGO_SECRET_KEY` environment variable is set, our application will use it. If no value is provided, the `get_random_secret_key()` function is called and will return a random secret key.

By default, the Django application runs in `DEBUG`. When deploying the application in production, for security and performance purposes `DEBUG` mode needs to be disabled. We will change the default value to disable DEBUG until it is explicitly enabled via the environment variable `DJANGO_DEBUG=True`.

```python
DEBUG = os.environ.get('DJANGO_DEBUG', 'False') == 'True'
```

Next, update the `ALLOWED_HOSTS` directive to read the `DJANGO_ALLOWED_HOSTS` from the environment. `ALLOWED_HOSTS` is required to contain the list of strings representing the host/domain names that the Django application can serve.

```python
ALLOWED_HOSTS = os.getenv("DJANGO_ALLOWED_HOSTS", "localhost,127.0.0.1").split(",")
```

Last, go to the bottom of the file and add the following directive below `STATIC_URL`:

```python
STATIC_URL = '/static/'
PRISM_INSERTED STATIC_ROOT = os.path.join(BASE_DIR, "staticfiles")
```

This setting is used to set the directory from which we want to serve static files.
The first part of the Django configuration is done, we can move to the next step and configure Django to use a PlanetScale database.

## Configure Django to use a PlanetScale database

To use a PlanetScale database in our Django application, create a new database
from the PlanetScale [control panel](https://app.planetscale.com/) or using the
[CLI](https://github.com/planetscale/cli#installation) running:

```bash
pscale database create django-on-koyeb
```

Once the database is created, create a new database password for the main branch. On the PlanetScale control panel, click the **Connect** button and new password to generate a user and password to connect the database.
You can also perform this action using the PlanetScale CLI by running:

```bash
pscale password create django-on-koyeb main production-password
```

Keep these credentials in a safe place, we will need them when deploying our application on Koyeb.

On PlanetScale, the main database's schema is immutable and all changes must occur on a development branch. As we will need to perform a migration propagate our models into our PlanetScale database,
we need to create a new branch.

On the PlanetScale control panel, click the **New branch** button and name the branch migration. You can also perform this operation using the PlanetScale CLI running:

```bash
pscale branch create django-on-koyeb migration
```

Once the branch is created, create a new database password to access the branch. On the PlanetScale control panel, click the **Connect** button and new password to generate a user and password to connect the database.
You can also perform this action using the PlanetScale CLI by running:

```bash
pscale password create django-on-koyeb migration migration-branch-password
```

As PlanetScale uses Vitess behind the scenes, we need to use a custom database engine to make our Django application work with Vitess.
To retrieve the Vitess custom engine, clone the Vitess repository and copy the `custom_db_backends` directory in your Django application root folder:

```bash
git clone https://github.com/vitessio/vitess.git ~/vitess
cp -r ~/vitess/support/django/custom_db_backends ./
```

Next, open the Django application `settings.py`:

```bash
nano django_on_koyeb/settings.py
```

By default, Django is using an SQLite database, we need to edit the
`settings.py` to use a Vitess database and retrieve the configuration via the `DJANGO_DATABASE_URL` environment variable.
If the `DJANGO_DATABASE_URL` is not set, the application will raise the exception `DJANGO_DATABASE_URL environment variable not defined` when starting.
In the `settings.py` replace the following part:

```python
# DATABASES = {
#     'default': {
#         'ENGINE': 'django.db.backends.sqlite3',
#         'NAME': BASE_DIR / 'db.sqlite3',
#     }
# }
if os.getenv("DJANGO_DATABASE_URL", None) is None:
    raise Exception("DJANGO_DATABASE_URL environment variable not defined")

DB_PARAMS = dj_database_url.parse(os.environ.get("DJANGO_DATABASE_URL"))
DB_PARAMS["ENGINE"] = "custom_db_backends.vitess"
DATABASES = {
    "default": DB_PARAMS,
}
```

Let's not forget to import the `dj_database_url` module we're using:

```python
import os
from django.core.management.utils import get_random_secret_key
PRISM_INSERTED import dj_database_url
from pathlib import Path
```

You can now save and close the `settings.py`. Let's run Django migration to propagate the application models to the PlanetScale database. In the terminal run the following command:

```bash
DJANGO_DATABASE_URL=mysql://PS_USERNAME:PS_PASSWORD@PS_HOST:3306 python manage.py migrate
```

Replace the `PS_USERNAME`, `PS_PASSWORD`, `PS_HOST` values with the PlanetScale values you generated for the `migration` branch.

Then on the PlanetScale control panel, create a new **Deploy request** to propagate the schema changes performed on the `migration` branch to the production branch `main` and deploy it. You can also perform this operation using the PlanetScale CLI by running:

```bash
pscale deploy-request create django-on-koyeb migration
pscale deploy-request deploy django-on-koyeb 1
```

The application is now configured and ready to be deployed on Koyeb.

## Deploy the Django application on Koyeb

As the Django application will be deployed on Koyeb using Git, we need to initialize a new git directory for our Django project.
In your terminal run:

```bash
git init
```

To keep only necessary files in our repository, we will add a `.gitignore` file to exclude undesired files.

```bash
curl https://raw.githubusercontent.com/github/gitignore/master/Python.gitignore > .gitignore
```

Then, execute the following command to add files to commit to your repository and commit the changes

```bash
git add .
git commit -m "Django app initial commit"
```

Open a new tab in your browser and go to [GitHub](https://github.com/). Create a new repository named `django-on-koyeb` and click the **Create repository** button.
Then, go back to your terminal and add GitHub as a remote repository by running:

```bash
git remote add origin git@github.com:YOUR_GITHUB_USERNAME/django-on-koyeb.git
```

Rename the repository default branch to main executing:

```bash
git branch -M main
```

Last, push your changes to the GitHub repository by running:

```bash
git push -u origin main
```

Everything is now ready to deploy the Django application on Koyeb.

Go to the [Koyeb control panel](https://app.koyeb.com) and on the **Overview** tab, click **Create Web Service** to begin:

1. Choose **GitHub** as the deployment method.
2. Select the repository you created from the list.
3. In the **Builder** section, click the **Override** toggle associated with the **Run command** and enter `gunicorn --worker-tmp-dir /dev/shm django_on_koyeb.wsgi` in the field.
4. In the **Environment variables** section, click **Add variable** to create a `DJANGO_DATABASE_URL` environment variable of type **Secret**. In the form that appears, name your Secret `django-db-url` with `mysql://PS_USERNAME:PS_PASSWORD@PS_HOST:3306` as the value. Take care to replace `PS_USERNAME`, `PS_PASSWORD`, and `PS_HOST` with the PlanetScale password information you created for your main production branch.
5. Create an environment variable with the name `DJANGO_DEBUG` set to `True`.
6. Create an environment variable with the name `DJANGO_ALLOWED_HOSTS` set to `django-on-koyeb-<KOYEB-ORG>.koyeb.app`.
7. Choose a name for your App and Service, for example `django-on-koyeb`, and click **Deploy**.

Your application is now being built and deployed on the Koyeb serverless platform. Within a few minutes, you will be able to access your application by clicking your App URL: `https://django-on-koyeb-<KOYEB-ORG>.koyeb.app`.

## Conclusion

In this guide, we explained how to deploy a Django application on the Koyeb serverless platform with PlanetScale as the database backend for our application.
By deploying on Koyeb, your Django application is secured with native TLS encryption and benefits from all the Koyeb serverless features including autoscaling, auto-healing, and a high-performance edge network.

Thanks to the git-driven deployment feature offered by Koyeb, each time you push new modifications to your GitHub repository, a new deployment of your application occurs automatically.
This allows you to focus on your application code while Koyeb takes care of building, deploying your application, and ensure your service is always up and running.

Questions or suggestions to improve this guide? Join us on the **[community platform](https://community.koyeb.com/)** to chat!


In this guide, we will demonstrate how to deploy a Python Django application using Koyeb Serverless Platform and PlanetScale.

How to Deploy a Python Django Application using PlanetScale and Koyeb Serverless Platform


## Introduction

Flask is a popular and lightweight web framework written in Python offering developers all the tools and features to create web applications with flexibility and ease.

In this guide, you will create a simple "Hello World" application written in Flask, create a Dockerfile using multi-stage builds to minimize the Docker image size, run
the Dockerized application locally and deploy it to the Koyeb platform.

By deploying your Python Flask application on Koyeb you benefit from native autoscaling, Automatic HTTPS (SSL) on all apps you deployed, high availability, and global load-balancing & CDN to make your applications global.

## Requirements

To successfully follow and complete this guide, you need:

- A [Koyeb account](https://www.koyeb.com) to deploy and run your Python Flask application

## Steps

To successfully dockerize your Flask application and deploy it to Koyeb, you have to follow the following steps:

1. Create a "Hello World" Flask application
2. Create the Docker image to run the Flask application
3. Push the Docker image on the Docker Hub
4. Deploy the Flask application on Koyeb

## Create a "Hello World" Flask application

To get started, we need to create a new folder to store the Flask application. In your terminal create a new folder by executing the command below:

```bash
mkdir -p ~/demo/flask
```

In my case, I create a `demo/flask` folder in my home directory.

In the folder you created, create a new virtual environment folder using `python -m venv venv`.
Virtual environments provide isolation from the system environment allowing each virtual environment you create to have its own installation directories, dependencies, etc.

Then to activate and load your virtual environment, type:

```bash
. venv/bin/activate
```

### Install dependencies

Now that our environment is created and activated, let's install Flask by running:

```bash
pip install Flask # Install Flask
pip freeze > requirements.txt # Create the requirements.txt to store the dependencies and version of each package required to run our application.
```

### Create the Hello World Flask application

In the `demo/flask/app` folder, create a new file `app.py`. We will use this file to store our minimalist Flask application code.
In the snipper below, we define a route to handle `/` request and return "Hello, World!" as a response.

```python
from flask import Flask
app = Flask(__name__)

@app.route('/')
def hello_world():
    return 'Hello, World!'


if __name__ == "__main__":
    app.run()
```

### Run the application locally.

Back to your terminal and run the command below to launch the Flask application.

```bash
python app.py
```

In your browser, navigate to `http://localhost:5000` you will see `Hello, World!`.

## Create the Docker image to run the Flask application

To create our Docker image, we will Docker multi-stage to keep the Docker image size down by building an image that only contained what our application needs to run.

The Docker image is composed of two stages:

1. Create the virtual environment and install dependencies
2. Copy the virtual environment and launch the app

In the project root, create a Dockerfile containing the following:

```dockerfile
# Stage 1
FROM python:3-slim-buster AS builder

WORKDIR /flask-app

RUN python3 -m venv venv
ENV VIRTUAL_ENV=/flask-app/venv
ENV PATH="$VIRTUAL_ENV/bin:$PATH"

COPY requirements.txt .
RUN pip install -r requirements.txt

# Stage 2
FROM python:3-slim-buster AS runner

WORKDIR /flask-app

COPY --from=builder /flask-app/venv venv
COPY app.py app.py

ENV VIRTUAL_ENV=/flask-app/venv
ENV PATH="$VIRTUAL_ENV/bin:$PATH"
ENV FLASK_APP=app/app.py

EXPOSE 5000

CMD ["python", "-m" , "flask", "run", "--host=0.0.0.0"]
```

### Build and run the Docker image locally

To build the Docker image execute the following command in the `~/demo/flask` folder:

```bash
docker build . -t <YOUR_DOCKER_HUB_USERNAME>/koyeb-flask-demo
```

_In this guide, we will push the Docker image on the Docker Hub registry. Replace `<YOUR_DOCKER_HUB_USERNAME>` with your Docker Hub username._

Once the Docker build achieved, let's run the Docker image locally to ensure everything is running as expected:

```bash
docker run -d -p 5000:5000 <YOUR_DOCKER_HUB_USERNAME>/koyeb-flask-demo
```

If you open your browser and navigate to `http://localhost:5000` you will see `Hello, World!` served by the Docker container.

## Push the Docker image on the Docker Hub

With our Docker image built and working as expected, we can now push it to the [Docker Hub](https://hub.docker.com/).
In your terminal run the command below to push the image.

```bash
docker push <YOUR_DOCKER_HUB_USERNAME>/koyeb-flask-demo
```

Within a few minutes, you will see your Docker image available on your Docker Hub account repository.

## Deploy the Flask application on Koyeb

The most exciting part is happening now as we will deploy our Flask application on Koyeb!

On the [Koyeb control panel](https://app.koyeb.com/), on the **Overview** tab, click **Create Web Service** to begin:

1. Choose **Docker** as the deployment method.
2. Enter name of the image you created in the **Docker image** field. For example, `<YOUR_DOCKER_HUB_USERNAME>/koyeb-flask-demo`.
3. Click the **Private image** toggle and, in the select field, click **Create secret**. In the form that appears, choose a name for the Secret, for example `docker-hub-secret`, the registry provider, in our case **Docker Hub**, and the registry **username** and **password**. We recommend you to [generate an access token](https://hub.docker.com/settings/security) from the Docker Hub to use instead of your password. Once you've filled all the fields, click the **Create** button.
4. In the **Exposed ports** section, set the existing port to **5000**.
5. Choose a name for your App and Service, for example `koyeb-flask-demo`, and click **Deploy**.

You will automatically be redirected to the Koyeb App page where you can follow the progress of your Flask application deployment. In a few seconds, once your app is deployed, click on the _Public URL_ ending with `koyeb.app`.

Your Flask application is now running on Koyeb and benefits from native autoscaling, automatic HTTPS (SSL), auto-healing, and global load-balancing across our edge network.


This guide shows how to build a Docker image for your Flask application and how to deploy it on Koyeb serverless platform.