
The need for AI-generated images has been growing rapidly in recent years. These images are not only used for artistic purposes, but also for practical applications in various industries. For example, in the fashion industry, AI-generated images can be used to create virtual models for showcasing clothing. In the automotive industry, AI-generated images can be used for designing and testing new car models. And the best part? You can now run your own AI image generation machine on Koyeb GPUs.

The [FLUX.1 \[dev\] model](https://huggingface.co/black-forest-labs/FLUX.1-dev) (by BlackForestLabs) is an advanced AI image generation model that produces outstanding output quality. [It is a 12 billion parameter rectified flow transformer capable of generating images from text descriptions](https://huggingface.co/black-forest-labs/FLUX.1-dev). It features competitive prompt following, and is trained using guidance distillation. Additionally, the generated outputs can be utilized for personal, scientific, and non-commercial purposes as outlined in the [FLUX.1 \[dev\] Non-Commercial License](https://huggingface.co/black-forest-labs/FLUX.1-dev/blob/main/LICENSE.md).

In this tutorial, we will learn how to set up a Streamlit application, integrate the FLUX model for real-time image generation, and deploy the application using Docker and Koyeb, ensuring a scalable image generation service.

You can deploy the FLUX application as built in this tutorial using the [Deploy to Koyeb](https://www.koyeb.com/docs/build-and-deploy/deploy-to-koyeb-button) button below:

[![Deploy to Koyeb](https://www.koyeb.com/static/images/deploy/button.svg)](https://app.koyeb.com/deploy?name=flux-pytorch&type=git&repository=github.com/koyeb/example-flux-1-pytorch&branch=main&builder=dockerfile&instance_type=gpu-nvidia-rtx-4000-sff-ada)

## Requirements

To successfully follow this tutorial, you will need the following:

- [Git](https://git-scm.com/) installed
- [Python 3.6+](https://www.python.org/downloads/release/python-360/) or later
- A [Hugging Face](https://huggingface.co/) account
- A [Koyeb](https://app.koyeb.com/) account

## Understanding of the components

### Text-To-Image Generation and FLUX Model

The process of text-to-image generation involves the model interpreting input text and translating it into visual representations. This process consists of several steps: first, the model encodes the textual input into a latent space, capturing the semantic meaning of the words. Next, it employs a generative process to sample from this latent space, producing images that align with the described concepts. The FLUX model, developed by BlackForestLabs, is a state-of-the-art 12 billion parameter rectified flow transformer trained on extensive datasets containing pairs of images and their corresponding textual descriptions. This training enables the model to learn the intricate relationships between language and visual content. Consequently, users can input detailed prompts, and the FLUX model generates images that reflect those prompts with accuracy (and creativity).

### Streamlit

Streamlit is an open-source Python library designed to create interactive data applications, often referred to as dashboards. It empowers developers to build and share data apps simply and intuitively, eliminating the need for extensive web development expertise.

Streamlit apps are created as Python scripts, which are then executed within the Streamlit environment. The library offers a set of functions that can be utilized to add interactive elements to the app such as upload file button.

## Steps

1. [Set up the environment](#set-up-the-environment): Start by setting up your project directory, installing necessary dependencies, and configuring environment variables.
2. [Set up Streamlit](#set-up-streamlit): Next, install Streamlit and create the initial user interface for your application.
3. [Generate AI Images with FLUX Model](#generate-ai-images-with-flux-model): Using the FLUX model to generate AI images based on the user prompt and configuration.
4. [Dockerize the Streamlit application](#dockerize-the-streamlit-application): Create a Dockerfile to containerize your application for consistent deployment.
5. [Deploy to Koyeb](#deploy-to-koyeb): Finally, deploy your application on the Koyeb platform.

## Set up the environment

Let's start by creating a new Streamlit project. To keep your Python dependencies organized you should create a virtual environment.

First, create and navigate into a local directory:

```bash
# Create and move to the new directory
mkdir example-koyeb-flux-images
cd example-koyeb-flux-images
```

Afterwards, create and activate a new virtual environment:

```bash
# Create a virtual environment
python -m venv venv

# Active the virtual environment (Windows)
.\venv\Scripts\activate.bat

# Active the virtual environment (Linux)
source ./venv/bin/activate
```

Now, create a `requirements.txt` file with the following dependencies:

```txt
streamlit
watchdog
diffusers
torch
torchvision
einops
huggingface_hub[hf_transfer]
safetensors
sentencepiece
transformers
tokenizers
protobuf
requests
invisible-watermark
accelerate
peft
```

In terms of dependencies, we have included Streamlit for building a web app in Python, Hugging Face for real-time local use of the FLUX model, and watchdog to monitor file system events. The `accelerate` package is used to streamline the training and inference processes of machine learning models, making it easier to manage distributed training and optimize performance. The `invisible-watermark` package is utilized to embed invisible watermarks in generated content, ensuring uniqueness of the outputs produced by the model.

Now, you can install the dependencies with the following command:

```bash
pip install -r requirements.txt
```

Now, let's move on to creatng a new Streamlit project.

## Set up Streamlit

In this step, you will set up the Streamlit UI that will define the visual layout of the page, and the ability for users to load the FLUX model to start generating images with AI. All the logic for the project will reside in this file, so you can start by creating an `app.py` file with the following code:

```python
# File: app.py

import os, torch, streamlit
from diffusers import FluxPipeline
from huggingface_hub import login, snapshot_download

# Log in to Hugging Face using the provided token from environment variables
login(token=os.getenv('HF_TOKEN'))

# Set the title of the Streamlit application
streamlit.title("AI Image Generation with FLUX.1-dev")

# Create a text input field for the user to enter a prompt for image generation
prompt = streamlit.text_input("Enter your prompt:", "Astronaut in a jungle, cold color palette, muted colors, detailed, 8k")

# Create four columns for user inputs
col1, col2, col3, col4 = streamlit.columns(4)

# Column 1: Input for image width
with col1:
    num_width = streamlit.number_input("Width:", min_value=1, value=256)

# Column 2: Input for image height
with col2:
    num_height = streamlit.number_input("Height:", min_value=1, value=256)

# Column 3: Input for the number of images to generate
with col3:
    num_images = streamlit.number_input("Images:", min_value=1, value=3)

# Column 4: Input for the number of inference steps
with col4:
    num_inference_steps = streamlit.number_input("Steps:", min_value=1, value=3)

# Button to load the model from Hugging Face
if streamlit.button("Load Model"):
    snapshot_download(repo_id="black-forest-labs/FLUX.1-dev", cache_dir="./FLUX_1_dev")
```

The code above does the following:

- Begins with importing the Streamlit module.
- Logs into Hugging Face using a token stored in environment variables, allowing access to the model repository.
- Defines text values for each step and page's title and description
- Creates four columns to organize user inputs for image dimensions, the number of images, and inference steps. Each column contains a number input field with specified minimum values and default values.
- Creates a `Load Model` button to load the FLUX model from Hugging Face. When clicked, it downloads the model snapshot to a specified cache directory using `snapshot_download` function.

With this, you have setup a UI that is able to load the FLUX model per user request. Now, let's move on to generating AI images based on the user prompt and settings.

## Generate AI Images with FLUX Model

In this step, you will invoke FLUX model to generate AI images based on the user prompt. Per the default values set in the input, the model will be generate 3 images for each prompt by user. Make the following additions in the `app.py` file:

```python
# File: app.py

# Existing code

# Button to generate images using the FLUX model
if streamlit.button("Generate Image"):
    # Load the FLUX model with specified parameters
    pipe = FluxPipeline.from_pretrained("black-forest-labs/FLUX.1-dev", torch_dtype=torch.bfloat16, cache_dir="./FLUX_1_dev")
    
    # Generate images based on the user input parameters
    images = pipe(prompt=prompt, num_inference_steps=num_inference_steps, height=num_height, width=num_width, num_images_per_prompt=num_images).images
    
    # Create three columns to display the generated images
    cols = streamlit.columns(3)
    
    # Loop through the generated images and display them in the columns
    for i, image in enumerate(images):
        cols[i % 3].image(image, caption=prompt)
```

The code addition above does the following:

- Creates a button titled `Generate Image`. When clicked, it will instantiate a Flux pipeline for text-to-image generation.
- Invokes the pipeline with the user configured parameters to generate the desired height and width, number of inference steps to be taken, and number of images to be generated.
- Creates three columns to organize the generated images.

Now, you can run the Streamlit application with:

```bash
streamlit run ./app.py --server.port 8000
```

The application would now be ready on [http://localhost:8000](http://localhost:8000). Test the application by generating AI images by altering the default input values and your own custom prompt.

Now, let's dockerize the application to ensure consistency between multiple deployments.

## Dockerize the Streamlit application

Dockerizing deployments helps by creating a consistent and reproducible environment, ensuring that the application runs the same way on any system. It simplifies dependencies management and enhances scalability, making deployments more efficient and reliable. To dockerize, create a `Dockerfile` at the root of your project with the following code:

```Dockerfile
FROM python:3.12 AS runner

WORKDIR /app

COPY app.py requirements.txt .

RUN pip install -r requirements.txt --root-user-action=ignore
RUN pip install --pre torch --index-url https://download.pytorch.org/whl/nightly/cu121 --root-user-action=ignore # [!code ++]

RUN apt-get update 
RUN apt-get install -y libsm6 libxext6 git git-lfs # [!code ++]
RUN rm -rf /var/lib/apt/lists/*

EXPOSE 8000 # [!code ++]

ENV HF_HUB_ENABLE_HF_TRANSFER=1

ARG HF_TOKEN # [!code ++]
CMD ["streamlit", "run", "./app.py", "--server.port", "8000"] # [!code ++]
```

Apart from the usual Dockerfile to deploy Python applications, following tweaks/additions have been made in this code:

- `pip install --pre torch --index-url https://download.pytorch.org/whl/nightly/cu121 --root-user-action=ignore` to install PyTorch with CUDA support for GPU acceleration.
- `RUN apt-get update && apt-get install -y libsm6 libxext6 git git-lfs && rm -rf /var/lib/apt/lists/*` is used to install `git-lfs` and `git`, and then clean up package lists to reduce image size.
- `EXPOSE 8000` is used to specify the port on which the Streamlit application will run.
- `CMD ["streamlit", "run", "./app.py", "--server.port", "8000"]` is used to define the command to start the Streamlit app on port 8000.

With all configured, let's move on to deploy the application to Koyeb.

## Deploy to Koyeb

Now that you have the application running locally you can also deploy it on Koyeb and make it available on the internet.

Create a [new repository on your GitHub account](https://github.com/new) so that you can push your code.

You can download a [standard `.gitignore` file](https://raw.githubusercontent.com/github/gitignore/main/Python.gitignore) for Python from GitHub to exclude certain directories and files from being pushed to the repository:

```bash
curl -L https://raw.githubusercontent.com/github/gitignore/main/Python.gitignore -o .gitignore
```

Run the following commands in your terminal to commit and push your code to the repository:

```bash
git init
git add .
git commit -m "first commit"
git branch -M main
git remote add origin [Your GitHub repository URL]
git push -u origin main
```

You should now have all your local code in your remote repository. Now it is time to deploy the application.

Within the [Koyeb control panel](https://app.koyeb.com/), while on the Overview tab, initiate the app creation and deployment process by clicking Create Web Service.

1. Select **GitHub** as the deployment method.
2. Select your repository from the menu. Alternatively, deploy from the [example repository associated with this tutorial](https://github.com/koyeb/example-flux-1-pytorch) by entering `https://github.com/koyeb/example-flux-1-pytorch` in the public repository field.
4. In the Instance selection, select a GPU Instance.
5. Set your HuggingFace access token in the `HF_TOKEN` environment variable.
6. Finally, click the **Deploy** button.

Once the application is deployed, you can visit the Koyeb service URL (ending in `.koyeb.app`) to access the Streamlit application.

## Conclusion

In this tutorial, you built an AI Image Generation application using FLUX dev model with Streamlit framework. During the process, you learned how to invoke the Flux pipeline in python to generate AI images on the go, customized with user fed prompt and settings, and use the Streamlit framework to quickly prototype the user interface.

Given that the application was deployed using the Git deployment option, subsequent code push to the deployed branch will automatically initiate a new build for your application. Changes to your application will become live once the deployment is successful. In the event of a failed deployment, Koyeb retains the last operational production deployment, ensuring the uninterrupted operation of your application.

Perform real-time inference on high-performance infrastructure with zero-configuration.

Real-Time Inference with No-Ops

Learn how to use the FLUX model to generate images based on prompts in real-time.

Use FLUX, PyTorch, and Streamlit to Build an AI Image Generation App

Understand how the YOLO algorithm works and use it to identify and manipulate images through computer vision.

Using YOLO for Real-Time Object Detection with Koyeb GPUs

Deploy your AI applications across continents and run on an edge network of 250 locations.

Low latency AI workloads

Learn how to use Groq, speech-to-text (STT), and text-to-speech (TTS) to build an app to automatically translate between languages in real-time.

Using Groq to Build a Real-Time Language Translation App


## Introduction

[LlamaIndex](https://www.llamaindex.ai/) is a data framework that makes it simple to build production-ready applications from your data using LLMs. Specifically, LlamaIndex specializes in context augmentation, a technique of providing custom data as context for queries to generalized LLMs allowing you to inject your specific contextual information without the trouble and expense of fine-tuning a dedicated model.

In this guide, we will demonstrate how to build an application with LlamaIndex and [Streamlit](https://streamlit.io/), a Python framework for building and serving data-based applications, and deploy it to Koyeb. The application will deploy an example web app that allows users to ask questions about custom data. In our example, this custom text will be the story ["The Gift of the Magi" by O. Henry](https://en.wikipedia.org/wiki/The_Gift_of_the_Magi).

You can deploy and preview the example application from this guide with [our LlamaIndex One-Click app](https://www.koyeb.com/deploy/llamaindex) or by clicking the [Deploy to Koyeb button](/docs/build-and-deploy/deploy-to-koyeb-button) below:

[![Deploy to Koyeb](/static/images/deploy/button.svg)](https://app.koyeb.com/deploy?name=example-llamaindex&type=git&repository=koyeb%2Fexample-llamaindex&branch=main&instance_type=micro&builder=dockerfile&env%5BOPENAI_API_KEY%5D=CHANGE_ME&ports=8000%3Bhttp%3B%2F)

Be sure to set the `OPENAI_API_KEY` environment variable during configuration. You can consult the [repository on GitHub](https://github.com/koyeb/example-llamaindex) to find out more about the example application that this guide uses.

## Requirements

To successfully follow and complete this guide, you need:

- [Python 3.11](https://www.python.org/downloads/) installed on your local computer.
- A [GitHub account](https://github.com/) to host your LlamaIndex application.
- A [Koyeb account](https://app.koyeb.com) to deploy and run the preview environments for each pull request.
- An [OpenAI](https://openai.com/) API key so that our application can send queries to OpenAI.

## Steps

To complete this guide and deploy a LlamaIndex application, you'll need to follow these steps:

1. [Set up the project directory](#set-up-the-project-directory)
2. [Install project dependencies and fetch custom data](#install-project-dependencies-and-fetch-custom-data)
3. [Create the LlamaIndex application](#create-the-llama-index-application)
4. [Test the application](#test-the-application)
5. [Create a Dockerfile](#create-a-dockerfile)
6. [Deploy to Koyeb](#deploy-to-koyeb)

## Set up the project directory

To get started, create and then move into a project directory that will hold the application and assets we will be creating:

```bash
mkdir example-llamaindex
cd example-llamaindex
```

Next, create and activate a new Python virtual environment for the project. This will isolate our project's dependencies from system packages to avoid conflicts and offer better reproducability:

```bash
python -m venv venv
source venv/bin/activate
```

Your virtual environment should now be activated.

## Install project dependencies and fetch custom data

Now that we are working within a virtual environment, we can begin to install the packages our application will use and set up the project directory.

First, install the LlamaIndex and Streamlit packages so that we can use them to build the application. We can also take this opportunity to make sure that the local copy of `pip` is up-to-date:

```bash
pip install --upgrade pip llama-index streamlit
```

After installing the dependencies, record them in a `requirements.txt` file so that we can install the correct versions for this project at a later time:

```bash
pip freeze > requirements.txt
```

Next we will download the story that we will be using as context for our LLM prompts. We can download a PDF copy of "The Gift of the Magi" by O. Henry from [TSS Publishing](https://theshortstory.co.uk/about/) a platform for short fiction that hosts [free short stories](https://theshortstory.co.uk/resources/free-short-stories/).

Create a `data` directory to hold the contextual data for our application and then download a copy of the story by typing:

```bash
mkdir data
curl -L https://theshortstory.co.uk/devsitegkl/wp-content/uploads/2015/06/Short-stories-O-Henry-The-Gift-of-the-Magi.pdf -o data/gift_of_the_magi.pdf
```

You should now have a PDF file that we cat load into our application and attach as context to our LLM queries.

## Create the LlamaIndex application

We have everything in place to start writing our LlamaIndex application. Create an `app.py` file in your project directory and paste in the following content:

```python
# app.py
import os.path
import streamlit as st

from llama_index.core import (
    VectorStoreIndex,
    SimpleDirectoryReader,
    StorageContext,
    load_index_from_storage,
)


# check if storage already exists
PERSIST_DIR = "./storage"
if not os.path.exists(PERSIST_DIR):
    # load the documents and create the index
    documents = SimpleDirectoryReader("data").load_data()
    index = VectorStoreIndex.from_documents(documents)
    # store it for later
    index.storage_context.persist(persist_dir=PERSIST_DIR)
else:
    # load the existing index
    storage_context = StorageContext.from_defaults(persist_dir=PERSIST_DIR)
    index = load_index_from_storage(storage_context)

query_engine = index.as_query_engine()

# Define a simple Streamlit app
st.title('Ask Llama about "The Gift of the Magi"')
query = st.text_input("What would you like to ask? (source: data/gift_of_the_magi.pdf)", "What happens in the story?")

# If the 'Submit' button is clicked
if st.button("Submit"):
    if not query.strip():
        st.error(f"Please provide the search query.")
    else:
        try:
            response = query_engine.query(query)
            st.success(response)
        except Exception as e:
            st.error(f"An error occurred: {e}")
```

Let's go over what the application is doing.

It begins by importing the basic packages and modules it will use to create the application. This includes Streamlit (aliased as `st`) as well as functionality from LlamaIndex for loading data and indexes from directories, building vector indexes, and attaching contexts.

Next, we set up some semi-persistent storage for the index files. This logic helps us avoid creating an index from our data document every time we run the application by storing index information in a `storage` directory the first time it is evaluated. The application can load the index data from the `storage` directory directly on subsequent runs to increase performance.

Afterward the index is created or loaded, we create a query engine based on it and begin constructing the application frontend with Streamlit. We add a input field that will be translated into our query and then display the results upon submission.

## Test the application

We can test that the application works as expected on your local machine.

First, set and export the `OPENAI_API_KEY` environment variable using your OpenAI API key as the value:

```bash
export OPENAI_API_KEY="<YOUR_OPENAI_API_KEY>"
```

Next, run the application by typing:

```bash
streamlit run app.py
```

This will start the application server. Navigate to `http://127.0.0.1:8501` in your web browser to view the page prompting for your questions about "The Gift of the Magi". You can submit the default query or ask any other questions you have about the story.

When you are finished, press <kbd>CTRL</kbd>-<kbd>C</kbd> to stop the server.

## Create a Dockerfile

During the deployment process, Koyeb will build our project from a `Dockerfile`. This gives us more control over the version of Python, the build process, and the runtime environment. The next step is to create this `Dockerfile` describing how to build and run the project.

Before we begin, create a basic `.dockerignore` command. This will define files and artifacts that we don't want to copy over into the Docker image. In this case, we want to avoid copying the `venv/` and `storage/` directories since the image will install dependencies and manage cached index files at runtime:

```gitignore
# .dockerignore
storage/
venv/
```

Next, create a `Dockerfile` with the following contents:

```dockerfile
# Dockerfile
FROM python:3.11-slim

WORKDIR /app
COPY . .
RUN pip install --requirement requirements.txt && pip cache purge

ARG PORT
EXPOSE ${PORT:-8000}

CMD streamlit run --server.port ${PORT:-8000} app.py
```

The image uses the `3.11-slim` tag of the [official Python Docker image](https://hub.docker.com/_/python) as its starting point. It defines a directory called `/app` as the working directory and copies all of the project files inside. Afterwards, it installs the dependencies from the `requiremnts.txt` file.

The `PORT` environment variable is also defined as a build variable. This allows us to set it at build time to adjust the port that the image will listen on. We use this value in the `EXPOSE` instruction and again in the main `streamlit` command we run with the `CMD` instruction. Both values will use port 8000 as a fallback if `PORT` is not defined explicitly.

## Publish the repository to GitHub

The application is almost ready to deploy. We just need to commit the changes to Git and push the repository to GitHub.

In the project directory, initialize a new Git repository by running the following command:

```bash
git init
```

Next, download a generic `.gitignore` file for Python projects from GitHub:

```bash
curl -L https://raw.githubusercontent.com/github/gitignore/main/Python.gitignore -o .gitignore
```

Add the `storage/` directory to the `.gitignore` file so that Git ignores the cached vector index files:

```bash
echo "storage/" >> .gitignore
```

You can also optionally add the [Python runtime version](https://www.koyeb.com/docs/build-and-deploy/build-from-git/python#runtime) to a `runtime.txt` file if you want to try to build the repository from the Python buildpack instead of the Dockerfile:

```bash
echo "python-3.11.8" > runtime.txt
```

Next, add the project files to the staging area and commit them. If you don't have an existing GitHub repository to push the code to, [create a new one](https://www.github.com/new) and run the following commands to commit and push changes to your GitHub repository:

```bash
git add :/
git commit -m "Initial commit"
git remote add origin git@github.com:<YOUR_GITHUB_USERNAME>/<YOUR_REPOSITORY_NAME>.git
git branch -M main
git push -u origin main
```

**Note:** Make sure to replace `<YOUR_GITHUB_USERNAME>/<YOUR_REPOSITORY_NAME>` with your GitHub username and repository name.

## Deploy to Koyeb

Once the repository is pushed to GitHub, you can deploy the LlamaIndex application to Koyeb. Any changes in the deployed branch of your codebase will automatically trigger a redeploy on Koyeb, ensuring that your application is always up-to-date.

To get started, open the [Koyeb control panel](https://app.koyeb.com/) and complete the following steps:

1. On the **Overview** tab, click **Create Web Service**.
2. Select **GitHub** as the deployment method.
3. Choose the repository containing your application code. Alternatively, you can enter our public [LlamaIndex example repository](https://github.com/koyeb/example-llamaindex) into the **Public GitHub repository** at the bottom of the page: `https://github.com/koyeb/example-llamaindex`.
4. In the **Builder** section, choose **Dockerfile**.
5. Choose an **Instance** of size **micro** or larger.
6. Expand the **Environment variables** section and click **Add variable** to configure a new environment variable. Create a variable called `OPENAI_API_KEY`. Select the **Secret** type and choose **Create secret** in the value. In the form that appears, create a new secret containing your OpenAI API key.
7. Choose a name for your App and Service, for example `example-llamaindex`, and click **Deploy**.

Koyeb will clone the GitHub repository and use the `Dockerfile` file to build a new container image for the project. Once the build is complete, a container will be started from the image to run your application.

Once the deployment is healthy, visit your Koyeb Service's subdomain (you can find this on your Service's detail page). It will have the following format:

```
https://<YOUR_APP_NAME>-<KOYEB_ORG_NAME>.koyeb.app
```

You should see your LlamaIndex application's prompt, allowing you to ask questions about the story and get responses from the OpenAI API.

## Conclusion

In this guide, we discussed how to build and deploy an LLM-based web app to Koyeb using LlamaIndex and Streamlit. The application loads a story from a PDF on disk and sends this as additional context when submitting user-supplied queries. This allows you to customize the focus of the query without having to fine-tune a model for the purpose.

This tutorial demonstrates a very simple implementation of these technologies. To learn more about how LlamaIndex can help you use LLMs to answer questions about your own data, take a look at the [LlamaIndex documentation](https://docs.llamaindex.ai/en/stable/index.html).


Deploy LlamaIndex and your favorite cutting-edge AI tech across continents. No credit card required.

Learn how to use LlamaIndex, a data framework specializing in context augmentation, to quickly query general LLMs with custom data.

Streamlit

Use FLUX, PyTorch, and Streamlit to Build an AI Image Generation App

Using YOLO for Real-Time Object Detection with Koyeb GPUs

Using Groq to Build a Real-Time Language Translation App

Use LlamaIndex, Streamlit, and OpenAI to Query Unstructured Data