
## Introduction

[LangChain](https://www.langchain.com/) is a powerful framework for building applications with AI language models. It simplifies the process of interfacing with local or remote LLMs by making it easy to template prompts, configure query contexts, and chain discrete processes together to form complex pipelines.

[LangServe](https://www.langchain.com/langserve) is a LangChain project that helps you build and deliver these applications over a REST API. Under the hood, it uses [FastAPI](https://fastapi.tiangolo.com/) to construct routes and build web services, and leverages [Pydantic](https://docs.pydantic.dev/latest/) to handle data validation.

In this guide, we will demonstrate how to build an application with LangChain and LangServe and deploy it to Koyeb. The application will serve a REST API where users can submit queries. It will pass these, with contextual information, to OpenAI to generate responses.

You can deploy and preview the example application from this guide by clicking the [Deploy to Koyeb button](/docs/build-and-deploy/deploy-to-koyeb-button) below:

[![Deploy to Koyeb](/static/images/deploy/button.svg)](https://app.koyeb.com/deploy?name=example-langserve&type=git&repository=koyeb%2Fexample-langserve&branch=main&builder=dockerfile&env%5BOPENAI_API_KEY%5D=CHANGE_ME&ports=8000%3Bhttp%3B%2F)

Be sure to set the `OPENAI_API_KEY` environment variable during configuration. You can consult the [repository on GitHub](https://github.com/koyeb/example-langserve) to find out more about the example application that this guide uses.

## Requirements

To successfully follow and complete this guide, you need:

- [Python](https://www.python.org/downloads/) installed on your local computer.
- A [GitHub account](https://github.com/) to host your LangServe application.
- A [Koyeb account](https://app.koyeb.com) to deploy and run the preview environments for each pull request.
- An [OpenAI](https://openai.com/) API key so that our application can send queries to OpenAI.

## Steps

To complete this guide and deploy a LangServe application, you'll need to follow these steps:

1. [Set up the project directory](#Set-up-the-project-directory)
2. [Create the app directory structure and install dependencies](#Create-the-app-directory-structure-and-install-dependencies)
3. [Create the LangServe application](#Create-the-LangServe-application)
4. [Test the application](#Test-the-application)
5. [Adjust the Dockerfile](#Adjust-the-Dockerfile)
6. [Publish the repository to GitHub](#Publish-the-repository-to-GitHub)
7. [Deploy to Koyeb](#Deploy-to-Koyeb)

## Set up the project directory

To get started, create and then move into a project directory that will hold the application and assets we will be creating:

```bash
mkdir example-langserve
cd example-langserve
```

Inside, create a file called `.env` in your text editor. Inside, define the `OPENAI_API_KEY` environment variable by setting it to your OpenAI API key:

```ini
# .env
OPENAI_API_KEY="<YOUR_OPENAI_API_KEY>"
```

Our application will read the API key from this file to authenticate its requests to OpenAI services.

Next, create and activate a new Python virtual environment for the project. This will isolate our project's dependencies from system packages to avoid conflicts and offer better reproducability:

```bash
python -m venv venv
source venv/bin/activate
```

Your virtual environment should now be activated.

## Create the app directory structure and install dependencies

Now that we are working within a virtual environment, we can begin to install the packages our application will use and set up the project directory.

Standard Python installations include the [`pip` package manager](https://pypi.org/project/pip/) by default. However, the LangServe project uses the [`poetry`](https://python-poetry.org/) by default. Because of this, we'll install our dependencies in two stages.

First, install the `langchain-cli` package to get access to the `langchain` command line tool. We'll also take this opportunity to install `poetry` itself and make sure `pip` is up-to-date:

```bash
pip install -U pip langchain-cli poetry
```

Next, with the newly installed `langchain` command, initialize a LangChain project in the current directory:

```bash
langchain app new .
```

**Note:** Be sure to include the trailing dot to target the current directory.

You will be asked whether you wish to install any packages. Despite the wording, this prompt actually refers to [LangChain templates](https://python.langchain.com/docs/templates/) and not Python packages. Press <kbd>ENTER</kbd> to continue without adding any templates.

With the new project files, your directory should now look similar to this:

```
.
├── app/
│   ├── __init__.py
│   ├── __pycache__/
│   │   └── . . .
│   └── server.py
├── Dockerfile
├── packages/
│   └── README.md
├── pyproject.toml
├── README.md
└── venv/
    └── . . .
```

The `pyproject.toml` file is the primary file that the `langchain` command and `poetry` both use to record dependency information and configure project metadata. Because this now makes the directory a valid `poetry` project, we can use `poetry` to install the remaining dependencies:

- `langserve[all]`: The server and client components of the LangServe library.
- `langchain-openai`: The package containing OpenAI integration for LangChain.
- `python-decouple`: A package that can be used to read environment variables and `.env` files.

```bash
poetry add "langserve[all]" langchain-openai python-decouple
```

Our project directory now has all of the dependencies and project files necessary for us to begin working.

## Create the LangServe application

To create a basic LangServe application, open the `app/server.py` file in your text editor. Inside, replace the existing contents with the following code:

```python
# app/server.py
from decouple import config
from fastapi import FastAPI
from langchain_openai import ChatOpenAI
from langchain.prompts import ChatPromptTemplate
from langserve import add_routes


app = FastAPI()

model = ChatOpenAI(openai_api_key=config("OPENAI_API_KEY"))
prompt = ChatPromptTemplate.from_template("Give me a summary about {topic} in a paragraph or less.")
chain = prompt | model

add_routes(app, chain, path="/openai")

if __name__ == "__main__":
    import uvicorn

    uvicorn.run(app, host="0.0.0.0", port=8000)
```

Let's take a moment to go over what this application does.

The code begins by importing all of the necessary classes, functions, and other materials from the packages we've installed. Afterwards, it initializes a `FastAPI()` instance that will serve as the primary application object for the application.

Next, we initialize instances of the `ChatOpenAI` and `ChatPromptTemplate` classes and assign them to the `model` and `prompt` variables, respectively. For the `ChatOpenAI` instance, we pass in the OpenAI API key from our `.env` file using the `config` object from `python-decouple`. For the `ChatPromptTemplate`, we set the prompt to ask for a summary of the given topic. We then chain these two together in a `chain` variable.

We add a route to serve the new chain at `/openai`. Afterwards, we use `uvicorn` to serve the applcition on all interfaces using port 8000.

## Test the application

We can test that the application works as expected by typing the following inside the main project directory:

```bash
langchain serve
```

This will start up the application server. Navigate to `127.0.0.1:8000/openai/playground` in your web browser to view the prompt page. You can test that everything is working correctly by entering a question or topic.

When you are finished, press <kbd>CTRL</kbd>-<kbd>C</kbd> to stop the server.

## Adjust the Dockerfile

When we initialized a new LangChain project, it copied a `Dockerfile` to our project directory. Koyeb will build a Docker image from this file when we deploy later on. The generated `Dockerfile` works as expected, but we will adjust it slightly to allow Koyeb to pass in the port to run on during the build process.

Open the `Dockerfile` in your text editor and replace the final two lines (`EXPOSE` and `CMD`) with the following:

```dockerfile
# Dockerfile
. . .

EXPOSE 8000 // [!code --]
CMD exec uvicorn app.server:app --host 0.0.0.0 --port 8000 // [!code --]
ARG PORT // [!code ++]
EXPOSE ${PORT:-8000} // [!code ++]
CMD exec uvicorn app.server:app --host 0.0.0.0 --port ${PORT:-8000} // [!code ++]
```

The `ARG` line marks `PORT` as a build argument. Afterwards, we modify the hardcoded instances of port 8000 with a reference to that build argument, retaining 8000 as the fallback value if no build value is provided.

## Publish the repository to GitHub

The application is almost ready to deploy. We just need to commit the changes to Git and push the repository to GitHub.

In the project directory, initialize a new Git repository by running the following command:

```bash
git init
```

The project creation process includes a very basic `.gitignore` file. Replace it with a more complete one for Python projects by typing:

```bash
curl -L https://raw.githubusercontent.com/github/gitignore/main/Python.gitignore -o .gitignore
```

Next, add the project files to the staging area and commit them. If you don't have an existing GitHub repository to push the code to, [create a new one](https://www.github.com/new) and run the following commands to commit and push changes to your GitHub repository:

```bash
git add :/
git commit -m "Initial commit"
git remote add origin git@github.com:<YOUR_GITHUB_USERNAME>/<YOUR_REPOSITORY_NAME>.git
git branch -M main
git push -u origin main
```

**Note:** Make sure to replace `<YOUR_GITHUB_USERNAME>/<YOUR_REPOSITORY_NAME>` with your GitHub username and repository name.

## Deploy to Koyeb

Once the repository is pushed to GitHub, you can deploy the LangServe application to Koyeb. Any changes in the deployed branch of your codebase will automatically trigger a redeploy on Koyeb, ensuring that your application is always up-to-date.

To get started, open the [Koyeb control panel](https://app.koyeb.com/) and complete the following steps:

1. In the **Overview** tab, click **Create Web Service**.
2. Select **GitHub** as the deployment option.
3. Choose the repository containing your application code. Alternatively, you can enter our public [LangServe example repository](https://github.com/koyeb/example-langserve) into the **Public GitHub repository** field at the bottom of the page: `https://github.com/koyeb/example-langserve`.
4. In the **Builder** section, choose **Dockerfile**.
5. In the **Environment variables** section, click **Add Variable** to configure a new environment variable. Create a variable called `OPENAI_API_KEY`. Select the **Secret** type and choose **Create secret** in the value. In the form that appears, create a new secret containing your OpenAI API key.
6. Name the App and Service, for example `example-langserve`.
7. Click the **Deploy** button.

Koyeb will clone the GitHub repository and use the `Dockerfile` file to build a new container image for the project. Once the build is complete, a container will be started from the image to run your application.

Once the deployment is healthy, visit your Koyeb Service's subdomain (you can find this on your Service's detail page) with `/openai/playground` appended to the end. It will have the following format:

```
https://<YOUR_APP_NAME>-<KOYEB_ORG_NAME>.koyeb.app/openai/playground
```

You should see your LangServe application's prompt, allowing you to ask questions and get responses from the OpenAI API.

## Conclusion

In this tutorial, we demonstrated how to use [LangChain](https://www.langchain.com/), [LangServe](https://www.langchain.com/langserve), and the [OpenAI API](https://platform.openai.com/) to build a basic page with an AI prompt. LangChain provides an ergonomic interface for working with OpenAI and other models, while LangServe makes it easy to create and serve sites exposing these tools.

This tutorial is only a basic guide to get you started with these projects. To learn more about how to build robust services using more advanced features of these frameworks, take a look at the [LangChain documentation](https://python.langchain.com/docs/get_started/introduction) and try out some of their [project templates](https://python.langchain.com/docs/templates/).


Deploy LangServe and LangChain worldwide in seconds. No credit card required.

Accelerate AI workloads

Learn how to build and serve an API for LangChain applications with LangServe.

LangServe

Using LangServe to build REST APIs for LangChain Applications