Use pgvector and Hugging Face to Build an Optimized FAQ Search with Sentence Similarity
25 minFor Artificial Intelligence (AI)/ Machine Learning (ML) models to perform their tasks effectively, the data at hand, including videos, images, and texts, must be presented in a format that they can comprehend and use. Vector embeddings play a crucial role in this process by transforming complex data into numerical representations that AI/ML models can process effectively, enabling them to extract meaningful insights and make predictions based on the provided data.
Hugging Face provides an Inference API offering diverse models for a range of tasks, including sentence similarity tasks. In this tutorial, you will use Hugging Face's Inference API alongside pgvector, an open-source vector similarity search extension for PostgreSQL, to create and deploy a simple FAQ search system on Koyeb that leverages text similarity searching.
You can deploy the FAQ search as configured in this guide using the Deploy to Koyeb button below:
Note: You will need to replace the values of the environment variables in the configuration with your own HF_TOKEN
and DB_URL
. Remember to add the ?ssl=true
parameter to the DB_URL
value.
Requirements
To successfully follow this tutorial, you will need the following:
- Node.js and npm installed. The demo app in this tutorial uses version 18 of Node.js
- Git installed
- A Hugging Face account
- A Koyeb account to deploy the application
Steps
To complete this guide and deploy the FAQ text similarity search application, you'll need to follow these steps:
- Generate Hugging Face Token
- Create Postgres Database with Koyeb
- Start the text similarity search project
- Set up the database connection
- Add seed data to the database
- Set up the Express server to respond with related FAQ questions
- Set up the home page view for submitting search terms
- Add the search functionality
- Deploy to Koyeb
Generate Hugging Face Token
HTTP requests to the Hugging Face API require an authorization token. To generate this token, while logged into your Hugging Face account, navigate to the access tokens page and click the "New token" button. Enter a name for your token, and then click the "Generate a token" button to generate it. Copy and securely store this token for later use.
Create Postgres Database with Koyeb
To initiate the creation of a Postgres database, access your Koyeb control panel and navigate to the Databases tab. Next, click on the Create Database Service button. Here, you can either provide a custom name for your database or stick with the default one, choose your preferred region, specify a default role, or keep the default value, and finally, click the Create Database Service button to establish your Postgres database service.
Once you've created the Postgres database service, a list of your existing database services will be displayed. From there, select the newly created database service, copy the database connection string, and securely store it for future use.
Start the text similarity search project
In this section, we'll set up an npm
project with TypeScript and install the essential packages for the demo application. To begin, create a directory on your development machine to organize your code. You can do this by executing the following command in your terminal window:
The command above creates a faq_search
directory, which serves as the root directory for the application, and also a src
directory within it, which will house the project code. Next, run the commands below to initialize a Git repository within the just created faq_search
directory:
The initial command above switches your terminal's current directory to the faq_search directory, while the second command initializes a Git repository within that directory.
Next, initialize an npm
project in the faq_search
directory by executing the following command in your terminal window:
The command above establishes an npm
project in the faq_search
directory, creating a package.json
file. Using -y
skips the questionnaire and fills it with default configurations. Next, in your terminal window, run the command below to install the necessary libraries and packages for building the demo application:
The above command installs the packages passed to the install
command, with the -D
flag specifying the libraries intended for development purposes only.
The libraries installed include:
express
: A web framework for Node.js.ejs
: A JavaScript templating engine.pg
: A PostgreSQL client for Node.js.axios
: A Promise-based JavaScript HTTP client.pgvector
: A vector similarity search library for Node.js
The development-specific libraries include:
dotenv
: A library for handling environment variables.typescript
: Enables the execution of TypeScript code.nodemon
: Detects code changes to restart the application during development.ts-node
: To execute and rebuild TypeScript efficiently.@types/express
: Type definitions for express.@types/pg
: Type definitions for pg.
With the required libraries now installed, create a tsconfig.json
file in the root directory of your project and add the following code to it:
The tsconfig.json
file serves as a configuration file for TypeScript, outlining the parameters for transpiling TypeScript code within a project.
With this last code change, the project set-up is complete. In the following section, you will establish a connection to the Postgres database created in the preceding steps.
Set up the database connection
The node-postgres
(pg) library provides a low-level interface to interact directly with PostgreSQL databases using raw SQL queries.
To initiate the setup of a database connection, generate a .env
file in the root directory of your project and include the following code, replacing the placeholder values with your own:
The addition of the ssl=true
parameter to the DB_URL
value above indicates that the database connection should be established with SSL enabled.
The values added to the .env
file should be kept secret and not included in Git history. To ensure this, create a .gitignore
file by running the command below:
The command above creates a .gitignore
file and includes the .env
file, the node_modules
directory, and all TypeScript-generated JavaScript files into it, ensuring they're excluded from the Git history.
Create the database client
Following that, establish a database client to connect to the database. To achieve this, generate a db
directory in the src
directory. Inside this db
directory, create a db.ts
file and insert the following code:
The provided code begins by importing the dotenv
configuration, followed by importing the Pool
class from the pg
library. It then retrieves the database URL from the environment variables and uses it to create a new pool instance, which is subsequently exported.
Create the database schema
Next, create a schema.ts
file within the db
directory and add the following code to it:
The code above defines how data will be stored, organized and managed in the database. Using the pool
database instance, it executes an SQL query to create the vector
extension within the database if it does not already exist.
The vector
extension enables Postgres databases to store vector embeddings. After creating the vector extension, a subsequent SQL query creates a documents
table within the database. This table comprises three columns:
- An
id
column for storing auto-incrementing unique identifiers for each row in the table. - A
title
column with atext
data type for storing text data. - An
embedding
column with avector(384)
data type. This column will store vector data with a length of 384 elements.
After executing the SQL queries, a message is printed to the console to confirm success or show an error message.
To execute the code added to the schema file, update the script
section of your package.json
file with the following code:
The included db:setup
script runs the code within the schema.ts
file when executed.
Test the database setup locally
To execute it, run the following command in your terminal window:
If the command is executed successfully, you will see a message in your terminal window stating, Database setup complete.
, marking the completion of the database connection setup. The next section will center on adding some sample data to the database.
Add seed data to the database
The FAQ search application will operate by retrieving questions from the database with titles that closely match the submitted search term. In this section, we will insert 100 fictitious travel-related FAQ questions into the database.
Hugging Face provides the SentenceTransformers framework, featuring pre-trained models designed for generating embeddings from sentences, text, and images. Among these models is the all-MiniLM-L6-v2, which will be used to generate embeddings for the seed data to be added to the database.
To get started, create a helpers.ts
file in the src
directory and include the following code within the file:
The code starts by importing various libraries:
dotenv
for managing environment variablesaxios
and theAxiosError
type for HTTP requestspgvector
for handling vector embeddingspool
database instance for connecting to the database
Following that, the ID for the all-MiniLM-L6-v2
model is assigned to a variable named modelId
, and this variable is used to specify the API URL for generating vector embeddings with the model.
The code further defines and exports three functions: saveEmbedding
, findSimilarEmbeddings
, and generateEmbedding
.
- The
saveEmbedding
function executes an SQL query to insert a text and its corresponding vector embedding into thedocuments
table. - Conversely, the
findSimilarEmbeddings
function, when provided with a vector embedding, executes an SQL query to retrieve the top 5 most similar vector embeddings from the database along with their corresponding titles. - Lastly, the
generateEmbedding
function receives a text input and initiates a POST HTTP request to the Hugging Face API, including the Hugging Face authorization token in the request header. The vector embedding obtained from the API's response for the given text is then transformed into an SQL vector using thetoSql
method frompgvector
and subsequently returned.
To generate seed data for the database, create a seed.ts
file in the db
directory. Add the code below to the file:
The code added to the seed.ts
file imports the generateEmbedding
and saveEmbedding
helper functions and defines a travelFAQ
array, which contains 100 travel-related questions.
For each question in the travelFAQ
array, a vector embedding is generated using the generateEmbedding
function and then saved to the database using the saveEmbedding
function. Errors that occur while creating and saving a vector embedding are logged on the console.
To execute the code in the seeds file, update the scripts
section of your package.json
file with the code below:
Test the database locally
The db:seed
script added above executes the seed.ts
file. To run the script, run the code below in your terminal window:
Successfully running the command above should display no error message in your terminal window.
In this section, 100 travel-related questions and their corresponding vector embeddings have been added to the database. The next section will focus on setting up an Express server to use this data.
Set up the Express server to respond with related FAQ questions
An HTTP server is required to accept user input and respond to it with related FAQ questions. To set up an Express server, create an index.ts
file in the src
directory and add the following code to it:
The code above imports the following libraries:
- The
dotenv/config
for accessing environment variables. express
along with types forExpress
,Request
, andResponse
objects for setting up a web server.- The
body-parser
middleware, used for parsing request body data. - The
path
module, which manages file paths.
The code proceeds to create an Express app instance and sets the server's port to the value of the PORT
environment variable or defaults to 4000 if not specified. Additionally, it configures the Express server to use body-parser
for handling URL-encoded data from incoming requests, and it sets the server's view engine to EJS
with the views directory pointing to a folder named views
.
Furthermore, the code defines a route for handling HTTP requests to the root route (/) and renders the index
view upon matching the root route. Finally, the server is initiated, listens for requests on the specified port, and logs a message to confirm it is running.
The Express server setup is now complete. The following section will focus on setting up the view page for submitting search terms.
Set up the home page view for submitting search terms
The demo app needs a page to receive search terms and pass them to the Express server for processing. To set that up, start with creating a views
folder within the src
directory. Within this newly created views
folder, create an index.ejs
file and insert the following code into it:
The code within the index.ejs
file specifies the HTML structure for the index
view rendered by the root route handler. It includes CSS styles for:
- page formatting
- an HTML form featuring an input field for entering search terms
- a submit button for submitting the request.
Additionally, if there are any FAQ results, the titles of each FAQ item are presented in a list.
To view the index view page, modify the scripts
section within your package.json
file to incorporate the command for launching the application:
The dev
script added above serves the index.ts
file using nodemon
and ts-node
and automatically restarts the application whenever a file change is detected.
Test the UI locally
To execute the script, run the command below in your terminal window:
Executing the command above will present a message indicating that the server is up and running, along with the specific port it's using. To access the page, simply open your web browser and enter http://localhost:<YOUR_PORT>
, where you'll find the search form displayed on the web page.
The next section will focus on implementing the capability to search for FAQ questions based on a given search term.
Add the search functionality
The database contains 100 travel-related questions, and a web server is now in place to interact with the database. The final step involves handling search requests and responding with the most similar questions from the database.
To achieve that, start by adding the import below to the index.ts
file:
The code above imports the findSimilarEmbeddings
and generateEmbedding
helper functions. Next, add the code below to the index.ts
file:
The code introduced above defines a route handler to handle POST requests to the /search
route. This route is the action URL for the search form on the index page.
When the route matches a request, the code extracts the submitted search term from the request body and generates a vector embedding for the search term using the generateEmbedding
function. The findSimilarEmbeddings
function then receives the resulting vector embedding, queries the database and returns an array of the top five most similar questions.
Finally, the route handler renders the index
view, passing the array of similar questions to the view through a faqs
variable. Given that the code within the index
view is already set up to use the faqs
variable and display the search results in a list, no additional modification is required in the index
view.
Test the search functionality locally
To test out the functionality, start up the server by running the command below:
With the app running in the browser, search for a travel-related question such as "What happens if my flight is canceled?". You should see a list of related FAQ questions displayed on the page.
You've successfully integrated a search feature that uses the capabilities of ML/AI to provide results closely tied to the search term. In the upcoming section, you will proceed to deploy the application online on the Koyeb platform.
Deploy to Koyeb
With the app now complete, the final step is to deploy it online on Koyeb. Since the app uses a managed Postgres service, the deployment process doesn't include a database setup.
To get started, update the scripts
section of the package.json
file with the code below:
The code above introduced a build
and start
script to the package.json
file. The build
script runs the database schema and compiles the TypeScript code into JavaScript. The start
script executes the compiled JavaScript code with node
.
Next, create a GitHub repository for your code and execute the command below to push your local code to the repository:
To deploy the code on the GitHub repository, visit the Koyeb control panel, and while on the Overview tab, click the Create Web Service button to start the deployment process. On the App deployment page:
- Select GitHub as the deployment method.
- Choose the repository for your code from the repository drop-down menu.
- In the Environment variables section, click Add variable. For every environment variable listed in your
.env
file, input the variable name, select the Secret type, and in the value field, choose the Create secret option. In the ensuing form, specify the secret name along with its corresponding value, and finally, click the Create button. Remember to add the?ssl=true
parameter to theDB_URL
value. - Enter a name for the application or use the provided one.
- Finally, initiate the deployment process by clicking Deploy.
During the deployment on Koyeb, the process identifies the build
and start
scripts outlined in the package.json
file, using them to build and launch the application. The deployment progress can be tracked through the logs presented. Upon the completion of deployment and the successful execution of vital health checks, your application will be operational.
Click on the provided public URL to access your live application.
Conclusion
In this tutorial, you implemented text similarity search in a FAQ search application, leveraging the power of vector embeddings.
The Hugging Face Inference API offers diverse models for various ML/AI tasks. With Koyeb's managed Postgres service supporting the pgvector
extension, you can explore the API's offerings and seamlessly integrate ML/AI functionalities into your applications.
Given that the application was deployed using the Git deployment option, subsequent code push to the deployed branch will automatically initiate a new build for your application. Changes to your application will become live once the deployment is successful. In the event of a failed deployment, Koyeb retains the last operational production deployment, ensuring the uninterrupted operation of your application.