Customer and Developer ❤️

How to create an ecommerce search experience in Vantage

Technical

Introduction to Semantic Search

Modern semantic search based search utilizes sophisticated natural language processing (NLP) techniques to interpret search queries and retrieve relevant information. When a user enters a search query, the LLM analyzes the text to understand its meaning, context, and intent. Drawing from its extensive training on vast amounts of text data, the model grasps the semantic nuances of language, enabling it to comprehend not just the words but also the underlying concepts behind them.

Based on this understanding, the LLM retrieves relevant documents from its index, considering factors like keyword matching, relevance scores, and contextual information to rank the results. Furthermore, it incorporates contextual understanding to provide more precise results, such as factoring in the user's location for location-based queries like finding nearby restaurants. Personalization features are often integrated, leveraging user history, preferences, and behavior to tailor search results to individual interests.

One of the key strengths of LLM-based semantic search systems is their continuous learning capability. They analyze user interactions and feedback to refine their understanding of language and improve their search algorithms over time. This iterative process allows them to adapt to evolving user needs and preferences, delivering increasingly accurate and personalized search results. 

In this tutorial, we will walk you through how to build a modern LLM-based semantic search engine in Vantage Discovery that harnesses the power of advanced NLP to provide your users with more relevant, contextual, and personalized search experiences. Specifically, we will create and search furniture listings crawled from various sellers on Etsy. 

By the end of this tutorial, you will have a furniture collection available via an API and be able to use semantic search to enhance the discovery experience in your application.

We will follow the following steps: 


Prerequisites

You will need the following before we begin: 

  • Vantage Account: Sign up for a Vantage account
  • OpenAI API Key: Generate or retrieve your OpenAI keys on the API Keys page
  • Furniture Sample Data File: Download the data file, which contains approximately 5k listings with images and descriptions

Step 1: Configure Your LLM Provider API Key

Skip this step if you already have an API key. 


  1. 1.Log in to the Vantage console and navigate to API Keys
  2. 2.Click + Add new Model API Key to create a new Model API Key
  3. 3.Input the key from OpenAI
  4. 4.Click Add Key


Step 2: Create a New Collection

In the console, navigate to the collections page.


  1. 1.Create a new Collection
  2. 2.Name your new collection "Furniture Tutorial"
  3. 3.Ensure to set the Collection ID to furniture-tutorial. We will use this when we search.
  4. 4.For Model, select text-embedding-ada-002
  5. 5.Ensure the Model API Key from the previous step is selected
  6. 6.Click on Create Collection
  7. 7.Create Collection


Step 3: Upload Furniture Dataset

Ensure you're on the Upload Data page


  1. 1.Select the collection you just created. In this case "Furniture Tutorial"
  2. 2.Click the + or drag and drop the provided furniture data file to upload it
  3. 3.Click Upload
  4. 4.Upload Data

The upload may take a minute. Once complete you should see a success screen.


Note: Don't re-upload! It might take up to 5 minutes for your file to appear in the file status, but it's usually quicker.



Wait until you see the file processed.


  1. 1.Refresh the Collection Files section.
  2. 2.Review the log once the file has been processed. Your data is now indexed and included in the Vantage platform search!
  3. 3.Wait for the collection to be online, meaning your data is ready for search.
  4. 4.When the collection is online, proceed to Test Collection to search your data.



👍Ensure your collection status indicates Online before proceeding.

Step 4: Search the Collection


Time to search your collection!


  1. 1.Enter “modern sleek armchair for a small space” into the text box
  2. 2.Execute your first Vantage semantic query to search via AI embeddings for the most relevant semantic results based on your query's intent and understanding.
  3. 3.Review the results



The IDs for the furniture dataset don't really provide much context about the result. You may want to copy and paste the ID into a web browser to evaluate the results. 


Data about each product, supporting a different tutorial, is located at https://furniture-json.demo.vanta.ge/<<ID>>.json.


To get the top result, 50777d55cfc98140744d0caea184d594, visit 50777d55cfc98140744d0caea184d594.json

Step 5: Search the Collection using Python


To call the Search API from Python/cURL, you'll need two pieces of information:


  1. 1.
  2. Your account_id, found in the URL of the test collection or collection details page


  1. 1.
  2. Your Vantage API Key for inclusion in the request



In the code below, replace YOUR ACCOUNT_ID GOES HERE and YOUR VANTAGE API KEY GOES HERE with your actual account_id and API Key.


# Python example

import requests


url = "https://api.vanta.ge/v1/search/semantic"


payload = {

    "filter": {"boolean_filter": ""},

    "pagination": {"page": 0, "count": 5},

    "collection": {

        "account_id": "YOUR ACCOUNT_ID GOES HERE",

        "collection_id": "furniture-tutorial",

        "accuracy": 1

    },

    "text": "comfy traditional"

}

headers = {

    "accept": "application/json",

    "content-type": "application/json",

    "authorization": "Bearer YOUR VANTAGE API KEY GOES HERE"

}


response = requests.request("POST", url, json=payload, headers=headers)

print(response.text)


To finish up, run your Python script or cURL command to see the results.


Congratulations! You have completed the Furniture Discovery tutorial. You're now prepared to enrich your e-commerce experience with semantic search capabilities, powered by Vantage.

Why use Vantage Discovery for Semantic Search

As you can see, we are an extremely easy-to-use platform for AI search. We've built a platform that understands your proprietary data and handles all the complexity involved in integrating with AI to deliver breakthrough experiences.


With Vantage Discovery you will get the following out-of-the-box: 


  • Astounding results from natural language queries
  • New discovery experiences based on user-specific and site-specific context
  • Next-generation generative applications for fully personalized use cases


This opens up new opportunities for you to strengthen your customer relationships through innovation in personalization and generative capabilities. Try us out today here.