Build the catalog

In this article...

We'll discuss the product catalog that feeds product recommendations, how it is built, and what you can do to get greater product coverage.

Product coverage

In our experience, few clients have obtained complete product coverage by only collecting QProtocol events. For this reason, we recommend providing an alternative source of product data such as a Google product feed.

WARNING: For an alternative source to function correctly, the product Ids used must match the Ids that we collect from the ecProduct event.

The product catalog from QProtocol is produced from events collected over the last 14 days (rolling window). Products that don't appear in the last fourteen days are automatically removed. For alternative sources, products disappear from the catalog immediately after they have been removed from the source. We refresh any source every 4 hours.

Where the available products are populated by an alternative source, data from QProtocol is automatically mixed with data from the alternative source. For example, if name is not provided in the alternative source but is provided within QProtocol, that field will be available in the catalog. However, it will only be available for products that appeared within the last 14 days and hence has to be treated as optional.

INFO: Marking a product availability: out of stock inside a Google product feed will set its stock to 0, even if a different stock is provided in QProtocol.

When an alternative source URL is configured, the target server has to respond within 60s (time to first byte) and complete a read within 60s (time to read after first byte). We support the following protocols: HTTP, HTTPS, SFTP, FTP, and FTPS. Simple authentication via username and password can be provided.

Locales

A product catalog can exist in multiple languages and currencies. We build a product catalog for every combination of ecView.language and ecProduct.product.price.currency, sent to us via QProtocol.

Please refer to the following restrictions:

  • Language must be an IETF tag, e.g. en (ISO 639) or en-us (ISO 639 - ISO 3166-1)
  • Currency must be an ISO 4217 currency code, e.g. GBP, USD
  • The locale must be in the format <language>-<currency>, e.g. en-GB-GBP

When QProtocol is the only source of product catalog data, we only consider additional locales that have a product catalog coverage of 25% in comparison to the largest locale on the website.

For example, if a website with a locale "en-US-USD" has 2000 products, we would need to see 500 products over the past two weeks before the additional locale is available.

INFO: If a locale has not reached sufficient product coverage, it will not be available for selection when adding a new product feed.

INFO: Coverage can change over time based on your user's behavior. We therefore recommend providing an alternative source of product information. You can provide up to 10 different sources for locales.

Once a locale is available, it can be utilized in a call to the Recommendations API. When using the Qubit recommendation package, this is automatically done by introspecting the last ecView event.

https://recs.qubit.com/vc/recommend/2.1/TID?strategy=XXX&locale=en-USD

For websites without multiple languages and currencies, or where a client does not wish to show recommendations to other locales, it is not necessary to provide a locale. The default behavior when no locale is provided is to return the most common language/currency combination.

Limitations

A product within the catalog needs to conform to certain rules:

  • A product can't be part of more than 10 category hierarchies
  • A product can't be part of more than 25 categories irrespective of the hierarchy, e.g. "Sale > Shoes", "Fall Promotion" is 3 categories
  • A product image URL can't exceed 4000 characters
  • A product Id can't exceed 60 characters
  • The product data overall can't exceed 5KB - this means you cannot embed images

QProtocol setup

In this setup, Recommendations will automatically get product information from the QProtocol events generated on your site.

Refer to the following table for details of which events and event fields are required to run recommendations in the ecommerce vertical. We also identify fields that we recommend you implement for data completeness but are optional.

ecView

Field (implementation)

Description

type (required)

Reports the page type, e.g. home, category, search, product, basket, checkout, confirmation, help, contact, registration, content, account, or other

subtypes (recommended)

An unordered list of subtypes to describe the view

language (recommended)

The language used to render the page in this view, which must be an IETF language tag

country (recommended)

The selected country for the view, which must be an ISO 3166-1 alpha-2 code, e.g. GB, FR, US

currency (recommended)

The ISO 4217 currency for the user e.g. GBP, USD

ecProduct

Field (implementation)

Description

eventType (required)

The type of product event, e.g. ‘listing’, ‘detail’, or ‘linked_product’

product.sku (required)

Unique product identifier

product.productId (required)

A unique Id to identify a product and all of its size, color, pattern, material, age, group, gender variants

product.name (required)

The product’s name, which should match the name shown on the product page

product.stock (recommended)

The number of available units in stock

product.color (recommended)

The product's color

product.size (recommended)

The product's size

product.price.value (required)

The price of the product after discounts, promotions, etc, rounded to 2 decimal places

product.price.currency (required)

The ISO 4217 currency code e.g. GBP, USD

product.url (required)

The URL of the product’s page

product.description (recommended)

An accurate description of the product, which should match the description on the product page

product.categories (required)

A list of one or more product categories the product belongs to. Each category is a full category path, with each level separated by >

product.category (required)

An array of categories, highest level first, defining the product. Recommended to not exceed 4 items long

product.images (required)

An array containing the URLs of the product’s images. The main image you want to use to display the product must be the first element in the array

product.originalPrice.value (required)

The price the product was originally sold at before any promotions or discounts

product.originalPrice.currency (required)

The ISO 4217 currency code e.g. GBP, USD

product.manufacturer (recommended)

The product's manufacturer

ecBasketTransactionSummary

Field (implementation)

Description

transaction.id (required)

A unique transaction Id

basket.subtotalIncludingTax.value (recommended)

The basket subtotal, including tax, but before the application of discounts, promotions, shipping costs, etc, rounded to 2 decimal places

basket.subtotalIncludingTax.currency (recommended)

The ISO 4217 currency code e.g. GBP, USD

basket.total.value (required)

The basket total after the application of discounts, promotions, shipping costs, etc, rounded to 2 decimal places

basket.total.currency (required)

The ISO 4217 currency code e.g. GBP, USD

ecBasketItemTransaction

Field (implementation)

Description

transaction.id (required)

A unique transaction Id

basket.subtotalIncludingTax.value (recommended)

The basket subtotal, including tax, but before the application of discounts, promotions, shipping costs, etc, rounded to 2 decimal places

basket.subtotalIncludingTax.currency (recommended)

The ISO 4217 currency code e.g. GBP, USD

basket.total.value (recommended)

The basket total after the application of discounts, promotions, shipping costs, etc, rounded to 2 decimal places

basket.total.currency (recommended)

The ISO 4217 currency code e.g. GBP, USD

product.sku (required)

Unique product identifier

product.productId (required)

A unique Id to identify a product and all of its size, color, pattern, material, age, group, gender variants

product.name (recommended)

The product’s name, which should match the name shown on the product page

product.stock (recommended)

The number of available units in stock

product.price.value (recommended)

The price of the product after discounts, promotions, etc, rounded to 2 decimal places

product.price.currency (recommended)

The ISO 4217 currency code e.g. GBP, USD

product.url (recommended)

The URL of the product’s landing page

product.description (recommended)

An accurate description of the product, which should match the description on the product page

product.categories (recommended)

A list of one or more product categories the product belongs to. Each category is a full category path, with each level separated by >

product.category (recommended)

An array of categories, highest level first, defining the product. Recommended to not exceed 4 items long

product.images (recommended)

An array containing the URLs of the product’s images

quantity (recommended)

The number of products described by the line item

subtotal.value (recommended)

The subtotal for the current items taking into account the current price and quantity, rounded to 2 decimal places

subtotal.currency (recommended)

The ISO 4217 currency code e.g. GBP, USD

INFO: For other verticals, such as egaming, it is necessary to emit other events.

Google product feed setup

In this setup, Recommendations uses a server-side integration with a google product feed to get product information. You can import a google product feed into Qubit by following the steps outlined in Importing A Google Product Feed.

The following information needs to be available in the product feed:

Attribute

Description

Example

id

Your product’s unique identifier. Use the product's SKU where possible

A2B4

title

Your product’s name that accurately that matches the title from your landing page

Mens Pique Polo Shirt

description

Your product’s description that matches the description from your landing page

Red, 100% cotton, large men’s t-shirt

link

Your product’s landing page, starting with http or https

image_​​link

The URL of your product’s main image

availability

Your product's availability

in stock

price

Your product's price, including currency

15.00 USD

product_type

The full path of the category or the numerical category Id

Apparel & Accessories > Clothing > Outerwear > Coats & Jackets or 371

Refer to the Products data specification for details of optional attributes and full specifications.

Refer to the following examples:

<?xml version="1.0"?>
<rss xmlns:g="http://base.google.com/ns/1.0" version="2.0">
  <channel>
    <title>Dummy Client - Online Store</title>
    <link>http://dummy-client.qubitproducts.com</link>
    <description>This is a sample feed containing the required and recommended attributes for a variety of different products</description>

    <item>
      <g:id>UNIQUE_ID_00001</g:id>
      <g:title>Two tone dress</g:title>
      <g:description>Two tone dress</g:description>
      <g:link>http://dummt-client.qubitproducts.com/dresses/two-tone-dress.html</g:link>
      <g:image_link>http://dummy-client.qubitproducts.com/media/catalog/product/cache/1/image/800x800/9df78eab33525d08d6e5fb8d27136e95/d/r/dress_1.jpg</g:image_link>
      <g:availability>in stock</g:availability>
      <g:price>159.00 USD</g:price>
      <g:product_type>Women's clothing > Dresses</g:product_type>
    </item>

    <item>
      <g:id>UNIQUE_ID_00002</g:id>
      <g:title>Two tone shirt</g:title>
      <g:description>Two tone shirt</g:description>
      <g:link>http://dummy-client.qubitproducts.com/shirts/two-tone-shirt.html</g:link>
      <g:image_link>http://dummy-client.qubitproducts.com/media/catalog/product/cache/1/image/800x800/9df78eab33525d08d6e5fb8d27136e95/d/r/shirt.jpg</g:image_link>
      <g:availability>in stock</g:availability>
      <g:price>169.00 USD</g:price>
      <g:product_type>Men's clothing > Dresses</g:product_type>
    </item>
  </channel>
</rss>

It is possible to add custom values such as exclude_from_recommendations. It is important to follow the guidelines in Custom attributes in XML data feeds. An updated example is shown below:

<?xml version="1.0"?>
<rss xmlns:g="http://base.google.com/ns/1.0" xmlns:c="http://base.google.com/cns/1.0" version="2.0">
  <channel>
    <title>Dummy Client - Online Store</title>
    <link>http://dummy-client.qubitproducts.com</link>
    <description>This is a sample feed containing the required and recommended attributes for a variety of different products</description>

    <item>
      <g:id>UNIQUE_ID_00001</g:id>
      <g:title>Two tone dress</g:title>
      <g:description>Two tone dress</g:description>
      <g:link>http://dummy-client.qubitproducts.com/dresses/two-tone-dress.html</g:link>
      <g:image_link>http://dummy-client.qubitproducts.com/media/catalog/product/cache/1/image/800x800/9df78eab33525d08d6e5fb8d27136e95/d/r/dress_1.jpg</g:image_link>
      <g:availability>in stock</g:availability>
      <g:price>159.00 USD</g:price>
      <g:product_type>Women's clothing > Dresses</g:product_type>
      <g:sale_price>150.00 USD</g:sale_price>
      <c:exclude_from_recommendations type="boolean">false</c:exclude_from_recommendations>
    </item>

    <item>
      <g:id>UNIQUE_ID_00002</g:id>
      <g:title>Two tone shirt</g:title>
      <g:description>Two tone shirt</g:description>
      <g:link>http://dummy-client.qubitproducts.com/shirts/two-tone-shirt.html</g:link>
      <g:image_link>http://fashion-demo.qubitproducts.com/media/catalog/product/cache/1/image/800x800/9df78eab33525d08d6e5fb8d27136e95/d/r/shirt.jpg</g:image_link>
      <g:availability>in stock</g:availability>
      <g:price>169.00 USD</g:price>
      <g:product_type>Men's clothing > Dresses</g:product_type>
      <g:sale_price>160.00 USD</g:sale_price>
      <c:exclude_from_recommendations type="boolean">true</c:exclude_from_recommendations>
    </item>
  </channel>
</rss>

Next Steps

Once your data layer or product feed has been verified, you can add the Recommendations programmatic experience. You can do this in 2 different ways.

You can either add it as an experience in the Qubit platform or call the Recommendations API.

Last updated: January 2023
Did you find this article useful?