Open Taxonomy: Bringing LLM Understanding to the World's Products

Open Taxonomy

🌍 Global Standard: Building upon Shopify's open-source, standardized product taxonomy to create an extended universal language for product classification, empowering merchants and developers alike.

👩🏼‍💻 Integration Friendly: Designed with a stable structure and diverse formats for effortless integration into any system.

🚀 Industry Benchmark: Covering a wide array of manufactured products, Open Taxonomy offers extensive categories, attributes, and values, facilitating comprehensive classification.

Learn more about the original project on help.shopify.com

🗂️ Table of Contents

• Open Taxonomy: Bringing LLM Understanding to the World's Products
  • 🗂️ Table of Contents
  • 🕹️ Interactive Explorer
  • 📚 Taxonomy Overview
  • 🧭 Getting Started
  • 🐍 Python SDK
    • Features
    • Installation
    • 🧩 How to Integrate with the Taxonomy
    • 🧑🏼‍🏫 How to Make Changes to the Taxonomy
    • 👩🏼‍💻 How to Evolve the System
  • 🤿 Diving In
  • 🛠️ Setup and Dependencies
    • For users with minidev:
    • For others:
  • 📂 How This Is All Organized
  • 🧑‍💻 Contributing
  • 📅 Releases
  • 📜 License
  • TODO List

🕹️ Interactive Explorer

Ready to dive in? Explore our taxonomy interactively to visualize and discover the extensive categories, attributes, and values.

📚 Taxonomy Overview

Open Taxonomy is an open-source extension of Shopify's comprehensive, global standard for product classification. This project aims to provide an enhanced taxonomy for a variety of manufactured products, along with code abstractions that enable LLM agents to interact with and create their own taxonomies.

🧭 Getting Started

This repository is the home of Open Taxonomy. It houses the source-of-truth data, distribution files for implementation, and the source code that powers the taxonomy system.

We've structured it to be as user-friendly as possible, whether you're looking to integrate the taxonomy into your system, suggest changes, or delve into how it's developed and maintained.

🐍 Python SDK

We also offer a Python SDK to make it easier to interact with the Open Taxonomy project. This SDK provides tools for creating, managing, and querying taxonomy trees, enabling seamless integration with Large Language Models (LLMs).

Features

• Taxonomy Node Management: Create and manage nodes within the taxonomy tree, each representing a category or classification.
• Tree Operations: Perform operations like adding nodes, searching within the tree, and traversing the tree to understand its structure.
• Asynchronous Data Loading: Load taxonomy data asynchronously from a remote JSON source.
• Search and Classification: Utilize natural language processing to classify entries and search within the taxonomy based on a query.
• LLM Integration: Enable Large Language Models to interact with and create taxonomies, making it easier for AI agents to understand and use the classification structure.

Installation

To install the Open Taxonomy Project's Python SDK, clone the repository and install the necessary dependencies using pip:

pip install git+https://github.com/Intuitive-Systems/open-taxonomy-python.git

Head over to the Python SDK Repository for more information

🧩 How to Integrate with the Taxonomy

Head to the dist directory to find the files you need and integrate this taxonomy into your system. Currently available in txt and json formats, with more on the way. If you need a specific format, please open an issue to let us know!

🧑🏼‍🏫 How to Make Changes to the Taxonomy

🔵 Note: While in preview, we are not actively seeking PRs.

All source-of-truth data is located in the data/ directory. Submit PRs here to propose changes to the taxonomy.

👩🏼‍💻 How to Evolve the System

Beyond the data, you'll find the tools we use to manage the taxonomy and generate distributions. This is where the system evolves and improves.

🤿 Diving In

This project is built as a simple Ruby app with models and serializers. It parses the data in data/ into a structured format, stored in app/models/category.rb, and then serializes it to dist/. The app follows a Rails-like structure and uses ActiveRecord.

Primary commands include:

make [build] # Build the dist and documentation files
make clean   # Remove sentinels and all generated files
make seed    # Parse data/ into the local database
make test    # Run Ruby tests and cue schema verification
make server  # Start the interactive view at http://localhost:4000

🛠️ Setup and Dependencies

For users with minidev:

• Run dev up

For others:

• Install ruby, matching the version in .ruby-version
• Install cue, version 0.7.x or higher
• Install make
• Run bundle install

Note: For users with MacOS, make sure your XCode Command Line Tools are installed and up to date. You can do this by running xcode-select --install in your terminal.

When editing cue files, run cue fmt to format them correctly.

📂 How This Is All Organized

If you're diving deeper, here's a quick guide:

./
├── application.rb       # Handles file loading "app-wide"
├── Makefile             # Primary source of useful commands
├── Rakefile             # Only used for testing
├── app/
│   ├── models/          # Simple data object models
│   │   ├── category.rb  # Node-based tree implementation for categories
│   └── serializers/
│       ├── source_data/ # For reading/writing source data files
│       ├── docs/        # For generating documentation
│       └── dist/        # File-type-centric, one file per distribution type
├── bin/
│   ├── generate_dist    # Main entry point for generating dist/
│   └── generate_docs    # Main entry point for generating docs/
├── db/
│   ├── schema.rb        # Defines tables for models
│   └── seed.rb          # Seeds the database from data/
├── data/
│   ├── integrations/    # Integrations and mappings between taxonomies
│   ├── localizations/   # Localizations for categories, attributes, and values
│   ├── categories/      # Source-of-truth for categories
│   ├── attributes.yml   # Source-of-truth for attributes
│   └── values.yml       # Source-of-truth for values
└── test/

🧑‍💻 Contributing

We welcome contributions! However, PRs will be delayed until the Open Taxonomy approaches a STABLE release.

📅 Releases

You can find the current published version in VERSION. The changelog is available in CHANGELOG.md.

We use SemVer for versioning while in the preview stage. Upon stabilization, we'll transition to CalVer in sync with Shopify's API release schedule.

Formal releases are published as GitHub releases and available on the interactive docs site.

📜 License

Open Taxonomy is released under the MIT License. Explore, play, and build something awesome!

TODO List

• Add support attribute descriptions

• Add support for category descriptions

• Add basic descriptions for verticals

• Add Computer Components Category

  • Add CPU Category
  • Create CPU Attributes
  • Add GPU Category
  • Create GPU Attributes

• Standardize and Update Attributes for Computer Systems

  • Update Attributes for:
    • Barebone Computers
    • Computer Servers
    • Desktop Computers
    • Handheld Devices
    • Interactive Kiosks
    • Laptops
    • Smart Glasses
    • Tablet Computers
    • Thin & Zero Clients
    • Touch Table Computers

• Add support for dynamic taxonomy construction based on a list of products from an existing taxonomy

• Add MVP HP and Lenovo Manufacture Name Taxonomies

• Add support for disjointed Category Links (for linking to Manufacture naming taxonomies)