Skip to content

Latest commit

 

History

History
126 lines (93 loc) · 6.08 KB

mep-3355.md

File metadata and controls

126 lines (93 loc) · 6.08 KB
Raw
MEP: 3355
Title: MEP-3355 LLM Friendly Transaction Memo Format  
Status: Draft
Type: Standards
Created: 2023-06-20

MEP-3355: LLM Friendly Transaction Memo Format

Abstract

MEP-3355 proposes a standard format for transaction memos on Moonchain that is optimized for processing by large language models (LLMs). The standard leverages the Brotli compression algorithm to efficiently compress memo data before sending it to Moonchain. This allows for more expressive and detailed memos while minimizing transaction costs. The standard memo format enables powerful LLM-based applications built on Moonchain transaction data.

Motivation

As LLMs become increasingly powerful tools for processing and analyzing on-chain data, there is an opportunity to structure transaction memo data in a way that facilitates LLM understanding. By defining a standard LLM-friendly memo format and leveraging compression, Moonchain aims to unlock a new generation of LLM-powered blockchain analytics, data extraction, and smart contract capabilities.

The key benefits of an LLM-friendly memo format include:

  • Enabling efficient and streamlined ingestion of Moonchain transaction data by LLMs
  • Facilitating better LLM understanding of transaction context and intent
  • Allowing for more expressive memos with minimized transaction cost through compression
  • Providing a foundation for building advanced LLM-based Moonchain applications

Specification

MEP-3355 proposes the following standard format for LLM friendly transaction memos on Moonchain:

{
  "format": "MEP-3355",
  "version": "1.0.0",
  "metadata": {
    "data_source": "Compressed data source information",
    "data_collection_method": "Compressed data collection method",
    "preprocessing": "Compressed preprocessing details"
  },
  "data": [
    {
      "type": "sensor",
      "content": "Compressed sensor data in jsonl goes here",
      "compression": "brotli"
    },
    {
      "type": "lpwan",
      "content": ["compressed", "tag", "list"]
    }
  ]
}

The memo data is represented as a JSON object with the following fields:

  • format (required): Specifies the MEP-3355 format to allow for future extension
  • version (required): The version of the MEP-3355 spec used
  • metadata (optional): An object containing additional information about the data
    • data_source (optional): Compressed information about the source of the data
    • data_collection_method (optional): Compressed details on how the data was collected
    • preprocessing (optional): Compressed information about any preprocessing performed on the data
  • data (required): An array of data objects included in the memo
    • type (required): The type of data (initially "sensor" or "lpwan")
    • content (required): The compressed data content
    • compression (optional): The compression algorithm used (defaults to "brotli")

Before being added to the memo, text content should be compressed using the Brotli algorithm. This allows for longer and more expressive memos while keeping transaction size and cost manageable.

Tag data can be used to concisely annotate key concepts, entities and topics to aid LLM understanding. The tag list should also be compressed.

Here's an example of how the memo data could be structured in JSONL format:

{"timestamp": "2024-06-19T22:00:00Z", "heartbeat": 72, "sleep_status": "awake"}
{"timestamp": "2024-06-19T22:15:00Z", "heartbeat": 70, "sleep_status": "asleep"}
{"timestamp": "2024-06-19T22:30:00Z", "heartbeat": 68, "sleep_status": "asleep"}
{"timestamp": "2024-06-19T22:45:00Z", "heartbeat": 67, "sleep_status": "asleep"}
{"timestamp": "2024-06-20T07:00:00Z", "heartbeat": 75, "sleep_status": "awake"}

Each line in the JSONL data represents a separate data point, with the relevant information structured as key-value pairs. This format allows for easy parsing and processing by LLMs.

Rationale

The proposed memo format is designed to optimize for LLM ingestion and understanding while working within the constraints of blockchain transactions. Key design choices include:

  • Using JSON for its widespread support and extensibility
  • Including metadata to provide additional context about the data source, collection methods, and preprocessing
  • Separately typing text and tag data to allow focused LLM processing
  • Compressing data to pack more information into memos at lower transaction cost
  • Specifying the compression algorithm for consistent decoding

Brotli compression was selected because it achieves high compression ratios on text data while remaining relatively fast. This makes it well-suited for compressing memos in the context of blockchain transactions.

Backwards Compatibility

Adopting the MEP-3355 format for Moonchain transaction memos would require a coordinated upgrade of Moonchain nodes, wallets and other key infrastructure to support the new format. Transactions using the legacy memo format could continue to be supported for a transition period as determined by the community.

Test Cases

Test cases should be developed to verify the correctness of MEP-3355 memo encoding and decoding across different Moonchain implementations. Key aspects to test include:

  • JSON validity and schema conformance of generated memos
  • Correct Brotli compression and decompression of memo data
  • Graceful handling of invalid or malformed MEP-3355 memos
  • Performance impact of memo compression and decompression

Implementation

Implementing MEP-3355 would involve:

  1. Adding support for MEP-3355 memo formatting and compression in Moonchain nodes
  2. Updating Moonchain wallets to provide user-friendly MEP-3355 memo creation
  3. Building Moonchain blockchain explorers and analytics tools that parse MEP-3355 memos
  4. Developing and releasing LLM-based applications that leverage MEP-3355 formatted data

A phased rollout is recommended, starting with infrastructure upgrades and ending with LLM application development.

References