Implement custom order filtering

[albrow]

This PR simply rebases #563 on the new development branch. I'm keeping the old PR around just in case anything was forgotten/lost during the messy rebase.

Fixes #336 and #337.

Overview

This PR implements custom order filtering and custom topics. A custom filter may be passed into Mesh as a JSON Schema via the new CUSTOM_ORDER_FILTER environment variable. Messages that contain orders that don't match this schema will be dropped. As a limitation, filtering is only possible by looking at the static fields of an order. So for example, it is not possible to filter orders by doing an on-chain check or sending an HTTP request to a third-party API. We don't expect that this limitation is going to be a problem in practice and it comes with the huge benefit of enabling cross-topic forwarding in the future (more on that later).

This PR also changes the way that pubsub topics work in Mesh. Previously, there was only one topic and it was hard-coded based on the chain ID. Now, there is a one-to-one correspondence between custom filters and topics. You can convert from custom order schema to a topic and vice versa. This is a breaking change and the old topic names are not compatible with this PR. (However, with this change we can potentially implement cross-version forwarding for forward compatibility from now on).

New order and message schemas.

All orders must match the following JSON Schema:

{
	"id": "/rootOrder",
	"allOf": [{
		"$ref": "/customOrder"
	}, {
		"$ref": "/signedOrder"
	}]
}

• /signedOrder is the same JSON Schema as before and will match only valid 0x orders.
• /customOrder is the custom schema passed in through the CUSTOM_ORDER_FILTER environment variable.

Organizing the JSON Schema for orders like this means that CUSTOM_ORDER_FILTER can be relatively small. It doesn't need to contain all the required fields for a signed 0x order. It just needs to contain any additional requirements on top of the default ones.

The new schema for order messages sent through pubsub looks like this:

{
	"id": "/rootOrderMessage",
	"properties": {
		"messageType": {
			"type": "string",
			"pattern": "order"
		},
		"order": {
			"$ref": "/rootOrder"
		},
		"topics": {
			"type": "array",
			"minItems": 1,
			"items": {
				"type": "string"
			}
		}
	},
	"required": ["messageType", "order", "topics"]
}

Notably this adds the topics field which is a list of topics that the message was sent on. This is critical for cross-topic forwarding.

Example custom order schemas

The following CUSTOM_ORDER_FILTER doesn't add any additional requirements. All valid signed 0x orders will be accepted. This is the default value if no custom filter is passed in.

{}

The following CUSTOM_ORDER_FILTER matches any valid signed 0x orders with a specific sender address:

{
	"properties": {
		"senderAddress": {
			"pattern": "0x00000000000000000000000000000000ba5eba11",
			"type": "string"
		}
	}
}

This can easily be tweaked to filter orders by asset type, maker/taker address, or fee recipient.

New topic strings

The new topic format looks like this:

`/0x-orders/version/${versionNumber}/chain/${chainID}/schema/${encodedSchema}`

• versionNumber is the version for the topic string. This PR changes the default version from 2 to 3.
• chainID is the chain ID that Mesh is configured to use.
• encodedSchema is the base64-encoded custom order schema. This trick allows us to extract the custom order schema from the topic string.

Future improvements

Cross-topic forwarding

Part of the reason this PR is implemented in a particular way is to enable "cross-topic forwarding" in the future. What this means is that a Mesh node that is subscribed on the default topic (all 0x orders) can receive orders on any topic and forward them to nodes that are only subscribed to that topic without subscribing to or even knowing about the topic ahead of time. This PR does not implement cross-topic forwarding but it does add everything we need to do so in the future without breaking backwards-compatibility.

Here's a rough overview of how it will work:

1. Mesh nodes which are using custom filters will publish every order to two topics: (a) the topic corresponding to their custom filter and (b) the default topic. Because of how the JSON Schemas are implemented with allOf, we know that any order which is valid for a custom topic would also be considered valid on the default topic.
2. Mesh nodes with cross-topic forwarding enabled will check every message that they receive to see which topics they were originally published to. They can then decode the custom schema from the topic string and validate that the order does match the custom schema. If it matches, they can forward the message to the custom topic. Hopefully we can do this while retaining the original pubsub signature, but it's possible we need to re-sign the message before forwarding.

A similar approach can be used to forward messages between topic versions. In other words a message that was originally sent on topic version 3 could be modified and forwarded to topic version 4. This means we could potentially break backwards compatibility at the pubsub layer without completely fragmenting liquidity (although this approach would add additional latency).

[albrow]

This is an auto-generated file and yarn is grabbing the version number from the package.json for the @0x/mesh-browser package locally. I know this looks weird but it's a side-effect of using a relative import in our package.json instead of importing from npm. I say we leave it be.

[albrow]

This PR is ready for review. In the interest of not making this PR even bigger than it already is, the following will be added in separate PRs:

1. In-depth documentation about how to create custom order filters. This will probably exist as a page in the "Advanced Topic" section on https://0x-org.gitbook.io/mesh/.
2. Updating peer scores when receiving an order that doesn't pass your custom filter. This will help you stay connected to peers that are compatible with your custom order filter, which will increase the efficiency of sharing orders. However there are already some important changes and optimizations that will have an even greater impact on order propagation speed and have higher priority (e.g. Improve algorithm for sharing old orders #638 and Is GossipSub working as expected? #551).