Expose metadata about a homeserver, useful for server discovery and general experience

[kaiyou]

Context

I was thinking about the way the Mastodon community handled the general server discovery issue, and if we could profit from that. Generally speaking, the problem was solved late on Mastodon and two separate mechanism are in place:

• metadata exposed by a server (user count, owner contact, description, etc.)
• metadata declared on instances.social.

Unfortunately, the declarative bit is what causes most of the bias and I read that most Mastodon users where either choosing a server recommended by a friend/community/company/whatever, which is good, or one of the flagship instances, which is less good for distribution.

Goals

I think it would bring some consistency to the Matrix federation if we could have the following set of resulting properties:

• users can chose a homeserver based on various types of criteria (ideological, geographical, etc.)
• users do not have to worry too much about reliability or security, unless they explicitely ask about it
• users have a generic way of contacting their homeserver admins or technical support (see We should mandate a postmaster@ style admin account for HSes so admins are contactable when needed (SYN-224) #1257)
• homeserver owners can make announcements to their community of users
• homeserver owners can discover similar servers and contact their owners

Technical implementation

I think all of the goals can be met with a single and pretty simple feature: a client/server API endpoint that displays the server metadata. These metadata could be setup in configuration first, and simply displayed as a JSON dictionary. All fields would be optional of course.

Some of the metadata would be normalized, including:

• the main language code,
• the country codes for where data is hosted,
• the type of structure managing the server (foundation, company, etc.)
• whether or not registration is public,
• whether or not registration requires interaction with the technical support,
• whether or not this homeserver should be displayed in public lists (see later, and that could also apply to Voyager or similar projects)
• whether or not this homeserver provides commercial support,
• a level of confidence the admins have in their server,

Some fields would have a more flexible form:

• the list of homeservers managed by the same entity,
• the list of optional features provided, with codenames maybe (bridges, turn, web client, etc.) and probably a way to display a related resource (address of the web client for instance)
• the list of contacts, per contact type (administrative, technical, legal, commercial), could be displayed as a dictionary where keys are normalized, and values could be mxids, room aliases, email addresses or Web addresses; this would include the project website (commercial url) as well as an emergency email address for legal requests,
• the list of entrypoints for discovering the community (basically structured list of rooms and users that would generate the homepage)
• the markdown description of the instance (limited length?)
• the markdown text including an ethical statement/set of rules for the homeserver (limited length?)
• a list of free tags for later searches (political, ideological, community names, etc.)

Regarding announcements, I was simply thinking that among contact points, there could be a "communication" type, and that clients could suggest the user joins that room for announcements. Or maybe the client could silently join that room and display messages in a special way that makes them look a bit more like announcements.

Resulting experience

Matrix clients could first display relevant metadata to the users of a homeserver, for instance contacts could be included on the homepage, as well of the entrypoints that currently make the list of general purpose rooms. If the homeserver becomes unavailable, the client could display technical contact information, when reporting an illegal message, the client could display legal contacts, etc.

On top of that, a third party, or many third party fwiw, could build an instance list without the homeserver owners having to explicitely register, thanks to the flag that tells whether or not the server should be publicly listed. These lists could be consolidated with technical data, including the current server version, average response time from various places in the world, average reliability, and basically everything that Hello Matrix has been doing for some time now.

Finally, clients could use those lists and consolidated meta to perform searches. Then, upon registration, the user could be greated with a couple fields to fill in, then a list of candidate servers.

[ara4n]

+1 all the way. Ironically this has a lot of overlap with our first ever Synapse bug (back when we were using JIRA before moving to GH issues): matrix-org/synapse#1199). It's more than frustrating that onboarding/polishing features like this get forever bumped by bugs or major new core functionality.

[kaiyou]

Here would be a list of fields.

owners (type: list of dictionaries)

Each dictionary is a co-owner of that server. Each dictionary contains the following fields:

• type, one of person, nonprofit, company, state, other
• name, name of the owner as a string

related (type: list of dictionaries)

Each dictionary is a related federated servces. Each dictionary contains the following fields:

• federation, the federation this service is in, be it matrix, mail, mastodon, jabber, etc.
• relation, how the servers are related, among owned, friend, helped, helping
• server, the server (usually domain name)
• name, the server name

lang (type: list of strings)

Each string in the list is one of the main languages ISO639-1 codes on this server. Can be declared by the homeserver owner or could be automatically deduced from clients some day.

location (type: list of strings)

Each string in the list is an ISO3166-1 country code for a country this homeserver is technically or administratively related to.

description (type: markdown string)

Description of the homeserver purpose.

rules (type: markdown string)

Set of rules for this homeserver.

tags (type: list of strings)

Each string is a short tag (lowercase alpha plus hyphen, max 50chars) describing a feature, an ideology, any concept that this server is related to.

public (type: boolean)

Whether this server should be displayed in public lists.

registration (type: string)

One of open (open to any registration), support (registration requires that one first contacts the support), external (registration is bound to another resource), closed (registration is closed, one should not register, independently of the registration feature being available or not).

reliability (type: dictionary)

The dictionary keys are picked among:

• availability for general day to day availability
• durability for the confidence in long term management
• security from a technical security point of view
• privacy for more general privacy concerns, including legal ones

Dictionary values are scores as integers bound between 0 (no confidence) and 5 (full confidence).

size (type: dictionary)

Provides information about the server size, each key being picked among:

• rooms (number of rooms, i.e. room aliases for Matrix)
• users (number of users)
• links (number of links to other server on the federation)

features (type: list of dictionaries)

Each feature in the list is a dictionary, with the following fields:

• type: among bridge, turn, content, voip, etc. (list to be established)
• name: name of the feature, "Gitter bridge", or "Jitsi VoIP" for instance
• resource: specific resource for this feature, external URI, room alias, etc.

contacts (type: list of dictionaries)

Each contact in the list is a dictionary, with the following fields:

• type, among technical, administrative, legal, commercial
• name: name of the contact, e.g. name of a person, of an entity, or simple description
• resource: specific resource for this contact, email address, external URI, room alias, mxid, phone number, etc.

entrypoints (type: list of dictionaries)

Each dictionary in the list is a category on the homepage. It has the following fields:

• name: name of the category
• items: list of room aliases, mxids, items in that category
• children: subcategories

[kaiyou]

I am also thinking about making this generic enough to not be bound to Matrix specifically, but being usable by other federations, including Mastodon.

I just pushed most of the above contents to https://github.com/kaiyou/wellfed, hoping to implement this on Mastodon, which I know a bit better than Synapse for now.

[turt2live]

This sounds like an amazing feature. For the size field, having the users split out into bridged vs not bridged (vs other things like bots?) would be useful.

[kaiyou]

I do not believe any network, be it Matrix, Mastodon or even Jabber will usefully distinguish bridged users from bots. Those are basically service users, and largely consist of bridged users in Matrix, plus a couple bots. I agree about distinguishing users from services, I will add that to the spec.

Also, I will try and write a first version this week of a list website for manifests being crawled from multiple servers.