MSU message keys

MSU message keys

MSU message keys are used to identify a set of messages that correspond to the communication instance.

For example, in the webserver MSUs, the key can be used to identify messages which all originate from the same 4-tuple, thus providing a mechanism to group communications with the same sender.

Message key structure

The structure of an MSU message key is:

/**
 * The composite key is used to store a key of
 * arbitrary length (up to 192 bytes).
 * This is used for storing/retrieving state.
 */
struct composite_key {
    uint64_t k1;
    uint64_t k2;
    uint64_t k3;
};

/**
 * Used to uniquely identify the source of a message,
 * used in state storage as well as routing
 */
struct msu_msg_key {
    /** The full, arbitrary-length, unique key (used in state) */
    struct composite_key key;
    /** The length of the composite key */
    size_t key_len;
    /** A shorter, often hashed id for the key of fixed length (used in routing) */
    int32_t id;
    /** Used to mark a route ID when storing state, enabling state migration
     * within a specific group */
    int group_id;
};

Message key use

There are two primary uses for message keys:

State storage

The MSU state module passes the composite_key into the UT_hash hashmap to store and retrieve state. In addition, it utilizes the id and group_id to enable transfer of state between MSUs. When state transfer is requested, all states with a matching group_id and an id in the given range are transferred to the target MSU (work in progress).

Routing

When using the default routing function to determine the target of an MSU call, the routing module utilizes the id field of the message key to determine which MSU receives the message. This helps to ensure that a single MSU receives all messages that correspond to a given communication session.

Setting message keys

There are two provided functions to initialize a message key:

/**
 * Sets the key's ID and composite-ID to be equal to the provided id.
 */
int set_msg_key(int32_t id, struct msu_msg_key *key);

/**
 * Sets the key's composite-ID to the provided value,
 * and sets the key's ID to a hash of the value.
 */
int seed_msg_key(void *seed, size_t seed_size, struct msu_msg_key *key);

set_msg_key should be used in situations where the ID is easily mapped to an integer, the provided integers are (approximately) uniformly distributed, and it is important that the ID stays recognizable. In practice it is used for profiling, when the key provided by the client has to be the same as the message key so the times can be matched up 1:1.

seed_msg_key should be used with arbitrary-length, arbitrary-type keys (such as the sockaddr structure, for capturing the 4-tuple). This function automatically hashes the provided seed value, and sets the 32-bit ID to be equal to the output of this hash so it does not have to be re-computed on each route.

An example of the use of seed_msg_key:

// Generate the seed
struct sockaddr_in addr;
socklen_t addrlen = sizeof(addr);
getpeername(fd, (struct sockaddr*)&seed.sockaddr, &addrlen);

// Set the key
struct msu_msg_key key;
seed_msg_key(&addr, sizeof(addr), &key);
    
// Use the key in an MSU call...