Skip to content
This repository has been archived by the owner on Jan 2, 2024. It is now read-only.

Network Table Communication

Jump to bottom
Alex Macdonald edited this page Oct 20, 2020 · 2 revisions

This document explains how to communicate with the Network Table.

  • Connecting to the Network table
  • Sending requests to the network table
  • What the network table will reply with

Some details on how the Network Table and the client code is implemented is included near the end of the document.

Network Table Functionality

  • Holds latest information from sensors, AIS antenna, etc
  • Holds motor output data, which is updated by boat control code (In the future, it may also be updated via the website in case of emergency)
  • Simple key value store
  • Uses pub/sub and get/set

Connecting

Messages are sent using ZeroMQ sockets. Communication between the Network Table server and client is done using ZMQ_PAIR sockets. Every time a client connects to the Network Table, a new pair of sockets is created. When a client disconnects from the Network Table, the socket is destroyed. However, to create a new pair of sockets, the client first must send a request to connect to Network Table. The body of the message is the string "connect". If you are sending strings to the network table and not using C/C++ make sure your strings are null terminated. The Network Table will reply with a string which tells the client where to connect their ZMQ_PAIR socket to. Here are example steps for the client in order (C++):

Create a ZMQ_REQ socket and connect it to ipc:///tmp/sailbot/NetworkTable.

zmq::context_t context(1);
zmq::socket_t init_socket(context, ZMQ_REQ);
init_socket.connect("ipc:///tmp/sailbot/NetworkTable");

Send a message requesting to connect.

std::string request_body = "connect";
zmq::message_t request(request_body.size()+1);
memcpy(request.data(), request.c_str(), request_body.size()+1);
init_socket.send(request);

The reply body contains the location to connect to using a ZMQ_PAIR socket.

// receive reply
zmq::message_t reply;
init_socket.receive(&reply);
std::string reply_body(static_cast<char*>(reply.data());
 
// connect with ZMQ_PAIR socket.
zmq::socket_t socket(zontext, ZMQ_PAIR);
socket.connect(reply_body);

You can now communicate with Network Table over the ZMQ_PAIR socket. From now on, data is first serialised using the Google Protobuf library.

Disconnecting

Send a message to the ZMQ_PAIR socket you are connected to. The body of the message is the string "disconnect". After sending this message, the Network Table will no longer check this socket for messages.

std::string request_body = "disconnect";
zmq::request_t request(request_body.size()+1);
memcpy(request.data(), request_body.c_str(), request_body.size()+1);
socket.send(request);

NetworkTable Messages

Messages are serialised using Google Protobuf. Messages from client to server are sent as Request.proto. Messages from server to client are sent as Reply.proto. The following section is an explanation of each of the proto files.

Basic Datatypes

Value.proto:

Value {
    Type type
    double double_data
    bool bool_data
    int32 int_data
    string string_data
    bytes bytes_data

    enum Type {
         DOUBLE
         BOOL
         INT
         STRING
         BYTES
    }
}

Represents the basic datatype which is stored in the network table. Note that the actual .proto file might contain some UBC Sailbot specific info, like AIS boat data, GPS waypoints etc. Check src/proto/Value.proto to see the actual latest definition.

Node.proto:

Node {
   Value value
   map<string, Node> children
}

This recursive datatype is used to create a tree structure of values. Typically only leaf nodes will contain values, although there is nothing that stops you from setting non leaf nodes to have a value.

Requests

Requests sent to the server use Request.proto.
Request.proto:

Request {
    Type type
    string id
 
    SetValuesRequest setvalues_request
    GetNodesRequest getnodes_request
    SubscribeRequest subscribe_request
    UnsubscribeRequest unsubscribe_request
 
    enum Type {
        SETVALUES
        GETNODES
        SUBSCRIBE
        UNSUBSCRIBE
    }
}

Depending on what the type is, one of the optional fields should be set. If the corresponding field is not set, the request is ignored. Ex. if the type is SETVALUES, but the setvalues_request field in unset, the request is ignored. id is an optional field. When the server replies to a request, it will make sure the reply has the same ID. The client can use this info to discard stale replies (Ie when the client times out while waiting for a reply, then gets the reply later). Here is an explanation of each type of request:

SetValuesRequest.proto:

SetValuesRequest {
    map<string, Value> values
}

The map is a map from uris to values to be inserted into the table. This request will not receive a reply from the server. If the uri does not already exist, it will be created. If the uri already exists, it will be overwritten. If the uri already exists and is not a leaf, the value field of that node will be set, and the node will be left with its children intact.

GetNodesRequest.proto:

GetNodesRequest {
    repeated string uris
}

The server will send back a Reply.proto containing a GetNodesReply. If any uris are requested which do not exist in the server, the response will contain a Value of type NONE. Note that unlike SetValuesRequest, this reply contains Nodes, not Values. Nodes are a recursive data type, possibly forming a tree, while a Value is a simple non-recursive datatype. For example, if you sent a GetNodesRequest containing just the uri "/", you would get back the root node.

SubscribeRequest.proto:

SubscribeRequest {
    string uri
}

The server will send a Reply.proto containing a SubscribeReply.proto to the clients socket anytime the value located at "uri", or in a child of "uri" changes. For example, if you subscribe to "bat/BAT0", and the value "bat/BAT0/charge" changes, you will receive an update containing the new value of node "bat/BAT0" (note that you get back the original node you subscribed to). Subscribing to the same uri multiple times has the same effect as subscribing once to that uri.

UnsubscribeRequest.proto:

UnsubscribeRequest {
    string uri
}

This will cause the server to stop sending updates to the socket. Unsubscribing from a uri that hasn't been subscribed to has no effect.

Replies

Reply.proto:

Reply {
    Type type
    string id

    GetNodesReply getnodes_reply
    SubscribeReply subscribe_reply
    ErrorReply error_reply

    enum Type {
        GETNODES
        SUBSCRIBE
        ERROR
        ACK
    }
}

Depending on what you requested, the type field will be set appropriately, and one of the 3 member fields will contain a more specific type of reply. There is a special reply called ACK (acknowledge). An ACK is sent to a client whenever their request is processed. Some requests, like GetNodesRequest, do not generate an ACK, since the server will send a reply anyways. This can be used to make the client to wait until they are sure their request has gone through. The ACK is typically used in conjunction with the id field (an ACK will also have the id field set to the id of the request which triggered the ACK)

SubscribeReply.proto:

SubscribeReply {
    string uri
    Node node
    map<string, Value> diffs
    string responsible_socket
}

The uri corresponds to the uri that was originally subscribed to. Node is the new node located at the uri. For example if you subscribed to the root node "/", and something in a leaf changed, you will receive the entire tree located at "/". diffs is a bunch of string/Value pairs, representing the specific changes which occured. This makes it easier for the client to tell exactly what changed, especially if they're subscribed to the root. diffs is extremely easy to find on the server side, so I decided to include it in the SubscribeReply. On the other hand, it is annoying for the client to try to figure out what exactly changed on their own, especially if the SetValues request which triggered the SubscribeReply doesn't actually change the values, and instead just writes the same value (which will still trigger a SubsribeReply). responsible_socket contains the filepath to the socket with triggered the SubscribeReply (by sending a SetValues request). The client can use this to check if a SubscribeReply came from their own SetValuesRequest (and then possible ignore it, or treat it differently). Without this, problems can occur when synchronizing two network tables, where changes can bounce back and forth between two network tables forever.

GetNodesReply.proto:

GetNodesReply {
    map<string, Node> nodes
}

Contains a mapping from uris (string) to nodes.

ErrorReply.proto:

ErrorReply {
    Type type
    string message_data

    enum Type {
        NODE_NOT_FOUND
    }
}

Used when something goes wrong/a request cannot be satisfied. For example, GetNodes was sent, but nothing exists at the requested uri. Other

For uris, the first slash is always ignored. So uri "/a/b/c" and uri "a/b/c" are the same There are other protofiles in the network table repo which contain sailbot specific data, such as sensor data and uccm data. These can have different purposes (for example improving compression ratio as opposed to using generic Node.proto). These do not affect the operation of the network table in anyway, and the network table itself does not use them, so they are not included in discussion here.

Implementation details

The network table server has a fairly simple implementation. In a while true loop, it listens to all of its sockets (all ZMQ_PAIR "client" sockets, as well as a single "welcome" socket used for new connections) and handles requests appropriately. It is single threaded. When it gets a request from a client, it will go through a switch statement based on the request type, and handle the request appropriately.

The client side code is more complicated. It uses a background "socket listener" thread to listen to the socket. This is necessary because a subscribe reply could occur at any time.
Communication between the main thread and "socket listener" thread is done via a ZeroMQ IPC socket. The "socket listener" thread will listen to this socket and the ZMQ_PAIR socket connected to the server. If it receives a message from the server, it will check if this was a SubscribeReply. If it is, it calls the associated callback function for the specified URI. Otherwise, it will just send the reply to the main thread. If it gets a message from the main thread, then this is a request for the network table. The "socket listener" thread will generate a unique ID, and send this request to the server.

Sometimes the client might send a request to the server while it is very busy. The server might end up replying to an old request instead. To prevent the client from using the wrong reply, the ID field is used to make sure the reply is for the request that was just sent.

In order to "slow down" the clients (especially during the stress test), clients will wait until they get an ACK when sending certain types of requests, like SubscribeRequest and SetValuesRequest. Of course, for GetValuesRequest, the client already has to wait for the actual reply, so explicit ACK is not needed.

Clone this wiki locally