design details?

[anarcat]

Hi!

I wonder how innernet actually works. What I understand is that there's a server and clients, but it's not clear to me exactly how things are interconnected.

It would be nice to have some diagrams showing the various parts and how they interact. For example, there's an SQLite database? where? can it be made highly available? what happens when the server dies?

I have this mental model right now, does it look about right?

+----------------------------+
| server                     |
|                            |
|  ------       ----------   |
| (sqlite)<--->(web server)  |
|  ------       ----------   |
|              /      |      |
+-------------/-------+------+
             /        |
            /         |
 +---------/+  +------+---+
 |    client|  |    client|
 |    /     |  |     |    |
 |   /      |  |     |    |
 |  /       |  |     v    |
 | wg <-----+--+--> wg    |
 |          |  |          |
 | node     |  | node     |
 +----------+  +----------+

Here I abstract away the idea that the server is also in the wireguard mesh, that there are possibly more than two nodes, and ACLs. Hopefully I don't miss much else?

[mcginty]

Hey @anarcat, your model is correct. Agreed, some diagrams would probably be helpful...

The SQLite database on the server is used mostly for keeping track of (and assigning) IP addresses, user roles, groups (aka CIDRs), and associations (which CIDRs can see which).

The server's main purpose is, generally speaking, to pass on information about peers to each other to facilitate them directly connecting to each other. As your diagram shows, the server does not act as a relay of traffic between peers, it's a discovery/metadata provider.

High availability of the server and its SQLite database is an interesting one - it is possible to have a "hot" primary and "cold" backup server running that can be swapped out via DNS, since innernet clients will resolve a DNS entry for servers when connecting. That said, innernet itself does not offer any special functionality for high availability of the server, so that's up to the administrator of it.

That said, depending on your use case high availability of the server might not be necessary for high availability of the network as a whole.

In more detail, here are some scenarios:

Server is down

• ✔️ Peers that already know each other can continue to communicate and establish connections with each other.
• ❌ No changes to groups, associations, etc., can be made.
• ❌ New peers cannot be invited (or redeem their invites).
• ❌ The server cannot help peers traverse NATs, so roaming might result in more connection issues.

Server is compromised

• ✔️ Existing peers that know about each other cannot be impersonated (their public key is cached and pinned).
• ❌ New peers can be generated and added to the network.
• ❌ Peers can be marked deleted, made admins, etc.

All that to say, if your network is mostly static (peers only being added or removed every once in a while), it's not so important that the server have 100% uptime. If there's a lot of movement in the network (peers changing IPs often or peers being added/removed, etc.) then the importance of server uptime increases.

[anarcat]

All that to say, if your network is mostly static (peers only being added or removed every once in a while), it's not so important that the server have 100% uptime. If there's a lot of movement in the network (peers changing IPs often or peers being added/removed, etc.) then the importance of server uptime increases.

Super interesting, thanks! So basically, the server needs to be up only when nodes are changing or are behind NAT, right? In that sense, assuming no NAT, one could open up the server only as part of the procedure for setting up new hosts, for example, and keep it otherwise quiet or even shutdown the rest of the time?

Also, this works over NAT, right? ie. peers can be behind a NAT firewall without problems, the server helps with hole punching but traffic goes direct? but the server itself needs a publicly available HTTPS port, right?

All this is over HTTPS except WG I guess? Do we need a HTTPS proxy or is some (let's encrypt?) HTTPS thing builtin?

Thanks again!