Multiworld on EverDrive

[fenhl]

Now that we have a proof of concept of communicating with the game via the EverDrive's USB protocol (PC side, N64 side), it's time to figure out what the communication protocol is supposed to look like. Here's my current draft:

Design notes

All PC→N64 packets are exactly 16 bytes long to simplify the receiving code on the N64 side. N64→PC packets may be of any size supported by the EverDrive, i.e. anywhere between 16 and 512 bytes; the packet's size depends on its ID (first byte). This is because we may want to have N64 send large amounts of data when a file is selected, to make auto-tracking integration work. Unless noted otherwise, packets in either direction with a total length less than 16 bytes are padded with trailing zero bytes which are reserved for use in future versions of the protocol.

The protocol must seamlessly recover from state loss due to console reset. Fortunately, the console being reset causes the PC to see an existing connection as timed out even if the game is restarted within the timeout. It can then reconnect and start over. To achieve this, N64 sends periodic pings (every 100 frames, which corresponds to every 5 seconds while in-game and not paused), and PC sets a read timeout of 10 seconds (twice the ping interval).

Handshake

1. Communication is initiated by PC sending ASCII cmdt followed by a null suffix. The format of this handshake is chosen to also be a valid command if N64 is in the EverDrive's main menu, which allows PC to determine the console's state based on the reply. The suffix must remain null in future versions for compatibility with the main menu.

2. N64 sends ASCII OoTR followed by:

   • The protocol version byte (currently 0x01). If PC does not support this protocol version, it must close the connection now, any reply indicates support for this version.
   • get_version_bytes, i.e. a 5-byte sequence encoding the randomizer version number: major, minor, patch, branch identifier, supplementary version.
   • The world number.
   • CFG_FILE_SELECT_HASH, i.e. the 5 icons displayed at the top of the file select screen, represented as indices into the HASH_ICONS list in Spoiler.py.

   If PC instead receives ASCII cmdr or cmdk, the N64 is in the EverDrive main menu. PC can either continue to use this connection via the main menu protocol, or disconnect and retry after an amount of time.

3. PC sends ASCII MW followed by:

   • the protocol version byte (must be the same one that was received)
   • MW_SEND_OWN_ITEMS (a Boolean flag indicating whether to enable this feature)
   • MW_PROGRESSIVE_ITEMS_ENABLE (a Boolean flag indicating whether to enable this feature)

   This step is designed to allow reusing the handshake for something other than multiworld, e.g. auto-tracking, debugging, or crowd control. For those alternative use-cases, the PC would respond with a different packet prefix here, and the “normal operation” part of this protocol would be ignored in favor of a different protocol for the given purpose.

4. Normal operation commences with N64 sending a state packet and PC simultaneously sending player names. Once PC has received the state packet, it may also immediately send items to restore the queue. More info in the following two sections.

Normal operation: PC→N64

The following packet IDs are used:

• 0x00 Ping
  • sent every 5 seconds
• 0x01 Player Data
  • the world number (1 byte)
  • the player name (8 bytes in OoT's internal player name format)
  • progressive items state (4 bytes, see mw_progressive_items_state_t in item_upgrades.c, PC may set this to 0 if MW_PROGRESSIVE_ITEMS_ENABLE is off)
• 0x02 Get Item
  • item kind (2 bytes)
  • Only one item may be sent after each time the N64 sends “State: In Game” or “Item Received”, and only items past the internal item count are sent
• 0x63 (ASCII c) indicates that PC has reset (e.g. the multiworld app has been relaunched). N64 should reset its connection state and treat the incoming packet as the first part of a new handshake.

Normal operation: N64→PC

The following packet IDs are used:

• 0x00 Ping
  • sent every 5 seconds
• 0x01 State: File Select
  • the player name: 8 bytes in the internal player name format, 8 spaces (DFDFDFDFDFDFDFDF) if not set
  • Sent after completion of the handshake if not ingame, or if the selected filename changes
• 0x02 State: In Game
  • internal item count (2 bytes)
  • relevant parts of save data (TODO which?)
  • Sent after completion of the handshake if ingame, or if a file is opened
• 0x03 Send Item
  • override key (8 bytes)
  • item kind (2 bytes)
  • target world (1 byte)
• 0x04 Item Received
  • tells PC that another Get Item packet may be sent

TODO

• Feature parity with MH MW for emulators
  • on file load, send the subset of save data required for auto-tracking and maintaining progressive items state
  • send current scene
  • check if there's anything else missing
• Ensure docs of both multiworld interfaces mention that the other needs to be updated at the same time

[Polprzewodnikowy]

What about flashcarts other than EverDrive64? N64 homebrew community has developed an universal USB library - UNFLoader - that supports most common flashcarts available on the market. It's been successfully integrated in several projects, including:

• libdragon
• n64sdkmod
• HackerSM64
• 64dddump

Locking this feature to EverDrive64 v3/X7 doesn't really make sense since other flashcarts (64drive/SummerCart64) include USB port as a standard feature, while most EverDrive64 models out there doesn't even have this port available.

[fenhl]

I'm not familiar enough with projects like libultra and libdragon to be able to integrate this into OoTR, but I think if someone were to contribute support for these additional flashcarts, it would most likely be accepted.

[fenhl]

The current work-in-progress implementation of this protocol can be found on my fork of the randomizer.