Auto Update Internals

This wiki page documents how the auto-update system works.

Update tracks

We will originally support 3 update tracks that users can choose from:

• Releases: updates every few months synced with progress reports (internally known as beta).
• Dev: updates on every single change pushed to dolphin-emu.org.

Builds downloaded from dolphin-emu.org will default to the Releases track. Dev is only for the most hardcore users who are willing to update every time they use our software, so it will always have to be picked manually.

Update check protocol

To check for updates, Dolphin can hit the following API endpoint on dolphin-emu.org:

GET /update/check/v1/<track>/<hash>/<system>

• <track> is the update track the user wants to follow (see previous section).
• <hash> is the SHA1 hash of the current Dolphin version.
• <system> is a short identifier of the operating system (or build flavor) to get artifacts for. Example: win, macos.

This returns a JSON object with the following parameters:

{"status": "up-to-date"}

# OR

{
  "status": "outdated",
  "content-store": "https://update.dolphin-emu.org/content/",  # Content store base URL
  "new": {
    "hash": "aabbccdd...",  # Version SHA1
    "name": "5.0-xxxx",     # Readable version name
    "manifest": "https://update.dolphin-emu.org/manifest/...",  # Manifest URL
  },
  "old": {
    "hash": "00112233...",  # Version SHA1
    "name": "5.0-yyyy",     # Readable version name
    "manifest": "https://update.dolphin-emu.org/manifest/...",  # Manifest URL
  },
  "changelog": [
    # One entry per version that is between old (non incl) and new (incl).
    {
      "shortrev": "5.0-xxxx",
      "short_descr": "one line description of the change",
      "author": "delroth",
      "author_url": "https://github.com/delroth",

      # For changes that were issued from a pull request:
      "pr_id": 4242,
      "pr_url": "https://github.com/dolphin-emu/dolphin/pull/4242",

      # For changes on a manually curated update track:
      "changelog_html": "Arbitrary <b>HTML</b> that describes the version.",
    },
  ],
}

Update manifest

The update manifest is a gzip compressed file which describes for each Dolphin version what files it contains. The format is as simple as it gets:

• One file per line. Filename then '\t' then hex-encoded hash of the contents of the file.
• The hash algorithm we use is SHA256 truncated to 128 bits (SHA256-128).
• An empty line ends the list of files.
• The list of files is followed by base64-encoded ed25519 signatures of the list of files. We support multiple signatures to handle key rotation.

Basically:

Dir/File1	00112233445566778899aabbccddeeff
Dir/File2	00112233445566778899aabbccddeeff

BASE64-SIGNATURE-HERE
OPTIONAL-MORE-BASE64-SIGNATURES-HERE

The current public key of our build infrastructure is the following: 2ab3d1dc6ef507f6a06c7c54df54f44280a6288b6d7014b54c3495204dd4d35d.

Manifests are stored in the manifest store, publicly mapped to https://update.dolphin-emu.org/manifest/. The manifest of a version with SHA1 hash "aabbccddeeff...00112233" is located at /manifest/aa/bb/ccddeeff...00112233.manifest. Note however that you should always be using the manifest URLs from the update check response.

Content store

All files that are present in a Dolphin release can be fetched using their SHA256-128 hash from our content store, mapped at https://update.dolphin-emu.org/content/. Note that the URL of the content store is returned in the update check response -- use that one instead of hardcoding.

The gzip'd contents of a file with hash 00112233445566778899aabbccddeeff can be found at /content/00/11/2233445566778899aabbccddeeff. You can verify that the contents do match the hash, for example:

$ curl -s https://update.dolphin-emu.org/content/86/80/ed3a0deaed20812389f1a97bfd70 | gzip -cd | sha256sum | head -c -32
8680ed3a0deaed20812389f1a97bfd70

Running the updater

Note: this section focuses on the Windows auto-update flow. There are small differences in how the macOS version operates, but the general operating principle remains the same.

The Dolphin updater is Updater.exe, a separate binary from Dolphin.exe. It is designed to be able to run without depending on anything within the Dolphin install directory.

When triggering an update, Dolphin.exe copies Updater.exe to a temporary location, and runs it with the following information provided through command line arguments:

• URL of the current version and target version manifest (from the update check response).
• URL of the content store (from the update check response).
• Directory of the Dolphin installation to update.

The updater is in charge of fetching the two manifests, diff-ing their contents, and generating:

• A list of hashes to download from the content store.
• A list of files to update within the install directory.
• A list of files to delete from the install directory.

Since these lists are fully generated from the manifest, before replacing or deleting any file, the updater can first check whether these files have been manually modified by the user. If that is the case, the modified contents are backed up to a .bak file before replacing / deleting. Only files present in the manifest are touched -- anything else that a user might have added manually to their directory is not touched (portable.txt, User/, ...).

Downloaded contents are cached in a TempUpdate directory within the Dolphin install directory while the update process is going on. The updater is responsible for cleaning what it downloaded after execution, and remove the TempUpdate directory if it is empty.

Update protocol versions changelog

The update check protocol is versioned to allow for forward and backward compatibility. Older versions of the protocol need to be maintained for older Dolphin versions to continue supporting auto-updates. It is sufficient to support auto-updating from vN only to the first version of Dolphin that supports vN+1, i.e. requiring the user to update in multiple steps is acceptable. But when possible, keeping compatibility all the way is preferred.

This is the list of existing update protocol versions and what changes they introduced:

• v1 (added 2019-02-06): Added the <system> information to support macOS auto-update.
• v0 (added 2018-03-23): Initial version.