Update Log

TFC 1.24.04 Update log

Dependencies

Dependencies were updated to latest versions.
Starting from *buntu 23.10, global python packages must be installed via APT, necessary changes were made. This unfortunately reduces the number of dependencies that can be provided with pinned hashes.
Relay Program now uses cryptography library instead of PyNaCl for deriving ed25519 keys from seeds.

Installation

Tails installation can now rely on almost exclusively on pre-installed packages, which means the configuration now installs much faster.
Fixed Tails control port number that was changed due to a MITM vulnerability.

Security

Remove kernel minimum version check, as all kernels with improper CSPRNG seeding checks have now reached EoL.

Documentation

Removed outdated LRNG description. More details will be added once the new design receives in-depth documentation.

TFC 1.23.04 Update log

Dependencies

Updated
- certifi to 2022.12.7
- cryptography to 40.0.2
- distlib to 0.3.6
- filelock to 3.12.0
- Flask to 2.2.3
- importlib-metadata to 6.5.0
- MarkupSafe to 2.1.2
- pip to 23.1
- platformdirs to 3.2.0
- requests to 2.28.2
- setuptools to 67.7.0
- typing-extensions to 4.5.0
- urllib3 to 1.26.15
- virtualenv to 20.22.0
- Werkzeug to 2.2.3
- zipp to 3.15.0

Security

Replaced SHA512 for pinned file checks with BLAKE2b
Replaced SHA256 for project's gpg signing key fingerprint with BLAKE2b

Code quality

Fixed typos here and there

Documentation

Added a FAQ entry for selecting correct baudrate
Removed requirements badge and FAQ entry about it as the service no longer exists.

TFC 1.22.11 Update log

Dependencies

Updated
- cryptography to 38.0.3
- exceptiongroup to 1.0.3
- mypy to 0.991
- types-requests to 2.28.11.4
- setuptools to 65.5.1

Security

Fixed an issue where the requirements-pre.txt (that contains pip and setuptools) was authenticated only after the requirements were installed. The attack surface was wider than necessary, but PyPI's TLS and internal authentication mechanisms have been present all along, and the risk was low.
The new cryptography package patches an issue where Linux wheels were compiles with OpenSSL 3.0.7 that has two vulnerabilities:
- CVE-2022-3602 is an RCE vulnerability related to X.509 certificate verification. TFC only uses OpenSSL/cryptography for X448 key exchange, thus this vulnerability does not concern TFC.
- CVE-2022-3786 is a DoS vulnerability also related to X.509 certificate validation, and it also does not concern TFC.
The new setuptools package patches a ReDoS vulnerability (Note: No CVE entry exists) that is another DoS vulnerability when "the user is fetching malicious HTML from a package in PyPi or a custom PackageIndex page." Custom PackageIndex page is not used, but it's not clear how packages that use setuptools are affected. The risk rating by Snyk sets the vulnerability as low, thus most likely no action is needed.

Fixes

Minor cleaning of code wherever issues were spotted.

TFC 1.22.10 Update log

Dependencies

Updated
- types-requests to 2.28.11.2
- pytest to 7.2.0 with new sub-dependencies
  - exceptiongroup 1.0.0rc9
  - iniconfig 1.1.1
- more-itertools to 9.0.0
- pytest-cov to 4.0.0
- certifi to 2022.9.24
- coverage to 6.5.0
- importlib-metadata 5.0.0
- mypy to 0.982 with new sub-dependencies
  - tomli 2.0.1
- pytest-xdist to 3.0.2
- setuptools>=65.5.0
- stem to 1.8.1
- typing-extensions to 4.4.0
- zipp to 3.10.0

Fixes

Fixed Tails configuration's onion-grater profile that prevented Relay Program from launching the Onion Service on Tails.

Release-pipeline

Added test-run option for Relay Program that spins a test Tor Onion Service without the requirement of having a Source Computer output the Onion Service private key to it first. This makes it easier to spot issues as the one discussed in the fix above.

TFC 1.22.09 Update log

Dependencies

Updated
- argon2_cffi to 21.3.0
  - With argon2_cffi_bindings 21.2.0 as new sub-dependency
- backports.entry-points-selectable to 1.1.1
- certifi to 2022.9.14
- cffi to 1.15.1
- charset-normalizer to 2.1.1
- click to 8.1.3
- cryptography to 38.0.1
- distlib to 0.3.6
- filelock to 3.8.0
- Flask to 2.2.2
- idna to 3.4
- importlib-metadata to 4.12.0
- itsdangerous to 2.1.2
- Jinja2 to 3.1.2
- MarkupSafe to 2.1.1
- pip to 22.2.2
- platformdirs to 2.5.2
- PyNaCl to 1.5.0
- pyparsing to 3.0.9
- requests to 2.28.1
- setuptools to 65.3.0
- typing-extensions to 4.3.0
- urllib3 to 1.26.12
- virtualenv to 20.16.5
- Werkzeug to 2.2.2
- zipp to 3.8.1

CI

Switched from Travis CI to GitHub Actions for unit tests
Switched from Coveralls to Codecov that supports GitHub Actions.

Installation

Installation is now less crash-prone, as the one-liner and installer use an until-loop to ensure Torsocks makes the first connection. For some reason this may take up to 10 minutes.

TFC 1.21.11 Update log

Dependencies

Updated
- argon2-cffi to 21.1.0
  - cffi to 1.15.0
  - pycparser to 2.21
- cryptography to 35.0.0
- Flask to 2.0.2
  - click to 8.0.3
  - importlib-metadata to 4.8.1
  - zipp to 3.6.0
  - typing-extensions to 3.10.0.2
  - Jinja2 to 3.0.2
  - Werkzeug to 2.0.2
- Requests subdependencies
  - certifi to 2021.10.8
  - charset-normalizer to 2.0.7
  - idna to 3.3
  - urllib3 to 1.26.7
  - setuptools to 58.5.3
- virtualenv to 20.10.0
  - distlib to 0.3.3
  - filelock to 3.3.2
  - importlib-metadata to 4.8.1
  - zipp to 3.6.0
  - platformdirs to 2.4.0
  - typing-extensions to 3.10.0.2

Platform support

Added support for Zorin OS 16.

CI

Updated Travis config to use Focal, added py3.8 and py3.9.

TFC 1.21.09 Update log

Stability

Fixed broken Tails installer.

TFC 1.21.08 Update log

Dependencies

Updated
- charset-normalizer to 2.0.3
- pip to 21.2.1
- setuptools to 57.4.0
- virtualenv to 20.6.0
  - platformdirs to 2.1.0

Stability

Improved release pipeline to better detect hash mismatches.

TFC 1.21.07 Update Log

Dependencies

Updated
- cffi to 1.14.6
- execnet to 1.9.0
- importlib-metadata to 4.6.1
- mypy to 0.910 (with following new sub-dependencies)
  - types-requests 2.25.0
- packaging to 21.0
- pip to 21.1.3
- pytest-xdist to 2.3.0
- requests to 2.26.0 (this finally resolves the pinned idna version)
  - charset-normalizer 2.0.1 that replaces chardet sub-dependency
- setuptools to 57.1.0
- urllib3 to 1.26.6
- virtualenv to 20.5.0 (with following new sub-dependencies)
  - backports.entry-points-selectable 1.1.0
  - platformdirs 2.0.2
- zipp to 3.5.0

TFC 1.21.06 Update Log

Dependencies

Updated
- The Entire Flask ecosystem
  - Flask to 2.0.1
  - click to 8.0.1
  - itsdangerous to 2.0.1
  - Jinja2 to 3.0.1
  - MarkupSafe to 2.0.1
  - Werkzeug to 2.0.1
- attrs to 21.2.0
- certifi to 2021.5.30
- execnet to 1.8.1
- importlib-metadata to 4.5.0
- more-itertools to 8.8.0
- pip to 21.1.1
- pydocstyle to 6.1.1
- pytest to 6.2.4
- pytest-cov to 2.12.1
- setuptools to 57.0.0
- six to 1.16.0
- typed-ast to 1.4.3
- typing-extensions to 3.10.0.0
- urllib3 to 1.26.5
- virtualenv to 20.4.6
- zipp to 3.4.1

Security

Fixed

PIP #9827 - Don't split git references on unicode separators which is a CVE numberless vulnerability that allowed hijack of commit based pinned dependencies. This does not concern TFC users as no commit based pinned dependencies are used, only releases.
Unencrypted (but authenticated) anonymous installation of apt dependencies is now also encrypted with apt-transport-https.

TFC 1.21.04 Update Log

Dependencies

Updated
- cryptography to 3.4.7
- importlib-metadata to 3.9.0
- setuptools to 54.2.0
- urllib3 to 1.26.4
- virtualenv to 20.4.3

Security

Fixed

CVE-2021-28363 which is a vulnerability in the way the library validates TLS certificates. This does not concern TFC users as Tor manages the TLS-like encryption when requests client connects to Onion Services.

TFC 1.21.03 Update Log

Dependencies

Updated
- cffi to 1.14.5
- cryptography to 3.4.6
- importlib-metadata to 3.7.2
- Jinja2 to 2.11.3
- requests to 2.25.1
- setuptools to 54.1.1
- urllib3 to 1.26.3
- virtualenv to 20.4.2
- zipp to 3.4.1
Redesigned release pipeline to allow pinned dependency versions, and replaced coarse requirements hash pinning with automated hash pinning of all wheels etc. that match supported Python3 minor versions.

Security

Fixed

CVE-2020-36242 which is a buffer overflow error when encrypting large files with pyca/cryptography's fernet module. This vulnerability does not concern TFC as pyca/cryptography is not used to encrypt data symmetrically in TFC. TFC uses different library (pyca/pynacl) for that.
CVE-2020-28493 is a DoS vulnerability that affects the urlize function of Jinja2 v2.11.2, which TFC does not use; Relay Program's Flask server does not render clickable links to the user.
CVE-2020-25659 is a Bleichenbacher timing attack against RSA in pyca/cryptography PKCS#1.5 RSA implementations. TFC only uses RSA for digital signature of the installer. The signing is done via gpg, which does not use pyca/cryptography. Also, in digital signatures the decryption key is the public signature verification key anyway, so timing attacks against the decryption operation (=obtaining authentic copy of the installer's hash) on user-side, doesn't give the attacker any kind of advantage.

Platform support

Added support for Python 3.9 and the upcoming *buntu 21.04 releases: TFC installs on current Ubuntu 21.04 daily build, but if the final release breaks something, TFC's installer will be fixed on the April's maintenance update.

TFC 1.20.12 Update Log

Dependencies

Updated
- attrs to 20.2.0
- certifi to 2020.12.5
- cryptography to 3.3.1
- cffi to 1.14.4
- chardet to 4.0.0
- importlib-metadata to 3.1.1
- more-itertools to 8.5.0
- packaging to 20.4
- pyserial to 3.5
- requests to 2.25.0
- setuptools to 51.0.0
- urllib3 to 1.26.2
- virtualenv to 20.2.2

Features

The Relay Program now caches outgoing packets on disk (under $HOME/tfc) from where they are loaded by Flask when correct URL token is provided. Previously if the Relay Program of the contact went offline, any message sent by the user would be lost in the void, if the user also turned off their Relay Program. This is no longer the case. All buffered messages/files are delivered to the recipient the next time both users are online.

This even applies to Tails if the user has enabled persistence: When running on top of Tails, the Relay Program now stores all user data under $HOME/Persistent/tfc instead of $HOME/tfc. If however persistence is not enabled, the $HOME/Persistence/ directory (along with the TFC installation) is lost in the void once the Networked Computer is rebooted.

The buffered data is protected by a new key called the buffer_key which is a domain separated value of the TFC Onion Service private key. The buffer_key is used for two things
- It is used as the symmetric key for additional layer of encryption added prior to storing data on disk. The key is static to allow cross-session decryption of buffered data. The cipher used for encrypting the persistent data is as always, XChaCha20-Poly1305.
- To store the information about to which contact a buffered ciphertext is intended, the Relay Program creates a subdirectory for said contact. The name of the subdirectory is a keyed BLAKE2b digest of the Onion Address. They key in question is the buffer_key.
The protection provided by the buffer_key is mediocre at best: it's not possible to protect any key delivered via the serial interface as malicious program might reserve the interface for itself first. The only alternative to delivered key would be to add a password for the Relay Program. This might cause the user to reuse their Transmitter/Receiver Program's password, and as the Networked Computer is by far the easiest to infect with a keylogger, we therefore argue a password would actually reduce overall endpoint security. The buffer_key provides some protection against low-strength adversaries (who have seized the endpoint) trying to e.g. discover to which Onion Addresses the user talks to. High-strength adversaries are assumed to be able to compromise the RAM allocated for the Relay Program, so they have access to all plaintext data (encrypted/decrypted with the buffer_key) anyway. The protection provided by the buffer_key is purely additive: the actual message protection is provided by the TFC message keys that are never available to the Networked Computer.

Stability

Fixed issue where aborting master password change on Source Computer would halt output of messages until the Transmitter Program was restarted.
Fixed issue where changing logging/file reception/notification setting for all contacts would display a malformed confirmation message on Receiver Program. Note that the setting changes still take place, thus this bug is mostly cosmetic.

Code quality

The number of required words in automatically generated master password is now calculated before choosing the words.

TFC 1.20.11 Update Log

Dependencies

Updated
- cryptography to 3.2.1
- setuptools to 50.3.2
- urllib3 to 1.25.11
- virtualenv to 20.1.0
- zipp to 3.4.0

Security

Updated cryptography to fix CVE-2020-25659. Note that since TFC exclusively uses X448 (the public key exchange algorithm is hard coded), this timing attack does not concern TFC. Users do not need to take any action.

Platform support

Added support for Pop!_OS 20.04 LTS / 20.10

Code quality

Small fixes here and there

Documentation

Improved Relay.py schematic

TFC 1.20.10 Update Log

Dependencies

Updated
- cffi to 1.14.3
- cryptography to 3.1.1
- importlib-metadata to 2.0.0
- setuptools to 50.3.0
- urllib3 to 1.25.10
- virtualenv to 20.0.33
- zipp to 3.3.0

Security

Switched inter-VM communication of TFC Qubes endpoint from UDP packets via sys-firewall to Qubes' qrexec RPC calls. This removes TCP/IP stack, which in turn makes the attack surface much smaller and ensures better TCB isolation.

Stability

Fixed improper exception throwing on Transmitter Program.

Code quality

Small fixes here and there

TFC 1.20.07 Update Log

Dependencies

Updated
- appdirs to 1.4.4
- argon2-cffi to 20.1.0
- certifi to 2020.6.20
- click to 7.1.2
- distlib to 0.3.1
- idna to 2.10
- importlib-metadata to 1.7.0
- PyNaCl to 1.4.0
- requests 2.24.0
- setuptools to 47.3.1
- six to 1.15.0
- virtualenv to 20.0.25

Platform support

Linux Mint 20 has updated its Tor package: TFC now runs on it.

Code quality

Minor code quality improvements

TFC 1.20.04 Update Log

Dependencies

Updated
- Certifi to 2020.4.5.1
- Click to 7.1.1
- cryptography to 2.9.2
- Flask to 1.1.2
- importlib-metadata to 1.6.0
- Jinja2 to 2.11.2
- setuptools to 46.1.3
- urllib3 to 1.25.9
- virtualenv to 20.0.18
- Werkzeug to 1.0.1

Security

Updated signing key prematurely for new signing environment

Platform support

Support for running TFC on *buntu 20.04 LTS with the bundled Python 3.8

Stability

Fixed issue where canceling file transmission gave incorrect error about missing file. This was most likely due to API change within Tkinter.
Updated Argon2 master.zip digest in unittests

Code quality

Fixed some type annotations
Removed unnecessary f-strings
Removed unnecessary setuptools removal from uninstaller

TFC 1.20.03 Update Log

Dependencies

Updated
- cffi to 1.14.0
- idna to 2.9
- pycparser to 2.20
- requests to 2.23.0
- setuptools to 45.2.0
- Werkzeug to 1.0.0
- Virtualenv to 20.0.8 (with following new sub-dependencies)
  - Appdirs 1.4.3
  - distlib 0.3.0
  - filelock 3.0.12
  - importlib_metadata 1.5.0
  - zipp 3.1.0

Platform support

Support for running TFC on Qubes 4.0.3.
- New installation configurations, launchers etc. for Qubes
- Add socket based connections for inter-VM communication with UDP packets.
- No Reed-Solomon erasure codes are used for packets in the Qubes version.
- Installer prompts the user for VM IPs and handles the iptables rules.
Linux Mint Debian Edition 4 has updated its Tor package: TFC now runs on it.

UX

Faster time cost search function for Argon2id. Transmitter and Receiver Programs now estimate the time per-round of key derivation to figure out some upper bound for time cost. The upper bound will then be used in binary search to find a suitable time cost value much faster. This feature is especially important for low memory platforms such as Qubes VMs where the required time cost value can be much higher than in normal Source/Destination Computers where gigabytes of RAM are usually available.

Stability

Fixed an issue where injected non-ASCII characters in URL-token would show stack traces in the recipient's Relay-Program (note: this would not crash the Relay Program).

Code quality

Greatly refactored the installer
Fixed typos here and there

Documentation

Unified installation and launching instructions under single wiki article.
Threat model and installation wiki articles now categorize information based on the endpoint configuration used.
Created a new rendering for threat model that shows differences between different endpoint configurations.
Hid some of the more detailed information in wiki articles under collapsible sections.

TFC 1.20.02 Update Log

Dependencies

Updated
- Jinja2 to 2.11.1
- setuptools to 45.1.0
- stem to 1.8.0
- six to 1.14.0
- urllib3 to 1.25.8
- virtualenv to 16.7.9
- Werkzeug to 0.16.1

Hardware

Instructions for a new PCB-based datadiode with case, designed by cxcorp. Massive kudos!
- Downloadable Gerber-files for ordering the PCBs.
- Stereolithography files for 3D-printing a case for your data diode.
- Optional LED-board that displays through the case when data transfer takes place.
- As always, step-by-step instructions for assembly.

UX

New account and public key import diff viewers
- When importing a public key, incorrectly entered public key will now be exported back to the Relay Program together with the associated account. Relay Program will then display where the user had made typos -- compared to the latest received public key from that contact.
  
  To ensure the user doesn't accidentally shoot themselves in the foot during use, only correct length purported public keys will be exported from Source Computer. Thus, it's very unlikely the user will accidentally do the wost possible thing and export the local key encryption key (KEK) used in local key exchange.
  
  As an additional layer of protection to prevent that from happening, the BLAKE2b hash of the B58 encoded local key encryption key is now stored as part of local contact's data (it replaces the empty tx-fingerprint placeholder value). When the correct size (but mistyped) B58-encoded public key is entered, Transmitter Program will iterate over the entire string to form all possible substrings that could be the B58 encoded key encryption key. It will hash them and compare them against the stored hash of the original B58 KEK. If a match is found, the public key will not be sent to the Relay Program for visual diff display.
- When importing contact's TFC account, an incorrectly entered account will also be exported back to the Relay Program. Unlike public keys, in the case of accounts it's not possible to automatically detect errors, as it's not always possible to guess which account the user intended to type. However, there's two ways to help here.
  - Relay Program will observe received contact requests, and if the account is similar (difflib.SequenceMatcher ratio >= 0.75), it will display diffs against that one.
  - If the ratio is less than 0.75 or no accounts are available, Relay Program will now open a GUI prompt into which the program asks the user to copy-paste the TFC account. This prompt will only accept valid TFC accounts. Once provided, the Relay Program will then display the diffs between the paste one and the one manually typed into the Transmitter Program.
  To protect from accidental input of sensitive values (such as the local key encryption key (KEK)) to the account prompt, the Transmitter Program only exports correct length mistyped accounts to the Relay Program for diff comparison. Trying to copy-paste a KEK into the account prompt will result in length error. Furthermore, even if the user decides to append arbitrary chars to the KEK to meet the length requirement, they will only be met with another error about the input not being lower case (the B58 encocded format contains with very high probability at least one upper case char). So in order for the user to shoot themselves in the foot they would have to
  - Not understand the difference between account and KEK
  - Not take the hint from input field size
  - Copy the KEK to clipboard for pasting to wrong field (which is pointless considering they're supposed to enter it by hand to Receiver Program on Destination Computer to proceed).
  - Concatenate arbitrary data around the KEK
  - Change all upper case KEK chars to lower case
Note: Diff display messages only help with inputting TFC accounts and public keys, thus local key decryption key (same as KEK) import has no such feature. Luckily, the format of KEKs is the most convenient with its three character segments and the key guide. Out of the three cases, this is -- by a significant margin -- where the least amount of typos happens.

Code quality

Undid black styling as it actually hurt readability of code.

Documentation

Rendered new pictures for the new PCB data diode
Created new pictures for key diff guides and updated how-to images.
Clarified stuff here and there

TFC 1.19.12 Update Log

Dependencies

Updated
- virtualenv to 16.7.8
- certifi to 2019.11.28
- setuptools to 42.0.2

Security

Tor port number is now chosen using SystemRandom instead of random.
Switched to using absolute paths for binaries called by the software (Travis prevents using them for tests however).

Usability

Initial master key derivation now displays parameters being tested and the key derivation time once the parameter testing round finishes. This ensures the user is aware of what is going on while the process is taking place.

Code quality

Split functions to smaller pieces to reduce McCabe complexity.
Solved last cyclic import that required in-function import.
Added mypy badge as --strict --ignore-missing-imports no longer complains. Removed more Any annotations.
Added Codacy code quality metrics badge.
Added CodeFactor code quality metrics badge.
The code style is now black.
Removed some duplicate code.
Refactored code here and there.

TFC 1.19.11 Update Log

Dependencies

Updated
- argon2_cffi to 19.2.0
- cffi to 1.13.2
- Setuptools to 41.6.0
- six to 1.13.0
- urllib3 to 1.25.7

Security

Enabled --no-deps flag for dependency installation via PIP. This prevents any third party packages not explicitly allowed.

Confidentiality

Changing master password now requires the user to enter the current master password first. This plugs a hole in accessing logs by changing the password to something the physical attacker chooses.

Availability

All TFC databases are now atomic. This should prevent corruption of databases in case of a sudden power outage.
Master password change and resulting database re-encryption is now also atomic.

TFC 1.19.10 Update Log

Dependencies

Updated
- certifi to 2019.9.11
- cffi to 1.13.1
- cryptography to 2.8
- jinja2 to 2.10.3
- PySocks to 1.7.1
- Setuptools to 41.4.0
- urllib3 to 1.25.6
- virtualenv to 16.7.7
- Werkzeug to 0.16.0
Removed
- asn1crypto

Installation

Added support for *buntu 19.10.
Added support for PureOS.
Added Relay Program support for Tails 4.0 - The installer uses a separate, more compact one-liner that ignores dpkg lock check. Tails makes it possible to run a more safe Onion Service: In case the networked endpoint is compromised in real time, the OS contains no identifying user data (hard disks should not be connected to enforce this). Furthermore, Tails uses the onion-grater program that whitelists applications that have the permission to use dangerous Tor control port commands: It e.g. prevents user-level-privileged malware on Tails from obtaining the real IP-address of the system.

Security

Entering the password generate now generates a +128-bit password for the user using the EFF wordlist. Note: The generated password will be visible on screen, thus the user must be alone when this feature is used. The password should be written down, and the copy should be kept secret and be disposed of as soon as the user has memorized it.
Fixed Argon2 to determine memory by looking at MemAvailable instead of MemFree. This fixes slow initial master key derivation and increases security as more memory is used.

Stability

Added missing confirmation codes for delivery of Tx-PSKs.

Code quality

Fixed Tor port type annotations.
Removed several Any type annotations.
Increased the amount of run-time validation for the outputs of crypto libraries.
Removed star imports.

UI

New icon color scheme that goes well with both OnionShare and Tor Browser icons.
Fixed typo in public key input guide.

Documentation

Added instructions on how to generate the password.
Updated color scheme for Tor related renderings.

Possibly bugs:

Tor requires accurate time information. If the system time has any offset (encountered at least with PureOS and Debian during testing), torsocks refuses to function properly and the TFC won't install.

To protect users' privacy, TFC is installed over Tor. Speed of Tor varies, so sometimes some component can time-out, and TFC won't install in those cases. There is no better fix than to try running the installer again.

TFC 1.19.08 Update Log

Dependencies

Updated
- virtualenv to 16.7.3
- PySocks to 1.7.0
- requests to 2.22.0
- certifi to 2019.6.16
- urllib3 to 1.25.3
- Flask to 1.1.1
- Werkzeug to 0.15.4
- cryptography to 2.7

Installation

Added support for Debian 10. Debian only runs FOSS software, and has reproducible builds for ~everything. Debian also runs Wayland instead of X11 by default, which offers GUI isolation between applications. This adds some protection against keyloggers Destination Computer might receive from the Networked Computer (or from a malicious peer over file transfer).
Non-dev install configurations now only make a shallow clone of the repository. This speeds up download and reduces bandwidth use.
Moved setuptools from APT to PIP dependency. This means more recent version is used, and it also reduces the number of dependencies that don't have a pinned hash.
Dev configuration installation now backups $HOME/tfc directory instead of removing it.

Security

Switched from Argon2d to Argon2id as the algorithm adds side-channel protection in a situation where malware from Networked Computer (or from a peer via file transfer) infects the Destination Computer and does a side-channel attack to extract information that speeds up password brute force. Note that if the Destination Computer is running X11, any user level program can access the keyboard and steal the master password. Using Debian 10 with Wayland (that features GUI isolation) instead of X11 is therefore highly recommended.
Updated Werkzeug to 0.15.5 to fix CVE-2019-14806. Fixing this security bug only requires the Relay Program on Networked Computer to be re-installed. This can be done without replacing hardware.
Logfile view and export now requires the user to enter the master password. This can be disabled from settings by setting ask_password_for_log_access to False, and then entering the master password. The purpose of the feature is to ensure even if TFC is left open, it's not possible to view message logs without the password. However, the user must still take care to reset the terminals with the /reset command to hide the ephemeral message log as well as any log files viewed during the session with the /history command.
Improved security of the ChaCha20 DRNG by requiring use of kernel version 4.17 or newer. This ensures that when CPU HWRNG is not trusted, in addition to fast_pool, the DRNG also seeds itself from the input_pool before considering itself fully seeded.
Removed the check_kernel_entropy function as input_pool state does not reflect the ChaCha20 DRNG seed level or state.

Code quality

Fixed type annotations so that mypy doesn't complain even with --strict (and --ignore-missing-imports) flag. There are still several Any annotations that need replacing so type checking is still a work in progress.
Significantly refactored src.common.crypto and its tests.
Significantly refactored install.sh.
Refactored dd.py and added tests for it.
Fixed deprecated syntax used in shred subprocess command executed during wipe.
Fixed typos here and there.

Documentation

Significantly improved docstrings of src.common.crypto and tests.common.test_crypto.

Stability

Fixed bug in automatic Argon2 key derivation parameter selection where incorrect value of memory parameter was stored in some cases.
Fixed name parsing in file transfer during traffic masking.

Development

Mocked os.system('reset') calls in unit tests: all tests can now be run in PyCharm as well.
Hid DeprecationWarnings from third party libraries when running tests with py.test.

Usability

Reduced longest online check delay of contacts from 16 to 8 seconds. This makes contact appear online sooner when they log in.

TFC 1.19.04 Update Log

Dependencies

Updated
- Platform to Ubuntu 19.04 (that has Tor 0.3.5 in its repository)
- Python to 3.7
- certifi to 2019.3.9
- cffi to 1.12.3
- Jinja2 to 2.10.1
- urllib3 to 1.24.2
- Werkzeug to 0.15.2

Installation

Network interface killer now leaves loopback interface on. This reduces the time it takes to install virtualenv dependencies for TCB configuration.

Security

Switched to best practices in Argon2 key derivation. Transmitter and Receiver Programs now use all free memory in the system, and increase time_cost (iterations) from one until key derivation takes as long as it needs (3..4 seconds). If after time_cost has been determined key derivation takes over 4 seconds, suitable value for memory_cost is determined. Closes #6. Unfortunately, due to TFC's endpoint security, providing automatic updates to TCB side software is impossible. Improving Transmitter or Receiver Program's Argon2 security requires reinstalling TFC on a new computer.
Changed Argon2 memory_cost for PSKs from 62.5 mebibytes to 512 mebibytes. This will break backwards compatibility with previous versions, but it's better to set more reasonable value sooner than later.
Fixed broken /wipe command during traffic masking. Unfortunately, due to TFC's endpoint security, providing automatic updates to TCB side software is impossible. Fixing this bug requires reinstalling TFC on a new computer.
Updated Jinja2 to 2.10.1 to address CVE-2019-10906. Fixing this security bug only requires the Relay Program on Networked Computer to be re-installed. This can be done without replacing hardware.
Updated urllib3 to 1.24.2 to address CVE-2019-11324. Fixing this security bug only requires the Relay Program on Networked Computer to be re-installed. This can be done without replacing hardware.

Code quality

Switched from custom implementation of PKCS7 padding to the function offered by the cryptography library (merges #7.) Thanks @tannercollin!
Removed kill_background_tor() function as TFC has not used separately compiled version of Tor for some time.
Replaced last assert with conditional expression (related to #4).
Refactored code here and there.
Fixed some comments and docstrings.
Fixed some broken tests.

TFC 1.19.03 Update Log

Dependencies

New
- Git
Updated
- virtualenv to 16.4.3
- cryptography to 2.6.1
  - cffi to 1.12.2
- markupsafe to 1.1.1
Removed
- deb.torproject.org-keyring
- setuptools PIP-dependency

Installation

Moved TFC public key from unreliable PGP key servers to the GitHub repository.
Re-designed installer one-liner is now more compact and secure: The one-liner now sets permissions of downloaded files before authenticating and running them.
Fixed issue where gpg threw error gpg: packet(13) too large when loading Tor's PGP key from the SKS key server. Installer now loads Tor's signing key from Tor Project's Onion Service (http://sdscoq7snqtznauu.onion) instead.
Fixed issue where installer duplicated lines in /etc/apt/sources.list.d/torproject.list.
Replaced torify with torsocks as torify is nowadays only a wrapper for the other.
Non-TCB installers now upgrade Tor to 0.3.5 before downloading further dependencies using torsocks.
Added separate terminator config file for dev install configuration that has correct paths (fixes #5). Added a launcher for the dev configuration.
Added uninstaller for the program. The uninstallation can be started by running bash /opt/tfc/uninstall.sh
The installer no longer downloads every file from GitHub repository individually. Instead, it now clones the repository, removes unnecessary files and authenticates the remaining ones. This is much faster.
The installer configurations for local testing, TCB and Relay configurations now appear identical until they reach the point where the TCB configuration disconnects. This makes it harder for the attacker to predict if the installation is for Source Computer.
The installer no longer creates backup of empty user data directory when the software is re-installed.

Security

Fixed bug where constant time context manager for URL token iterator returned prematurely. Note that this bug does not cause a timing attack side channel to determine URL tokens. At worst this bug allows an approved contact to determine how many contacts the user has added before them. Fixing this security bug only requires the Relay Program on Networked Computer to be re-installed. This can be done without replacing hardware.
Moved local testing configuration's Terminator config file to /opt/tfc/ from where it can not be edited without root privileges. This prevents malware from editing the path to TFC withing the config file*. Although local testing configuration is not intended to be endpoint secure, some degree of security can be achieved with access control list. Thus, users who use local testing configuration for communication should update their clients.

*It also prevents user from making edits to Terminator's TFC profile/layout without editing the Terminator config file manually. This also means users who have created their own terminator configuration file can keep using it without editing TFC's config file.
Replaced all asserts with conditional expressions (fixes #4).

Usability

Fixed issue where restarted Transmitter Program was unable to reconnect to running Relay Program because Relay Program did not display confirmation code of consecutive UNENCRYPTED_ONION_SERVICE_DATA packets.

Code quality

Refactored code here and there.
Updated tests where needed.

Documentation

Removed typos where found.

Known bugs

During installation, torsocks might throw 1551596718 ERROR torsocks[3869]: Unable to resolve. Status reply: 4 (in socks5_recv_resolve_reply() at socks5.c:683). This should be mitigated by rerunning the installer one-liner.

TFC 1.19.01 Update Log

Platform

Moved support from Tails 3.1 to future versions of Tails built over Debian Buster, that features Tor 0.3.5.
Moved support from *buntu 17.04 to *buntu 18.04 LTS that features
- Python 3.6.5
- uses GNOME 3.28 that ensures more reliable operation of network interface killer for TCB configuration
- Terminator 1.91 (Install candidate)

Dependencies:

Updated
- argon2_cffi to 19.1.0
- cffi to 1.11.5
- PyNaCl to 1.3.0
- six to 1.12.0
- virtualenv to 16.2.0
New dependencies (subdependencies of dependencies are indented):
- Cryptography 2.5
  - Asn1crypto 0.24.0
- Flask 1.0.2
  - Click 7.0
  - ItsDangerous 1.1.0
  - Jinja2 2.10
  - MarkupSafe 1.1.0
  - Werkzeug 0.14.1
- PySocks 1.6.8
- Requests 2.21.0
  - Certifi 2018.11.29
  - Chardet 3.0.4
  - Idna 2.8
  - urllib3 1.24.1
- Stem 1.7.1
- Tor 0.3.5
Removed dependencies
- Pidgin
- Pidgin OTR

Installation

TFC is now installed into /opt/tfc, and the program code can no longer be altered without root privileges. User data is still stored into $HOME/tfc/user_data and $HOME/tfc/received_files.
Ubuntu installation configurations no longer require rebooting to enable serial port access.
The Relay Program configuration for Ubuntu now uses Python 3.6.
The Relay Program installation on Tails now asks for sudo password only once at the start of the installation.
Added development environment installation configuration for the *buntu platforms.
If the installer finds an existing configuration file for Terminator, it now displays a warning that informs the user under what name the backup file was stored.
The installer now configures the Terminator program's font size according to user's screen resolution. The most common screen widths (1368px, 1600px, and 1920px) are supported. Note that for Ubuntu, the dash should be moved to the bottom of the screen.

Security

Switched from X25519 to X448 key exchange that provides 224 bits of symmetric security over the 128 bits of X25519.
Switched from XSalsa20-Poly1305 to XChaCha20-Poly1305 (IETF-variant) that provides better diffusion.
Switched from Pidgin and XMPP accounts to native Tor v3 Onion Service backend to provide anonymous communication.
Receiver Program now also shreds the content of received_files directory and removes the user data directories during the /wipe command.
Switched from BLAKE2s to BLAKE2b that's optimized for AMD64 architecture TFC is designed to be run in.
Replaced hash_chain that did not have a security proof with BLAKE2b. This increases the hash ratchet performance significantly.
Removed the risk of falling into short cycles of keys within the hash ratchet, by hashing the previous key together with the hash ratchet counter.
Added protection against zip bombs for received data. The maximum size of decompressed file data is defined by a new setting max_decompress_size. The maximum size for messages is fixed to 100,000 bytes.
Fixed issue where a frenemy could crash the user's Receiver Program with file packet, the file name header of which contains a slash ('/').
Changing TFC's database sizes with /set command now takes effect immediately on both Source and Destination Computer.
tm_static_delay or tm_random_delay now have minimum setting and changing the value displays a warning that altering the value can make the traffic and endpoint of user look unique to an attacker.
Added constant length padding for nicknames in packets containing contact keys, sent to the Receiver Program when adding a new contact.
Transmitter program now clears readline history when /reset command is issued. This prevents accessing sent messages via the readline history feature.
Added terminal reset, readline history clearing and clipboard clearing (of kdk is on clipboard) during local key setup. This ensures sensitive key material is not available to someone observing the system after local key exchange.
Prevented user from sending TFC databases to contacts as files.
The installer one-liner now first installs Tor before downloading and installing TFC anonymously. This may be slower, but it makes it harder for a global adversary to determine which endpoint it should compromise during install.

Added pinned SHA256 hash of public key to installer one-liner that is checked after the key is imported based on its SHA-1 fingerprint. Here is the formatted version of the one-liner:

  c='local'
  h='20b51fc9aaea3a5493c7cf513a3c875764980ebd33eea2689748761ae697652f'
  fp='EA8F38BA674B242E9944845C981370E9725A0FBA'
  f='tfc_key.asc'
  ks='hkp://jirk5u4osbsr34t5.onion'

  while sudo fuser /var/lib/dpkg/lock >/dev/null 2>&1; do
      sleep .5
      echo -ne "\rAPT is busy"
  done &&
  
  sudo apt update &&
  sudo apt install tor -y &&
  
  gpg --keyserver $ks --recv $fp &&
  gpg --export -a $fp > $f &&
  if sha256sum $f | grep -Eo '^\w+' | cmp -s <(echo $h); then
         rm $f &&
         torify wget https://raw.githubusercontent.com/maqp/tfc/master/install.sh{,.asc} -q &&
         gpg --verify install.sh{.asc,} &&
         bash install.sh $c
      else
          rm $f
          echo "ERROR: Signing key had invalid SHA256 hash"
      fi

The reason for the SHA256 hash is the delayed standardization of the next generation PGP (v5) fingerprints by the IETF OpenPGP WG, which means PGP users are currently vulnerable against nation-state actors capable of producing key pairs with colliding SHA-1 fingerprints.

Fixed a category of side-channel attacks where malware on Destination Computer could
- show sensitive data like keys in place of unknown accounts when the Receiver Program displays group management messages. To prevent the attack, group management messages are now displayed by the Relay Program on Networked Computer, that does not have access to sensitive keys.
- delay showing messages (like public keys, messages, and files) based on individual key bits, which could lead to user taking action on their Source Computer, that could in turn leak key bits to the Networked Computer. To solve this, the Relay Program delivers the 1ms resolution reception timestamp of received message to the Receiver Program. The user can detect a message from a contact displayed by the Receiver Program was timed according to key material, when no matching timestamp is also displayed by the Relay Program. Any difference between the current Destination Computer time and timestamp of received message that is significant enough to alter user behaviour, can be detected by the user. It's not perfect, but it's all that can be done.
Despite the protections, malware on Destination Computer might display questions from frenemy based on key material, and user's reply could be used to exfiltrate keys one bit at a time to that frenemy. Considering the security problem, such an attack cannot be fixed with security architecture, only by vetting contacts and if possible, isolating trusted and untrusted contacts on separate (best case scenario being, individual) TFC endpoints.
Indirect: https://pgp.mit.edu has upgraded its TLS configuration meaning slight improvement to overall security.

Stability

Switched from Serial().read() to Serial().read_all() to read the exact size of serial interface buffer every time.
Tweaked serial timeout settings to significantly increase data transmission reliability.
Fixed an issue where TFC would not immediately update Source Computer side key database size when the max_number_of_contacts setting was changed.

Usability

New user-friendly naming scheme:
- TxM is now Source Computer
- NH is now Networked Computer
- RxM is now Destination Computer
- TFC-TxM is now Transmitter Program
- TFC-NH is now Relay Program
- TFC-RxM is now Receiver Program
This allows referring to each program and computer with a single word (e.g. Source or Relay).
The Source Computer handles the storage of the persistent Onion Service private key. What this means is, the Relay Program is able to maintain static onion address across sessions, even if it's run on an amnesic distro like Tails.
When adding a contact, the Transmitter Program no longer needs to ask the user to enter their account that corresponds with the contact's account.
When adding a new contact, a friend request is sent to the recipient. It is thus enough one contact knows the TFC-account of the other.
Diffie-Hellman key exchange can now be interrupted with Ctrl+C, e.g. if the contact is not available, or if arrival of the public value takes too long. This sets the key exchange state to "Pending". The key exchange can be resumed by selecting the contact. This will reload the ECDHE private value generated earlier, and re-send the public value to the contact. Caching the ECDHE values prevents confusion about which public value is the correct one. Note however, that if Transmitter program is restarted, it will generate a new ECDHE keypair when pending contact is re-selected.
Added support for /add, /rm, and /connect commands in the contact selection screen. This prevents the user from being unable to add new contacts until the key exchange is completed for the first contact.
Significantly improved handling of local keys: Receiver Program now stores the BLAKE2b hashes of local key packets and local key decryption keys for the duration of session. Together, these allow the program to
1. notify the user in case they entered an expired key decryption key
2. notify the user in case the received local key packet was an old one
3. automatically drop repeated local key packets: these packets no longer have to be manually canceled
4. automatically attempt decryption of queued local key packets in case user enters local key decryption key for packet that the Receiver Program has received, but that it has not yet loaded for processing.
During key exchange, the fingerprint verification can now be skipped with Ctrl+C. This shows the user a warning and sets the fingerprint verification state to "Unverified" (displayed in the contact list). This is an easy way for the user to keep track of key authentication status; no more lying to the "I have verified fingerprints" question.
Added a new command, /verify, that displays contacts' fingerprints, asks the user to confirm whether they match, and depending on the answer, stores the Verified/Unverified value to the contact database. The value can be changed afterwards if the user, e.g., feels the verification channel was not secure.
/psk command now shows a confirmation code on Receiver Program after successful import of contact's PSK. When the code is entered to the prompt on Transmitter Program, the program removes the "(No contact key)" reminder from contact list. This helps the user keep track about which PSKs they have not imported yet.
Symmetric keys derived during ECDHE key exchange now also require confirmation code. This ensures key exchange succeeds even if the key delivery packet would drop.
Set Networked Computer bypass messages setting (nc_bypass_messages) to False (=disabled) by default. This feature is now marked for deprecation sometime in the future.
Selecting window with unread messages now displays a horizontal line between the old and new messages.
The Relay Program now shows the arrival time of public keys.
During traffic masking, the ephemeral notification about file being received is now directed to the active contact/group window. This is because during traffic masking the window can not be changed.
Moved settings for serial interface from the encrypted settings database to a JSON file in $HOME/user_data. This removes the need to display the annoying Receiver Program serial interface prompt that prevented the user from locking themselves out in case Relay and Receiver program serial settings accidentally end up in inconsistent state.
Groups now use random group IDs to identify themselves across peers. This means the user can choose the name for each group they join.
The user can now rename the active group with the /nick command.
Changing contact's nick now automatically reloads the window with the new nickname.
The Relay Program now displays when contacts come online or go offline.
Much faster file transmission. TFC now works as follows:
- Removed the /fe and /fi commands to make way for a new file transmission protocol.
- When traffic masking is enabled:
  - /file command sends the file the old way, in the background inside packets that are indistinguishable from messages.
- When traffic masking is disabled:
  - /file command encrypts the file and exports the ciphertext (CT), together with information about recipients, to the Relay Program's server that multicasts the CT to those recipients. The Relay Program of each recipient will forward the CT to their Destination Computer where the Receiver Program caches the CT if file reception is enabled for the sender of the file. The file packet is immediately followed by an automated, standard TFC message that delivers the decryption key for the CT. The key delivery message also contains the BLAKE2b digest of the CT to guarantee the integrity of the CT, to authenticate its sender, and to identify the correct CT for the decryption key.
As keys for file import/export are discarded, public keys are now using Testnet WIF addresses, meaning all keys that are input manually are distinct, and the user cannot mistakenly input the wrong key to the wrong place.
Increased TFC database padding settings to make databases of more users look identical
- 50 contacts
- 50 groups
- 50 members per group
Fixed an off-by-one error where TFC key/contact database would count local contact/key as part of members of the database, meaning actual number of contacts that could be added was one less than what the settings would imply.
Removed the multi_packet_random_delay setting as it's no longer needed.
Traffic masking can now be enabled and disabled without restarting the Transmitter Program.
The serial interface setup during first launch now prevents the user from choosing an unavailable USB-to-serial/TTL adapter or unavailable built-in serial interface.
Moved the definition string for built-in serial interface from gateway module to the serial settings JSON file. (This string might need to be manually edited for some OSs, e.g., Raspbian uses /dev/serial0 or /dev/ttyAMA0).

Performance

Updated the Reed-Solomon codec to the fork by Stephen Larroque that runs ~10% faster than the original implementation under default settings. Added type annotations to the implementation.
Databases now store data in more compact form:
- Databases no longer store accounts as 1024-byte, padded, UTF-32 encoded bytestrings, but as the 32-byte Ed25519 public key of the Onion Service, which is the most compact representation of v3 onion addresses and thus, TFC accounts. The new format means
  - 63% smaller contact databases
  - 75% smaller log databases
  - 85% smaller key databases
  - 92% smaller group databases (compared to the old setting of 20 members per group)
Re-designed group management messages to require no multicasting over the serial interface.
Faster data transmission over the network: Ascii85 encoding instead of Base64.
Removed a redundant layer of base85 encoding from file transmission during traffic masking. This increases transmission efficiency from 80% to 100%, meaning 25% faster file transfer during traffic masking.

Code quality

Removed redundant relay.gateway and relay.misc modules. Moved relay.settings under src.common.gateway.
Removed magic numbers from all database field values.
Expanded use of statics, ensured each field and header has distinct value for length.
Improved readability by refactoring code.
Improved tests and added mock.patch decorators.
Improved test coverage to 100% with practically no ignores. (Note that Travis requires skipping some tests, thus coverage will be capped to 99%.)
Improved mypy coverage: --disallow-untyped-defs no longer shows warnings. This still requires more work.

Documentation

The Wiki now provides URLs for images in case GitHub fails to render images behind camo URLs.
Created new renderings.
Expanded and clarified almost everything.
Fixed grammar and typos here and there.
Switched docstring parameters to non-standard (but more readable) in-line parameter descriptions.

TFC 1.17.08 Update log

TFC is looking good

To summarize

Protocol looks complete
TFC uses complete suite of latest and greatest ciphers and algorithms:
- X25519
- XSalsa20-Poly1305
- Argon2d
Hardware layout has improved a great deal since the project started.
Test have 99.71% coverage (98.18% with no ignores).
Mypy static type checking is complete, only typeshed related warnings are displayed
100% of user data at rest is encrypted and padded to the extent the only metadata that leaks, is the maximum number of contacts/groups and the maximum number of logged messages.
Installer has a built-in authentication mechanism.

That being said, after four years, it is a great pleasure to announce the system is ready enough to have its major version number bumped to one. There's still room for improvement. In future TFC will hopefully ditch Pidgin as its underlying ciphertext delivery mechanism and switch to native Tor Onion Service routing. Routing via Tor Onion Services can already be done with Pidgin, but it is not the default. Cipher choices also leave some room for improvement: X448 with ChaCha20-Poly1305 is something to aim towards if/once they become available.

Security

Fixed security issue where during traffic masking (the proper term that replaces "trickle connection") user could change GUI dialog setting so that it would show on NH, revealing the use of TFC.
Switched from argon2 0.1.10 to argon2_cffi 16.3.0 library that is better maintained and tested. Since there are no shared users or remote access to TxM/RxM, switched from Argon2i to Argon2d that provides more security against GPUs/ASICs.
Updated PyNaCl to version 1.1.2, the wheel format of which is much faster to install.
Updated pycparser to 2.18
Updated cffi to 1.10.0
Updated pyserial to 3.4
Added kernel version check that ensures TFC only runs on systems with kernel 4.8 or later. The version requirement ensures that kernel CSPRNG uses the new ChaCha20 compressor.
Reset command now clears TxM screen before running the slightly laggy reset command. The clear function gives a few milliseconds faster protection against shoulder surfers.
/exit command will now reset terminals before exit in case terminal stays open when exiting. The reset function helps especially when the setting double_space_exits is enabled.
Added /whisper command that allows the user to bypass enabled logging setting of the recipient. This is purely an agreement based (nothing prevents recipient from editing their client), but in situation where the recipient can be trusted, this feature is useful for sending, e.g., decryption keys for files that the recipient will import with /fi command, as well as "off-the-record" messages (nothing to do with OTR-protocol), if users agree they generally want to log messages but sometimes not.
Added /wipe command that resets terminals, overwrites and removes all encrypted user data and powers off TxM, RxM and NH. on NH TFC will also shred $HOME/.purple to hide metadata about what accounts user was using and communicating with. With standard hard-disks (i.e., not SSDs) and DDR3, this should be sufficiently secure against cold-boot attacks that try to recover user data.
Removal of contact with /rm command now removes their ephemeral message log from all windows.
Fixed issues with notify setting for contacts/groups not hiding messages for contact properly.
Installer now uses SHA512 instead of SHA256 for pinned hashes of downloaded TFC files.
Moved to support only Ubuntu 17.04 as it comes with APT 1.4 that
- deprecates Ubuntu's 13-year-old, 1024-bit DSA key (only a more recent, 4096-bit RSA key is used)
- doesn't support repositories that use signatures that rely on SHA-1
- is 10-20% faster (meaning shorter window of exposure for TxM)
- eliminates the need for additional repositories for Python3.6 and Terminator
- has latest version (9.0.1) of pip in its repository
TCB configuration now installs the net-tools application that does not come with 17.04. The program allows the use of the current network interface disabling function.
Added priority queue for encrypted TxM to RxM commands (and local keys). The queue ensures that NH sends all queued commands between every assembly packet (and public key) received from contacts. Prioritized commands over public keys on RxM to prevent DoS for command loading.
Removed log_file_placeholder_data setting. It is pointless to store file placeholder data to hide file transmission because an adversary can easily tell file was transmitted from the burst packets. The only situation where that is not the case is with traffic masking.

Noise packet logging was streamlined. Now, regardless of traffic masking setting, nothing gets logged if active contact/group has not got logging enabled. TFC now has a single setting, logfile_masking, that when enabled logs all authenticated and decrypted packets during traffic masking. Received files, ephemeral group management / whispered messages as well as traffic masking noise packets are logged as placeholder data. Additionally, on receiving side if, e.g., file transmission is canceled and the packet is never decrypted, logfile masking logs placeholder data instead.
Randomized keys that are not meant to be used, in case they due to some bug they would be loaded for encryption/decryption purpose.
Changed 8-byte group message time stamp to 16-byte random value. The new header hides system time from recipients, who could use the information to fingerprint TxM. The field length was doubled to deal with randomized nature, that would have otherwise caused a collision with 50% probability for every 2^32 messages. The probability now requires 2^64 messages, which is the lifetime of the key (limited by the hash ratchet counter field).
There was a slight chance adversary that would compromise NH, could first send a public key and then inject imported file. If contact were to use the public key from adversary to decrypt it, that adversary could import a possibly malicious file to RxM. File decryption keys now use Testnet format instead of Mainnet format, keys so wrong keys entered in confusion will raise checksum error.

Stability

Fixed crash when accessing group's log containing a group management packet
Fixed crash when viewing logs for removed contact
Fixed crash when reloading group window containing removed contact
Fixed crash when giving an empty command ('/ ')
Fixed errors resulting from editing sequence during enumeration
Fixed issue where a message sent to a group was only logged once
Fixed issue where the local testing window would not open to the active workspace.
Fixed issue where /notify off all would disable command messages
Fixed issue where file import decryption key's input widget drops when a valid but incorrect key is given
Added -e flag to the installer to exit script if any error occurs
Fixed exit codes for programs
Fixed corner cases for some setting values
Fixed laggy /cm and /cf commands
Fixed concurrency issues for NH management commands with low baud rates. All commands now work from 50 to 2M bauds.

Usability

Added command /rmlogs {account,nick,group} that can remove log entries for specified accounts or groups. If all logs for accounts that are members of a group are removed, messages will also be removed from that group. However, if messages are removed from the group, private messages will not be removed from the log file.
Transmitter will prompt the user to remove TxM/RxM logs for contact when contact is removed with /rm command.
Transmitter will prompt the user to remove TxM/RxM logs for the group when a group is removed with /group rm command.
Renamed references to ECDHE to X25519.
Setting names are no longer constant length atrocities.
Changed behavior of notifications. All notifications are now enabled by default, but the only number of unread messages is displayed. Added setting new_message_notify_preview that when changed from the default value of False to True enables preview of messages.
Receiver program now shows temporary notification about received messages in start window that is now command window until the user selects window using TxM.
One-liners and installer now automatically wait until APT's /var/lib/dpkg/lock is released.
Combined local testing launchers to one with alternative layouts in right-click context -menu.
Base58 representation of X25519 Public keys is now the exact implementation of Bitcoin's Wallet Import Format. It uses SHA256d instead of the hash chain and prepends Mainnet or Testnet byte depending on for what key is used. The goal here is to reduce fingerprinting capability of TFC public keys in case users for some reason read public keys displayed by their RxM instead of fingerprints.

The effect of this is all public keys are slightly longer than before, but always 51 chars long. Local testing still displays the key as one easy-to-copy string, but in normal use, the keys are displayed in 17 substrings of three chars (which feels like the right size for the humans' working memory), with lettering from A to Q that helps check position when typing the key:

     A   B   C   D   E   F   G   H   I   J   K   L   M   N   O   P   Q
    5Ka 52G yNz vjF nM4 2jw Duu rWo 7di zgi Y8g iiy yGd 78L cCx mwQ mWV

Updated description of Signal fingerprint verification before E2EE call, since SAS has been replaced with Signal's standard fingerprint.
Receiver program's message preview notifications will now always include the type of window the message was received in. For private messages, it notifies that the window in question was a private window, and for groups, it will notify the user that it was a group window and also include the name of the group.
RxM no longer shows group management messages for added/removed members or group exit notifications if the group does not exist on Receiver program, or if contact is not a member of the existing group. Invitation / join messages will still be displayed as they should.
TxM and RxM now print and export history in the same format as normal chats do.
Valid but incorrect local key decryption key no longer exits local key bootstrap phase (first time when setting up).
Added minimum value for notifications so that user does not hide them by setting delay very small.
New column based transmission window now shows the percentage of completeness instead of the progress bar

Added alias join for group creation to make use more intuitive. Following commands do the exact same thing

  /group join team alice@jabber.org charlie@jabber.org
  /group create team alice@jabber.org charlie@jabber.org

Receiver program no longer just shows that logging, notification or file reception setting was changed, it also tells if the setting changed or if it was already at that value.
Disabled factory from launchers using gnome-terminal, which prevents launching TFC accidentally twice.
Added resend command to public key input so the user can easily resend their public key to contact.

Code quality

Changed the majority of magic numbers and strings to descriptive static values
Rewrote many modules
Removed serial interface detection for Raspbian until the OS can be fully featured
Removed unused command class stub and terminal resize function
Removed duplicate code from tx.commands and nh.misc
Removed debug message printed when the packet is canceled during traffic masking
Refactored tests, switched to proper setUp and tearDown logic for most tests
Refactored terminator config file
Increased test coverage to 99.71%
Fixed rest of the mypy warnings (third-party libraries are ignored)
Added missing type annotations
Tons of small improvements here and there

Directory structure

Renamed src/common/errors.py to src/commons/exceptions.py
Renamed src/rx/window.py to src/rx/windows.py
Renamed src/tx/tx_loop.py to src/rx/input_loop.py
Renamed src/rx/rx_loop.py to src/rx/output_loop.py
Renamed src/tx/trickle.py to src/tx/traffic_masking.py
Removed src/tx/messages.py

TFC 0.17.04 Update log

Code quality

Rewritten in Python3: 3.6 for TCBs. NH still uses Python 3.5 because the Tails 3 ships with it and because some modules such as dbus have not yet been updated for 3.6.
Split huge files to clean modules; This greatly reduces duplicate code. Some functions for NH.py were duplicated to avoid the need to install crypto modules on NH's virtual env.
As requested, TFC now features PEP484 type annotations for mypy static type checker.
Enabled Travis CI and unittest coverage report.
NH-side now has much clearer abstraction that makes writing plugins for other IM clients with IPC/sockets much easier.

Crypto / Security

TxM/RxM are Python 3.6 only as the version allows the explicit use of os.getrandom that always blocks when the entropy pool of the kernel CSPRNG has less than 128 bits of entropy. This happens for example during at the start of Live distros. As the 128 bits is kind if low, TFC startup routine now waits for /proc/sys/kernel/random/entropy_avail to have at least 512 bits of entropy before continuing. All this combined with the modernization of kernel CSPRNG in version 4.8 of Linux kernel to use ChaCha20 among other things, means HWRNG can safely be deprecated.
As PyNaCl library finally includes getter for shared secret, a separate fork of the library is no longer needed.
As requested, PBKDF2-HMAC-SHA256 was replaced with state of the art Argon2 key-derivation function. The used version is Argon2i as it is designed against local attacks.

There are two configurations for Argon2 parameters depending on use case:
- TxM/RxM persistent database encryption
  - h: Number of threads in CPU
  - m: Amount of available RAM during the creation of master password (at least 128MB as failsafe)
  - x: Number of round starts at 1 and is doubled for every test of key derivation under 3 seconds.
    
    (Resource requirements are reduced by 50% during local testing to allow simultaneous login on TxM/RxM. This is fine because local testing is not intended for secure use).
- PSK password protection
  - h: 1 as the number of threads on recipient's system is unknown
  - m: 128MB as the amount of RAM on recipient's system is unknown
  - x: 16 so that derivation is slow even on faster systems. SBCs like Raspberry Pi will suffer but the inconvenience is a one-time thing (assuming user makes no typos when entering the password).
Hash ratchet now uses a hash chain construction built from SHA3-256, BLAKE2s and SHA256. The construction provides pre-image resistance and unpredictability as long as one of the hash function implementations is not compromised. All libraries are built into Python 3.6.
Fingerprints of public keys are now domain separated with the X25519 shared secret. This prevents deanonymization of clients by cross-correlating public key delivered over the network (e.g. when no OTR is used or when it's defeated by compromising NH) with fingerprint compared over an off-band channel that may not be anonymous and/or encrypted.

Hardware design

Major switch in NH and data diode design takes advantage of the full-duplex capability of serial interface meaning NH will only need one interface, but data diode designs will need to change accordingly. This lowers the price of endpoint setup by ~$15..25. It also streamlines NH-side code a great deal as autoconfiguration of interface directions is no longer needed.
New data diode design by Sancho_P uses USB-to-TTL adapters and Avago HCPL-7723 optocouplers that raise speed from 19 200 bauds to 1-2 million bauds, requires fewer components, very small amount of soldering and no batteries or external power supplies. Kudos!

Stability

As requested, any amount of errors Reed-Solomon corrects now raise a warning about possible serial/data diode failure so users can detect and prevent problems before they become unrecoverable.
Sent group messages now include the Unix timestamp of the time assembly packet creation started. This is a simple yet robust replacement to the previous approach where a counter was used to determine when all copies of group message had been received locally. Any unreceived packet would have caused the counter to desynchronize creating issues in displaying the sent message on RxM. From now on, RxM will display the first message it receives as the one sent to the group and shows only one of those messages. The timestamp is not secret and since it's global, the time zone of the sender is not revealed.
Fixed bad screen reset design where if window change packet was dropped during transmission, the cue for the active window on TxM would indicate that new window will get reset but in fact, it would be the old one still active on RxM. Reset command now includes the UID of the window which is to be reset.
NH and RxM now run a separate receiver process that does not handle the Reed-Solomon decoding. Serial interface's timeout was removed and inter-byte timeout now operates on external clock based on Python3's time.monotonic() that ignores clock jumps. This allows much more reliable reception of data with very tight inter-packet delays (that for the first timescale for very small baud rate settings such as 50..1200 that highest assurance data diodes based on LED-phototransistors / optoforks require).

Usability

New VT100 control-code based user interface is much more intuitive to use.
Switched to Signal style base-10 fingerprints: The main issue with previously used hexadecimal format was that '3', 'b', 'c', 'd' and 'e' are pronounced very similarly in at least the English language.
Removed manual public key import feature: In the event fingerprints do not match it's better users exchange PSKs than risk MITM in either channel when they are clearly being targeted.
Redesigned message output: TxM now always runs two processes. One reads input from the user, another outputs messages from queues. This means that when trickle connection is disabled, different packets are being output based on their priority: Commands have the highest priority, followed by messages and finally files, that usually takes a long time to transfer. This comes at the cost of losing the multi-line paste mode. Trickle or no-trickle, messages/files for active contact/group are canceled with commands /cm and /cf.
Contacts are now added from an interactive wizard that is launched with the command /add
User can view ongoing Pidgin-side file transmissions from file window with the command /fw
RxM notifications now display to which group window contact sent the message.
Completely new way to send encrypted files with TFC. Use the file export command on TxM: /fe

This encrypts file name and data with XSalsa20-Poly1305, outputs the ciphertext to NH that stores it under a name that consists of 32 random hex chars. The exported file can be sent to contact over the channel of user's choosing (email, cloud etc). The decryption key will be displayed on the screen, and it must be sent over TFC manually to contacts receiving the file.

The recipient that receives the file with their NH, can issue file import command from TxM to NH: /fi

This opens a file prompt on NH that allows the user to select the encrypted file. NH will transmit the file to RxM that prompts the user to enter the decryption key, and after successful authentication and decryption, the file is stored on RxM. This obviously does not hide the fact that the user is sending a file, but sometimes that is not a problem. The reason this is faster is that the file doesn't have to be encrypted in chunks, but also because there is no key rotation, database storage or similar. It also handles errors much better because sender and receiver can redo the process independently as many times with varying serial port speeds and error correction settings until ciphertext delivery over data diode succeeds.
/names command now lists groups below contacts with wrapping member list. This deprecates following commands: /groups /group
Added commands that the user to specify a new master password for TxM or RxM from TxM: /passwd tx /passwd rx
User can now view/export log of (n latest) messages of the active recipient/group with commands /history (n) /export (n)
Settings are now stored in an encrypted database. The user can control settings on TxM/RxM with /set

Some settings only change when TFC is restarted; User is notified of such events. The user must ensure all computers receive new settings by making sure they display notifications, before restarting programs. Otherwise, the user must change settings back so that NH and/or RxM can again understand sender device. The major issue is that due to encrypted settings database if the user enables/disables USB adapter and does not have integrated or USB adapter available, they would be unable to send data to RxM. This is prevented by making RxM ask the user to select between USB adapter and integrated interface at the beginning of every session. It is annoying but cannot be helped. Advanced users can comment out prompt trigger from
```
src/common/db_settings.py (lines 108-110).
```
Improved /help readability on narrow terminals.
File data is now encoded using Ascii85 instead of Base64, meaning 5% increase in file transmission speed.
Group management messages now display nicks of known contacts and accounts of unknown ones.
Group management commands are displayed on the active window after which they move into the command window.
Nicknames are now unique identifiers and can be used during group commands when removing contact etc.
Removed noise packet logging. Logging excessive amounts of noise data just to hide metadata about how much communication has taken place in total from an adversary who physically compromises TxM is unlikely to offer meaningful protection. It is much more safe to not log any data in the first place. Smaller log file size makes TFC more portable and easier to hide.

Installing

Installer is now a minimalistic shell script

Type of installation is selected with an argument (tcb, nhu, nht or lt)

The installer is run with one of the following one-liners that make authentication of the installer with known GPG fingerprint easy. It also minimizes the race condition that would follow from manual installer authentication.

TxM/RxM on Ubuntu/Mint (64-bit only)

gpg --keyserver pgp.mit.edu --recv 861BD0CE1E47A9D29F03C4F54064F05A4D17DE97 && wget https://raw.githubusercontent.com/maqp/tfc/master/install.sh{,.asc} -q && gpg --verify install.sh{.asc,} && bash install.sh tcb

NH on Ubuntu/Mint (64-bit only)

gpg --keyserver pgp.mit.edu --recv 861BD0CE1E47A9D29F03C4F54064F05A4D17DE97 && wget https://raw.githubusercontent.com/maqp/tfc/master/install.sh{,.asc} -q && gpg --verify install.sh{.asc,} && bash install.sh nhu

NH on Tails

gpg --keyserver pgp.mit.edu --recv 861BD0CE1E47A9D29F03C4F54064F05A4D17DE97 && wget https://raw.githubusercontent.com/maqp/tfc/master/install.sh{,.asc} -q && gpg --verify install.sh{.asc,} && bash install.sh nht

Local testing on Ubuntu/Mint (64-bit only)

gpg --keyserver pgp.mit.edu --recv 861BD0CE1E47A9D29F03C4F54064F05A4D17DE97 && wget https://raw.githubusercontent.com/maqp/tfc/master/install.sh{,.asc} -q && gpg --verify install.sh{.asc,} && bash install.sh lt

The trust chain works as follows: If the fingerprint in one-liner matches the one you know to be secure, imported 4096-bit RSA public signature verification key will also be authentic. With the authentic public key, only the installer install.sh with authentic digital signature is executed by the one-liner. The install.sh downloads TFC files and checks their SHA256 hashes. Among these files are the requirements.txt and requirements-nh.txt that contain pinned SHA512 hashes for dependencies downloaded over pip.

Installer creates virtual environments for Python3.6 (venv_tfc) and 3.5 (venv_nh) pip dependencies. This avoids littering the global Python environment.

Databases

Databases pad strings with Unicode chars and encode the padded string using UTF-32. This ensures constant length fields regardless of used symbols. Each data type is converted to constant length byte string in the most effective way that still has zero impact on security. This was actually done already in the previous version but the implementation was harder to evaluate as Python2 does not make a clear distinction between bytes and (Unicode) strings. The encoding in the previous version was UTF-8, thus ciphertext length would vary when some field contained Unicode chars that are encoded to longer strings.
Split keys and hash ratchets into a separate keys-database managed by the sender process. This ensures database i/o of input process does not block sender process and reveal use during trickle connection.
Moved logging of packets to separate process. That way i/o wait times when accessing the TxM logfile do not slow down sender_loop and reveal use during trickle connection.
TxM side database now also stores file reception and notification settings that are displayed with /names.

TFC 0.16.10 | Update log

Common

From this version onwards, TFC-NaCl is the only version that gets new feature updates. The reason for this is in-part time constraints, in-part the goal to unify the userbase and ultimately integrate cascading ciphers. The 'NaCl' name will be dropped starting with this version.

Security

Encrypted databases and login screen

Tx.py and Rx.py now feature an animated login screen built on curses, mocking all the things cyber. When the programs are launched for the first time, the user enters and confirms a master password. This password will be combined with salt (created with /dev/urandom) in PBKDF2-HMAC-SHA256, the iteration count of which is ensured to be within 2000..4000ms range, but with at least 65536 iterations. This means slow systems will suffer during login time; Security comes first. While the salt and long key derivation time are currently best practices, they offer negligible protection against weak passwords. No magical crypto dust exists to protect from this so make sure to use strong passphrases and possibly 2FA with something like Yubikey, that remembers a static part of the passphrase.

The output of HKDF is the master key, the SHA3-256 hash of which is stored together with the iteration count and salt to file .{tx,rx}_login_data. Forgetting the password, tampering with, or deleting the login data file renders all stored data useless: be careful. Under the assumption, the password remains unbreakable by the adversary, encrypted databases help against following problems:

Wear leveling:

All data is encrypted before it touches the hard drive: These are increasingly becoming SSDs that don't overwrite data the way magnetic drives do.

Physical data exfiltration:

In the case where the adversary has not remotely exploited RxM with a keylogger that copies TFC master password, physical access where encrypted data is merely exfiltrated means attacker is unable to access sensitive data. Previously similar protection could be obtained with Full Disk Encryption (FDE) and ensuring the encrypted disk is not mounted. TFC now keeps data secure even if FDE drive is mounted but TFC is not running. Since encryption scheme is authenticated (XSalsa20-Poly1305), any tampering can be detected as authentication of encrypted data fails with the correct password.

Impersonation:

In situations where TxM and RxM operating systems are left open (something user should never do) but when TFC master password hasn't been given, encryption of data prevents impersonating the user.

File metadata:

All fields in databases are padded prior to encryption. With groups and contacts, there is now a maximum number of contacts adjustable by the user. The database is padded accordingly to hide the length of data in fields, number of contacts, groups, and members of each group.

Encrypted PSK:s

When the user creates a new PSK, in addition to contact details, Tx.py generates a 256-bit salt and prompts the user to enter and confirm a password that protects the PSK during transit. An encryption key is derived from the password and salt using PBKDF2-HMAC-SHA256 over 65536 iterations, and PSK is encrypted with that key using XSalsa20-Poly1305. The salt, nonce, ciphertext, and tag are written to PSK file.

The recipient who adds PSK to their RxM now uses command /psk that replaces the old command /rxkey. Rx.py then parses salt, nonce, ct, and tag from PSK file, and then asks the user for the password to decrypt the PSK. If the user enters the correct password, the key and contact details are added to RxM, keyfile is shredded and the user is advised to physically destroy the transmission media.

Delivery of PSK password should be split over multiple media.

Encrypted SSH password

Tx.py now prompts the user to enter and confirm the RPi SSH password the first time the connection is established. The password length is protected by padding it to the next 255 chars before it's the master key using XSalsa20-Poly1305. The resulting ciphertext is stored into file .ssh_login_data.

Group trickle connection

The user can now select a group for trickle connection (must be created before trickle connection is enabled from Tx.py configuration).

Multi-account support for user

When the user adds a contact, they must now specify account of the user, associated with the account of the contact. When combined with Tails configuration for NH and Tor-hidden service XMPP server, this can effectively hide the metadata about group structure. For example, Alice could communicate to Bob and Charlie using separate accounts for both, and Bob and Charlie could do the same for the three. In such case, the contact list itself does not leak metadata about the structure of communicating parties' network.

All commands are now encrypted

The last unencrypted command (screen clearing) is now encrypted to prevent DOS of RxM in cases where NH is compromised. NH side screens are cleared with additional unencrypted command when trickle connection is not enabled.

Shell escaping

Enabled shell escaping for all shell-commands with variables. All inputs for shell commands come from the user by design and are thus trusted. Enabling it, however, prevents shell-injection of RxM by typing commands to Tx.py. This is unlikely an issue as any physical attacker can type commands on RxM screen.

Reset command

Added command /reset that resets Tx.py, Rx.py, and NH.py terminals. Reset also clears the ephemeral message history of session for active contact/group. All messages for that conversation that are not logged are lost for good; Use /clear for temporary screen clearing when you need a clean table or if you detect non-threatening shoulder surfing and /reset when the conversation needs to be removed.

Encrypted hash ratchet counter

The hash ratchet counter is no longer an unauthenticated plaintext counter, but an XSalsa20-Poly1305 encrypted, eight-byte value that is prepended to packet. This removes metadata datagrams in IM client without OTR-protocol would otherwise reveal about the number of sent packets, and removes the DoS attack vector in RxM from third parties (TFC is still vulnerable against frenemies, but they can be removed altogether from TFC.)

Usability

Chat 'windows'

Rx.py finally features separate windows or tabs for each conversation, including one for issued commands. If a message is received into an inactive window, either the message or a notification about the message is temporarily displayed. The duration of the notifications can be controlled with the setting new_msg_notify_dur. Information about commands is also temporarily displayed on the screen before moved into the command window. This helps keep conversations free from clutter.

Some warnings and notifications are not stored in the temporary message log, and they disappear when the user changes window by selecting same or another conversation with /msg command, or by opening the command window with /cmd or //.

The user can check if they have messages on unread tabs by typing either command /unread or (single space).

Shorter public keys

Moved from 72 hex char public keys to Bitcoin Wallet Import Format where keys are Base58 encoded and come with integrated SHA256 based checksum. Base58 encoding reduces the length to 48-50 chars that do not look like one another (unlike in the case of Base64).

New group messaging

Group creation

When Alice creates a group, her Tx.py asks her if she wants to send an invitation to group members. The invite will contain a list of members she added to the group, and it is displayed by Rx.py of invited members. Joining users must manually create a group with the same name using their TxM, that will share the group with RxM.

If Bob creates the same group and sends a list of members he added to Alice and Charlie, Alice will get a notification "Bob has joined the group", while Charlie (who Alice did not add) will get an invitation to group. Alice and Charlie will see that Bob has added the other to the group, so they know to add each other as contacts and later, to the group.

Adding members to the group

When Alice adds new members to the group, her Tx.py asks if she wants to send a list of added members to existing members, so they can synchronize their side of the group. Members added to the group will receive the same invitation that members who are invited to the new group get.

Removing members from the group

When Alice removes Charlie and David from her group, her Tx.py asks if she wants to send the list of removed members to remaining members. That way, the remaining members are aware that she is no longer receiving or sending messages to these contacts. Removed members never receive notification about being removed. The only thing that happens is, Alice's Tx.py stops sending messages to removed contacts, and her Rx.py stops showing messages from these members directed to the group's conversation window. Removed users can still exchange private messages with Alice. To block this, Alice should block the contacts in IM client.

Leaving the group

When Alice removes the group, Tx.py asks if she wants to send members of the group a notification that she's no longer sending messages to group or receiving messages from them. If she sends the notification, Rx.py of receiving contacts also warns the contacts, that unless they themselves remove Alice from their group, their Tx.py will keep sending the group messages to Alice, who could regardless of group exit message, still be decrypting and reading the group messages. All Alice needs to do is join back to the group, and not send a notification about it.

Preventing the user from doing this is comparable to sender based control, something that is not just snake oil, but impossible to achieve with free and open source software. The only way to prevent the user from receiving and decrypting messages from contacts when the parties by design share keys, is not to send messages to the user in the first place.

The user can now enable logging and file reception for groups with commands

/logging on <group name>
/store on <group name>

By default, the latter allows everyone in the group to send a single file to the user. Automatic disabling of file reception can be disabled by setting auto_disable_store to False.

Significantly improved local testing

Local testing uses now uses Terminator, that is installed alongside TFC. Tx.py, Rx.py, NH.py (and dd.py) are launched from one of two desktop entries: TFC (Standard TFC local testing layout) TFC DD (TFC local testing layout with data diode simulators)

More sequenced key exchanges

Rx.py now displays public keys after the local key has been delivered to RxM. Screens are cleared for each step to make them easier to follow.

Fallback for Tkinter file prompts

When Tx.py/Rx.py is unable to show Tkinter dialogs (for example, when Tx.py is used as blacker over SSH from a local network that sits behind TxM-NH data diode), Tx.py no longer demands the use of default directories, but asks the user to type the desired path over CLI (tab-complete is supported).

Improved /names command

/names command no longer shows ID, but instead, account, nick, key exchange type (Curve25519 or PSK) and whether logging is enabled or disabled.

Easier contact choosing

The user can now directly select group from select contact command or use nick instead of account to select a contact. Tab complete is supported.

Program design

Support for arbitrary length commands

This means better synchronization with TxM and RxM data in the future. It works even with trickle connection.

Changed naming of variables

Most of the variables in configuration were changed to more descriptive. New static length naming is easier on eyes.

Improved user interface

TxM/RxM/NH configurations for *buntu configurations now feature desktop entries and Terminator profiles. Each installation features their own layout and profile. Previously existing profiles will not be overwritten.

Switched from os.system("clear") that caused issues with Ubuntu 16 scrollback to VT100 control codes that remove the problem.

VT100 control codes are now also used to remove repeating questions (removes clutter), to move key input guides below text, to auto-complete previous lines (nick, key exchange choices, y/n answers etc). Notifications about file transfer and random delays when long_packet_rand_d is enabled, also have much cleaner notifications.

Changed naming policy and functionality of database files

Group files: Group files have been removed. Instead, all groups and values in fields that contain a list of members and a log setting for the group are stored in a fully padded format to file .tx_groups or .rx_groups.

Keyfiles: The "Keys" folder and aside PSK delivered to contact, all keyfiles have been removed. Keys are instead stored in the contact database.

Logfiles: The "Logs" folder and log files have been removed. Instead, all messages for all contacts are stored in a padded state into the file .tx_logs or .rx_logs. The logged data is not the assembled data, but the plaintext form of each assembly packet. This prevents metadata leaks from observing the ciphertext. Messages are parsed from assembly packets when the log file is viewed with the command /history or exported with the command /export (requires confirmation from the user).

Database files: The database file is like group file, each field is padded separately to hide contact specific plaintext length. The contact database is padded with dummy contacts that contain dummy data. This protects from an adversary who gains physical access, from deducing metadata about the user. The database holds the following information: -account of contact -account of the user (TxM only) -nickname of user -key for sent messages -key for received messages (RxM only) -hash ratchet header key for sent messages -hash ratchet header key for received messages (RxM only) -hash ratchet counter for sent messages -hash ratchet counter for received messages (RxM only) -logging setting for contact -file reception setting for contact (RxM only) -window notification privacy setting for contact (RxM only)

The sampler program hwrng-nacl.py was renamed to hwrng.py.

Significantly improved and refactored code

New input_validation function that gracefully exits and gives more details: Example:

    ERROR! Second parameter of function 'g_mgmt_print()':
    Got <type 'list'> instead of <type 'str'>.

The code for group management was improved significantly by changing For-loops into set operations among other things (Tx.py and Rx.py).

Simplified data type carrying in Tx.py.

File packet loading and delivery time evaluation was completely rewritten.

Plaintext data fields are now separated using non-printable Unit Separator character (0x1F). This reduces overhead and greatly clarifies structure.

Account name string on Rx.py no longer carries trailing .ld / .rx, but separate origin parameter (u for user and c for contact) is passed to each function.

Removed need for exit queue by adding a periodical check if the child processes are alive, and exiting gracefully if they are not.

Added even more unittests. Running unittests now warn they remove all TFC user-data, and ask for confirmation before running.

Improved serial interfacing

Transparent serial auto-configuration

Removed need for interface configuration packets sent by Tx.py during launch. NH.py now uses any packet to configure serial interface directions before forwarding it to Pidgin and/or RxM.

Programs wait for adapters

During start, Tx.py, Rx.py, and NH.py wait until missing USB-to-RS232 adapters have been connected before proceeding.

Adapter disconnection handling

Tx.py, Rx.py, and NH.py no longer crash when USB-to-serial adapter is removed. Tx.py won't send further packets until the adapter is reinserted. Rx.py won't read packets until adapter has been reinserted. NH.py won't send packets to NH until both interfaces have been reinserted and the interface for TxM has been determined by any packet from TxM. All three programs can remap interfaces regardless of device name assigned to it by the OS (/dev/ttyUSB#, where # changes unpredictably). To ensure the mapping works, make sure there aren't additional USB-to-Serial adapters connected to any of the computers TFC uses.

Reactive crossover resolving

In the situation where cables are crossed over, e.g. when the user changes the data diodes to opposing serial interfaces on NH (something the unidirectional interfaces can't detect), NH assumes it's outputting to RxM, while in fact, it's sending messages to TxM. In such situations packets will be lost until NH.py learns about the crossover by receiving a packet from TxM; NH.py listens to both interfaces for TxM data, and it automatically changes the port to RxM to the other interface, resolving the issue.

An exception to packet loss is during start when NH.py doesn't yet know which interface connects to TxM. The process that outputs messages to RxM, therefore, waits until user sends a packet from TxM. When that happens, NH.py maps the interfaces, and the process then outputs sent and received packets from the queue to RxM in order.

Adjustable forward Error Correction with Reed-Solomon erasure codes

Depending on the quality of serial interfaces and data diodes, transmission errors might occur occasionally. TFC now uses Reed-Solomon error correction to detect and fix errors and missing data. A simple setting e_correction_ratio is provided to users. This number describes the maximum number of byte errors in transmission that can be recovered from.

Faster default baud rate

The reference data diode design appears to remain stable with baud rates of 19200, this doubles the speed.

Raspbian Jessie -style serial interfacing.

Changed the integrated serial interface of Raspbian to /dev/serial0.

NOTE: While this makes it possible to run TFC on Raspberry Pi 3, these devices have wireless network interface embedded within the processor. This makes Raspberry Pi 3 impossible to physically air-gap, thus using it is NOT secure.

Program specific updates

Tx.py

Improved sampling

Tx.py now shows the progress of key generation over SSH.

TFC no longer loads multiple 256-bit keys but instead specifies the amount of entropy that is required (varies between 256 and 768 bits). This speeds up key generation, and multiple SSH connections are no longer necessary.

Account validation

Tx.py now checks accounts against regular expressions. This is generally bad practice with emails and XMPP accounts, so the rule set is just

<something>@<something>.<something>

This check ensures the user does not use reserved account name for the dummy accounts that are used as padding in the database.

Streamlined NH bypass

Tx.py no longer asks for NH bypass during key exchange. This feature is mainly needed for local key delivery where remote exploitation of NH can be combined with a keylogger or visual collection. If root local key is compromised, all keys can be accessed from NH. Therefore, it's still useful to send encrypted local key directly from TxM to RxM. NH bypass messages now halt Tx.py only twice during the entire procedure (second time after confirmation code has been typed).

Improved file transmission

Tx.py no longer encodes sent file to temp file before reading it, but reads data into memory and converts it there. This helps when sending large files, but since file transfer is already slow due to design limitations, it won't do much good. Tx.py now uses zlib to compress (maximum compression setting) the file prior to encoding. This also saves some time depending on the entropy of sent data.

Tx.py no longer appends hash of file data, but prior to encoding generates a 256-bit key from /dev/urandom, that will be used to encrypt the file data. Encrypted data is then concatenated with the key. The benefit is, the decryption key will be sent to the recipient in the one or two last packets, meaning if the user e.g. cancels file transmission to a wrong recipient, the file can not be decrypted. Long messages are also encrypted with additional layer and trailing key.

The file packet estimation counter was completely rewritten and is now much more accurate. Delivery time estimation now scales up to 29d 23h 59m 59s.

Tx.py also shows more detailed information about the preprocessing of sent file e.g. loading, compression, encryption and delivery time evaluation).

Tx.py no longer allows sending files with /file /path/to/file. Instead, the file is selection is opened with the command /file, after which either GUI prompt (or if that's unavailable or disabled by setting disable_gui_dialog to False, tab-complete supported CLI prompt) is displayed.

Simplified contact adding

New contacts are from now on always added with the command /add. Depending on which parameters user adds, Tx.py will interactively ask other ones:

/add alice@jabber.org bob@jabber.org alice ecdhe*    asks to use HWRNG (if avail.)
/add alice@jabber.org bob@jabber.org alice           also asks key exchange
/add alice@jabber.org bob@jabber.org                 also asks nickname
/add alice@jabber.org                                also asks user's account name
/add                                                 also asks contact's account name

Please note that you can't skip parameters!

Giving command

/add alice@jabber.org bob@jabber.org ecdhe

assumes Bob wants to set nick as ecdhe, and asks for the key exchange

*Key exchange can be

e or ecdhe for Curve25519 elliptic curve Diffie-Hellman
p or psk for pre-shared XSalsa20-Poly1305 keys.

Fixes

Fixed issue where long messages sent by Tx.py were not logged.
Fixed issue where Tx.py did not log any messages during trickle connection.
Fixed issue where the user can set contact's nickname as contact's account or as a name of a group during contact creation.
Fixed issue where canceling long msg/file transfer to a group doesn't cancel transmission to rest of contacts.
Added graceful exit for SSHError when IP is the same as TxM.
Added graceful exit for when hwrng.py is missing from Raspberry Pi's home directory.

NH.py

Support for multiple accounts

NH can now send messages from multiple Pidgin accounts to contacts.

Fixes

NH.py no longer crashes if Pidgin is not started first. Instead, it waits until DBus connection to enabled account in Pidgin is established, and then proceeds with startup. DBus has a timeout, however.
Fixed crash when NH.py is started while Pidgin is running but no account is logged in.
Fixed issues with error printing.
Fixed issue where no serial-interface is available for Raspberry PiNH.
Fixed crash when Tx.py sends clear screen command to NH.py with no account selected. An example of such situation is when an empty group is selected.
Moved to single startup animation based on login screen's curses animation. Animation scales according to terminal size and can be skipped with CTRL+C.

Rx.py

Much cleaner message printing

Messages for contact are now dynamically wrapped to screen according to the window size. Nicknames are dynamically updated, and date changes are displayed between messages. Messages are indented in front of the time stamp. Together these make a huge difference to usability compared to previous versions:

Moved from confusing messaging layout

10:20 Me > Alice: Hi Alice
10:21      Alice: Hi Bob!
Changed alice@jabber.org nick to alice.

to

10:20    Me: Hey Alice

10:21 Alice: Hey Bob!

10:22    -!- Changed alice@jabbim.pl nick to alice.   // this message is displayed only temporarily

that's easier on eyes as recipient's nick isn't always next to the message. The previous style was required, as all messages sent to all contacts were displayed in the same window, and the user had to be able to tell to whom their message was delivered to. Me is no longer allowed as a nick: this is to prevent confusion about the author of the message. RxM-side log files use the same layout.

The messages displayed are now word-wrapped and long messages have extra room between prints.

Better public key display

When the user initiates new ECDHE key exchange, Rx.py displays the latest public key received from contact during the session. The user no longer needs to scroll RxM terminal up to find the public key.

Local key backlog clearing

Rx.py now resets the terminal when the local key encryption key is correctly input. This removes backlogs with the intention of hiding the sensitive key decryption key. If local testing is enabled and the key was pasted to Rx.py from Tx.py, Rx.py clears the clipboard.

Fixes

Fixed issue where maliciously crafted invalid file header would crash Rx.py.
Fixed issues with nickname indentation with variable length encoding.

hwrng.py

Renamed hwrng-nacl.py back to hwrng.py as CEV/OTP versions use their own file names, and as CEV/OTP versions are slowly removed/integrated.

Key size in parameter

hwrng.py allows three key sizes to be loaded from GPIO-HWRNG: 256, 512 and 768 bits.

setup.py

Smarter configuration chooser

setup.py now detects if OS is Raspbian or Ubuntu derivative and allows the user to choose between four configurations. Unsupported OSs exit setup.

Selection #	Raspbian	Ubuntu
1	TxM	TxM
2	RxM	RxM
3	NH	NH
4	HWRNG	Local testing

setup.py will also detect Tails installation configuration and initialize the NH configuration setup automatically. Note that Tails installation configuration is not run as sudo.

Automatic signature verification key and signature download

When setup.py is run, the TFC public key and the digital signature for setup.py are downloaded automatically. It is up to the user to choose whether to verify the signature first or after installation by importing the key and running

gpg --verify setup.py.asc setup.py

The user shouldn't trust software downloaded from the network is actually downloading the correct public key. The only way to trust the signature verification key is by having prior knowledge of the authentic fingerprint (never use the short version) from the authentic source, e.g. from face to face meeting with the developer or via trustworthy Web-of-Trust.

Moved from apt-get to apt

Apt shows download progress more clearly.

Shuffled dependency downloads

Apt-commands and library downloads are now shuffled with CSPRNG when selecting TxM/RxM configurations. As the window of opportunity to remotely exploit TxM starts the second user downloads setup.py, this only delays detection of installation of those users who have obtained setup.py via sneakernet or different Tor session, where they weren't deanonymized.

TxM/RxM/HWRNG configurations for TFC can now be installed with single line:

TxM:

sudo echo && wget https://raw.githubusercontent.com/maqp/tfc/master/setup.py && sudo python setup.py 1

RxM:

sudo echo && wget https://raw.githubusercontent.com/maqp/tfc/master/setup.py && sudo python setup.py 2

NH:

sudo echo && wget https://raw.githubusercontent.com/maqp/tfc/master/setup.py && sudo python setup.py 3

HWRNG (Raspbian) or Local testing (Ubuntu)

sudo echo && wget https://raw.githubusercontent.com/maqp/tfc/master/setup.py && sudo python setup.py 4

The reason sudo echo is run is to not waste time writing sudo password after setup.py has been downloaded. While sudo wget would be equally fast, running wget as root is not a good idea. setup.py should be run as sudo because that way it has the permission to run sudo ifconfig # down for all network interfaces. Ethernet cable should be disconnected once installer tells the user to do so. All wireless network cards must be removed before TFC is installed on TxM/RxM.

HWRNG installation configuration now automatically sets RPi static IP 192.168.1.2.

TxM configuration for *buntu asks the user if they will connect to RPi HWRNG over SSH. If the user does this, setup.py will wait until at least one ethernet device is connected, and then asks the user to enter the one to be used (tab-complete supported). It then enables static IP 192.168.1.3 for TxM and restarts the network interface.

Smaller changes

Setup.py now enables execution permissions on .py files, so they can be run with

./<program name>.py

Setup.py now scales (if supported by the terminal) the terminal to fit all configurations in the main menu.

TFC NaCl 0.16.05 | Update log

Common changes

Removed from x import y style imports to avoid conflicts in namespace etc.
Refactored code here and there.
Added more unittests. test_tx.py and test_rx.py now evaluate XSalsa20-Poly1305 implementation in PyNaCl using official test vectors by djb. Unittests can now be run from RPi over SSH.
Improved CLI here and there.
Banner can now be skipped with Ctrl+C

Tx.py

User can now send an encrypted command to RxM with the command /rxkey that opens a file prompt. This allows the user to load the PSK for the contact without turning off Rx.py.
Added setting show_file_prompts that by default opens a storage prompt where the user can store contact's PSK. When false, keys are stored in keys_to_contact.
Resolved the CTR mode needs counter parameter, not IV issue with Paramiko's SSH connection with AES CTR mode.
Disallowed : and / chars in accounts to fix crashes.
Reversed PSK and ECDHE key choice answer to more intuitive order.
Fixed issues when sending dotfiles.
Changed trickle delay's time's constant delay time exceeding from critical error to a soft warning. Raspberry Pi tends to send the first packet with greater delay. The user will now be prompted to increase trickle_c_delay if the problem persists.

Rx.py

Opens a file prompt when /rxkey command is sent by TxM. This allows the user to load the PSK for the contact without turning off Rx.py.

Fixed issue where missing rx.<account>.e key on RxM crashes Rx.py when /logging on <account> command is issued from TxM.

Opens file prompts based on keyfile change commands.

setup.py

Added preinstalled hashes of tested (not audited) crypto libraries.

Improved SSH configuration during installation.

Added NH installation configuration.

Reduced the window of compromise time to minimum: setup.py now first downloads apt-packets. It then downloads and verifies crypto libraries and TFC. Finally, the installer runs sudo ifconfig down on every interface excluding lo, listed by ifconfig -a, before extracting and building libraries.

Fixed issues with file permissions when running the installer as superuser.

hwrng-nacl.py

Renamed hwrng.py to hwrng-nacl.py in case multiple versions of TFC (CEV/OTP) use same RPi to load entropy from HWRNG.

dd.py

dd.py now automatically re-sizes terminal window to a proper size.

TFC NaCl 0.16.01 | Update log

Local key transmission: Bootstrapping security

In the initial bootstrap, Tx.py generates a 256-bit local key and 256-bit key encryption keys with separate processes, where 32-byte /dev/urandom string is hashed with SHA3-256 (simplesha3 library), that is then passed through 25k iteration PBKDF2-HMAC-SHA256. If Tx.py is run on a Raspbian (Raspberry Pi), the user can mix entropy from free hardware design HWRNG, connected to GPIO 4 of the SoC board into the PBKDF2 function. Entropy is sampled with 10Hz frequency and Von Neumann whitened during sampling. Once the Von Neumann samples have been collected, the 256-bit entropy is compressed using the SHA3-256 hash function. If Tx.py is run on a computer that has a direct ethernet connection to Raspberry Pi with GPIO HWRNG, Tx.py can SSH into that computer using Paramiko library, and query entropy with program hwrng.py.

The local key is padded to 254 bytes and encrypted with the key encryption key using PyCrypto library's XSalsa20 stream cipher (192-bit nonce), and 128-bit Poly1305 MAC. The signed ciphertext is transmitted to RxM through data diode. The user can choose whether to leave data diodes connected to NH and let signed ciphertext pass through, or whether they want to transmit keys directly from TxM to RxM by connecting serial interfaces that way. Such configuration prevents the adversary from obtaining the signed ciphertext from NH in case it has been compromised at this point.

RxM will then ask the user to manually type the 64 hex char key decryption key displayed by TxM. The key decryption key is concatenated with 8 hex char checksum, that is the truncated SHA3-256 hash of the key decryption key. Once the 72 char hex key has been typed to Rx.py, the program decrypts the local key, adds it to the database, and shows a two char hex code that lets TxM know local key delivery succeeded. This is a pre-requisite for the next step, where account data/keys are transmitted to RxM, encrypted with the local key.

If RxM does not receive the encrypted local key, the user can send the packet again by typing "replay" instead of the device code. The local key transmission can later be redone with command /localkey.

First contact

The user is guided when adding the first contact. Tx.py first prompts for an account, e.g. alice@jabber.org and a nick. Empty nick uses nick Alice (automatically parsed from the account). Tx.py asks the user to confirm account and nick (choosing no jumps to the beginning of account creation). After this Tx.py prompts the user to choose key exchange method:

Use PSK instead of ECDHE (y/n)?

PSK is explained later so here we choose n (=no) to start ECDHE key exchange.

ECDHE Private key generation

Tx.py generates a Curve25519 ECDHE private key by with PyNaCl's PrivateKey.generate(), that has been modified to XOR in either bitstring of 256 zeroes, or if provided, 256 bits string of external entropy. Tx.py always provides the external entropy:

ext_ent = PBKDF2-HMAC-SHA256(SHA3-256(32-bytes from /dev/urandom, salt)).

If Raspbian with HWRNG is available, Tx.py will add the salt to PBKDF2-mix:

salt = SHA3-256( Von Neumann whitening (HWRNG samples))

The number of samples is dynamic and depends on results. This ensures SHA3 always takes in 256 Von Neumann whitened samples. If HWRNG produces bad entropy i.e. a chunk of just zeroes/ones, they are not added as samples.

PyNaCl's PrivateKey.generate()'s own entropy is loaded after external entropy is provided, thus even if external entropy adds zero entropy, it does not weaken the overall security, as in such case it would be the equal of XORing PyNaCl's entropy with bitstring of zeroes.

Once the final entropy has been generated, PyNaCl will use it to create a private key object.

ECDHE Public key delivery and verification of key authenticity

Tx.py generates the ECDHE public key from the private key object and transmits it to the recipient's RxM. Once the 64 hex char public key of contact has been received by Rx.py the program shows it concatenated with an eight char hex checksum like the one in the local key. The user must manually type the 72 hex char string to TxM. The manual process ensures no automated data channel from the network can infect TxM after setup.

Once the key has been successfully entered, Tx.py will ask the user to use Signal by Open Whisper Systems to verify the authenticity of contact's public key.

MITM against OTR session is possible if the adversary has exfiltrated respective keys from endpoints of users. If users find out during Signal call that the public keys have been generated by a man in the middle, they do not have to throw in the towel. Instead, users can read the public keys to each other over Signal, manually type in the new public key and effectively bypass the MITM in the network. Once the public key has been successfully typed, Tx.py will log the public keys for later, higher assurance verification is done face to face. This also enables retrospective detection of MITM, if for some reason users are unable to verify the authenticity of public keys during key exchange.

Shared secret derivation

TxM is the only device that receives no automatically input data after setup, so it is the only device trusted to generate secret keys. Tx.py will generate two symmetric keys (one for encryption, one for decryption) by concatenating ECDHE shared secret with one of the public keys and passing them through PBKDF2-HMAC-SHA256 (25 000 iterations). This is mandatory for the forward secrecy to work. Forward secrecy is obtained by passing the encryption keys through PBKDF2-HMAC-SHA256 (1 000 iterations) between every message. Were the ECDHE shared secret used as the symmetric key for communication in both directions, offset in key derivation could lead to decryption of collected ciphertexts by iterating physically compromised key, that lacks behind in PBKDF2 iterations.

Once the symmetric keys are generated, the account details are encrypted and signed with XSalsa20-Poly1305 using the local key and transmitted to RxM, where the contact is automatically added. While this packet doesn't differ in any way from encrypted command, the user can still choose to prevent NH from receiving the ciphertext by plugging the TxM's data diode directly to RxM once the public key of contact has been received to RxM. During this period any message sent by contacts is not received. Dropped packets do not cause de-synchronization of keys, as each packet contains the number of times Tx.py has iterated the initial symmetric encryption key with PBKDF2. This means both receiving devices can catch up with the Tx.py's key. If Rx.py detects dropped packets, it will display the catch-up progress. KeyID introduces a DoS attack vector where packets with great keyID block Rx.py from operating. This would, however, require a MITM attack against the OTR, or messing with the NH; Such DoS would blow the cover of the high-strength attacker. A more covert DoS can be mitigated from within the network or IM server, so this attack is an unlikely problem. Were this evaluation incorrect, an efficient fix would be to use a separate static MAC key that signs the keyID. For now, it is not implemented. It should be mentioned that this DoS attack can also be mitigated by frenemies, but they are easy to block from the IM client.

Additional accounts can be added using commands

/dh <account>( <nick>)

and

/add <account>( <nick>)

PSK keys

While TFC-NaCl is all about convenient public key crypto with exfiltration secure private keys, it's security will not hold in the long run. Quantum computers are making their way albeit slowly, and in the future, any symmetric key agreed using Curve25519 ECDHE will be broken. The only public key algorithm secure against Shor's quantum algorithm is McEliece with Goppa Codes. Long-term security would require users to type in 1 000 000 bit public keys, which is highly inconvenient regardless of encoding used. As an initial answer to quantum computing, TFC-NaCl retains the possibility to create pre-shared symmetric keys for quantum-secure 256-bit XSalsa20-Poly1305:

Answering yes to PSK question, in the beginning, will create symmetric keys with PBKDF-HMAC-SHA256 (25 000 iterations), using SHA3-256(urandom(32)) as password, and if HWRNG connected to Raspbian (SSH or not) is available, by using SHA3-512(VN(256-bit HWRNG entropy)) as salt.

The contact's account, nick and the symmetric key for local decryption are sent to RxM, encrypted with the local key (again, NH doesn't have to be in between if the user so prefers). Tx.py then prompts the user for his or her account name.

If Alice is adding bob@jabber.orgas contact, a copy of her symmetric key is generated for Bob, and conveniently placed inside folder "PSKs", under the name

rx.alice@jabber.org.e - Give this file to bob@jabber.org

Alice must then take a never-before-used thumb drive, copy the PSK to that and give it to Bob in a face-to-face meeting. Bob must also have generated a PSK for Alice in advance. Once Alice has received the thumb drive from Bob, she plugs it into her RxM (NOT TxM), and copies the

rx.bob@jabber.org.e - Give this file to alice@jabber.org

keyfile to folder "keys". She must then restart Rx.py. Rx.py automatically strips the trailing instruction from the keyfile and adds Bob as a contact. Since keys for encrypting messages to Bob are already installed, once Bob has copied keyfile generated by Alice to his RxM, they are ready to communicate.

To ensure forward secrecy and privacy of messages, both parties MUST ensure the thumb drive given by their contact is physically destroyed immediately after keys have been added.

If contacts have already been generated, new PSKs can be generated with the command

/psk <account>( <nick>)

Other updates over TFC 0.5.5:

Change of version style to 0.16.01 (0.YY.MM).
Fully PEP8 compliant coding style/variable naming.

New local testing IPC

Local testing of TFC with a single computer now uses multiprocessing sockets instead of files. This reduces i/o errors also means fewer files in TFC root directory.

Data diode simulators

Added a program (dd.py) that emulates data diodes on screen. The user needs to enable dd_sockets boolean in Tx.py and NH.py with the argument -d before this can take place. Each dd.py program is launched with a set of arguments so that they know which sockets to use and to what direction data visually flows between terminals. An example set of commands that launches TFC on *buntu is

gnome-terminal --title='TxM' --geometry=100x35+0+630  -x sh -c "python /dir/Tx.py -d -l"
gnome-terminal --title='NH'  --geometry=71x41+920+150 -x sh -c "python /dir/NH.py -d -l"
gnome-terminal --title='RxM' --geometry=100x20+0+0    -x sh -c "python /dir/Rx.py -l"
gnome-terminal --title='dd'  --geometry=25x12+740+630 -x sh -c "python /dir/dd.py txnhlr"
gnome-terminal --title='dd'  --geometry=25x12+740+425 -x sh -c "python /dir/dd.py nhrxlr"

Setting keyboard shortcut or alias for each of these makes local test startup fast.

Serial interface auto-config

User's device configuration is asked during installation. This will configure Tx.py, NH.py and Rx.py to look for either integrated ttyS# (or ttyAMA0in the case of Raspbian) serial interface, or for a USB interface ttyUSB# if the user decides to use those. The serial interface numbering is finally automatically detected, so if the user accidentally pulls out serial interface, the user doesn't have to manually edit the interface if OS remaps /dev/ttyUSB0to /dev/ttyUSB1.

NH interface auto-flip

Removed the need to run NH.py with -f flag to flip interfaces: Tx.py will send a serial interface configuration packet to NH.py during start. In the event NH.py has mapped TxM and RxM in opposite interfaces, initial listener processes detect Tx.py's configuration packet (or any packet for that matter), and map correct interfaces to TxM and RxM.

Organized files to folders

Moved location of group files, keyfiles, received files and logfiles to respective folders. Renamed txc.tfcand rxc.tfc to dotfiles .tx_contacts and .rx_contacts. The only TFC files visible in software root directory are .py files (and syslog.tfconce it's first generated).

Added some metadata about TFC to packet headers

In the future, this prevents interoperation with older versions of TFC. It allows fingerprinting of users (only if NH is remotely exploited or OTR is being MITM attacked / not used) but forces users to keep up with security updates. New protocol uses more simple and mature separation of payload components.

Re-designed trickle connection

input_process() now prepares all data to packets with length in the range [0, 253] and puts the data into message or file queue. The queue is periodically read by sender_process() that opens a thread that pads, encrypts, signs, adds headers and outputs messages. XSalsa20-Poly1305 algorithms are designed to run constant time. The system time in ms is recorded immediately before function starts; Once the function finishes, the timestamp is passed to function trickle_delay() for evaluation. This function will then sleep the difference between Thread runtime and value trickle_c_delay to hide platform speed. Platforms should not introduce problems, as older Raspberry Pi (generation 2), only takes about 400ms to encrypt a message, leaving 1600ms headroom for slower SoC boards.

Changed random sleep to use /dev/urandom instead of Python math.random. The CSPRNG prevents statistical attacks that might be able to calculate the internal state of math.random's Mersenne Twister state to detect whether communication takes place. When print_ct_stats boolean is enabled, Tx.py shows the statistics about how long the message/command output thread ran, how long constant time sleep was added, and how well the constant time matched trickle_c_delay: Average error in constant time delay was 2ms so the CSPRNG based random sleep will hide the slight differences effectively. Tx.py will gracefully exit if thread runtime exceeds trickle_c_delay.

The packet_delay used to prevent congestion / IM server spam with long messages is now also constant time, to prevent slower data transmission with slower devices. The default value is 500ms which should be enough to hide metadata about TxM device performance and make hardware fingerprinting harder. This also makes the prediction of file transfer duration easier.

Improved logging

Logging can now be enabled for individual contacts with command /logging {on,off} <account>. When logging is enabled, in addition to automatic logging of public keys, Tx.py will also log messages sent to contact. This allows the two users to manually audit whether malware has at some point infected RxM device(s) and substituted words in log files, by cross-comparing the clean TxM logfile of Alice and purported RxM logfile of Bob, and vice versa. Rx.py also stores information about dropped packets. Malware that replaces received messages with notification about the dropped packet is harder to detect; offline record keeping might be required.

Tweakable checksums for data diode error detection

Changed CRC32 checksums to truncated SHA-256 hashes (8 hex = 32 bits) so data diode error detection accuracy can be tweaked at will by changing variable checksum_len. The checksum doesn't have any effect on security: it only detects transmission errors inside data diode.

Newly written NH.py

Converted classes to functions to fit with functional programming style. The program is now much more structured and easier to audit. Added processes that exchange data via queues. DBus / purple should no longer output error messages, nor should they drop packets. NH multiprocessing now cleanly exits.

Re-designed file transmission

/file 1.txt still sends the file from TxM directory. Absolute paths can also be defined. Non-TFC filenames in Tx.py directory are now included in the tab-complete list. Command "/file" opens a file dialog for easy file selection. Tx.py will first encode file data to base64 and load it into memory. It'll evaluate and display an approximate amount of packets and time required for file transfer, based on delay settings.

File data is prepended with a header that contains the name, extension, file size and estimated number of packets and delivery time for transfer. This information is displayed to the recipient after the first packet of the file has been received. File reception must be enabled.

The user can control global file reception by setting boolean file_saving to True. /store {on,off} enables/disables file reception for all contacts /store on alice@jabber.org enables file reception for just Alice

By default, boolean setting a_close_f_recv is True. This means file reception for Alice is closed automatically after the file has been received. When the variable is false, file storage will remain open until the user enters /store off alice@jabber.org or /store off

During trickle connection, messages and files can be canceled with commands /cm and /cf. Already sent data is out of control of the user. Rx.py will discard received data when it receives cancel packet / new message/file, but this is a convenience feature, NOT a security feature. Sender-based control is snake oil: Nothing prevents the receiver from modifying their Rx.py to show/log partially transmitted data. The only solution to this would be to offer closed source version of TFC, which would defeat all security FOSS software is designed to offer.

Since the base64 encoding of transmitted files depends on TxM of contact, a frenemy could still send malware to user's RxM. Thus, the entire manual decoding feature was removed. The file is no longer generated with subprocess. This prevents shell injection with custom file names that would have otherwise become possible.

File data during trickle connection is loaded from a separate queue that has lower priority than message queue. This means that users can still exchange messages during file transmission: file data is output to contact when there are no messages to output. File data is indistinguishable from messages to NH, IM client, IM server and any adversary who might observe data from these systems. During trickle connection, it is extremely hard to determine whether the user is sending noise data, messages or files. No guarantees can, unfortunately, be made. Timing attacks are prone to side channel attacks anywhere from NH/smartphone microphone to the user moving the mouse on NH. Python is also a high-level language, which has its own problems.

Command line arguments

Added arguments for Tx.py, Rx.py, and NH.py to control many settings from the command line. View arguments by launching the program with -h or --help flag.

Hard coded packet length

Packet length was hardcoded to 254 to ensure everyone uses the same value, and to minimize the chance of errors when sending encrypted commands.

Unittesting

Wrote unittests for Tx.py, Rx.py, and NH.py to reduce the number of bugs. Created custom error classes for some functions for this purpose as well. Unittesting is a work in progress, so while TFC is more stable than ever, no guarantees are made.

New startup banner animation

I want to thank Tom Wallroth for his awesome work. I had so much fun editing his project to fit print_banner(). I'll leave the details of style as a surprise, check it out.

Made edits to installer

Installer now uses SHA256 instead of SHA512. As collision resistance against quantum computers is still 128 bits. The reason for this was the simplified import of file hashes when making changes.

More authentication was added with Trust-on-first-use (that being me) hashes for most downloaded libraries (PIP has no authentication). Provided that you can verify the setup.py's authenticity, you can detect MITM against crypto library downloads.

Added installation configuration for HWRNG Pi that TxM connects to over SSH. Ensured support for *buntu 16.04 OS and fixed issues with Lubuntu/Xubuntu in previous versions. Dropped NH support for Fedora and OpenSUSE and added it to Raspbian Jessie. Ensured NH configuration works with the latest release of Tails (2.2.1).

Changed installer naming style, instead of tfcOTPinstaller.py, it's just setup.py (run it as sudo).

TFC 0.5.5 | Update log

Whitepaper

Updated abstract
Changed CEV description, added proof of security for cascading cipher configuration
Added description of HMAC-SHA2-512 and SHA3-512 MAC
Upgraded network flow graph and attack graph
Updated key generation chapter

Manual

Updated abstract
Updated software settings
Updated software screenshots
Added URL source that helps the user to connect Pidgin to Tor hidden service (XMPP server)

CEV

Security

Upgraded all hashes from Keccak-256 to Keccak-512.
Upgraded Keccak-CTR PRF from 256-bits to 512-bits.
Upgraded Keccak-CTR key length from 256-bits to 512-bits.
Upgraded cyclic hashing of keys to also use Keccak-512: This should ensure keys stay secure over time.
XSalsa20, Twofish-CTR and AES-GCM use only 256 first bits from 512-bit keys.
Added SHA2-512 HMAC and SHA3-512 MAC (Keccak). These two MACs are concatenated to the cascaded ciphertext and GMAC and are independently verified using a constant time function. All three MACs need to be correct before the decryption takes place.

SHA-3 is immune against length extension attacks, thus the nested HMAC construction is not needed. However, according to paper* by Mostafa Taha and Patrick Schaumont, the optimal length of Keccak MAC-key is as close to (2 * rate - 1) as possible but not over the value. In the case of Keccak-512, the optimal key size is

(2 * 576 - 1) = 1151 bits ~ 143 bytes (1144 bits).

*http://rijndael.ece.vt.edu/schaum/papers/2013hosta.pdf

Added self-test functions for both MAC algorithms.
Upgraded key generation process:
1. Sampling speed is now set to 2000Hz: it no longer depends on RPi clock speed.
2. getEntropy now works without KeyboardInterrupt: Program now loads 1 500 000 samples over the period of ~15 minutes and exists.
3. Changed key generation process (pseudo-code):

old construction:

entropy = (VN(VN(HWRNG samples))) ⊕ /dev/(u)random
                
Keccak256(common kb-entropy input + entropy)
    > independent 256-bit key (Keccak-CTR)
    > independent 256-bit key (XSalsa20)
    > independent 256-bit key (Twofish-CTR)
    > independent 256-bit key (AES(GCM))

As Von Neumann whitening requires HWRNG sampling to be a Bernoulli process, it was requested a single layer of compression with a hash function would be added between them. For feces and jollies, added two, plus a third VN whitening pass. New construction:

entropy = 
(VN(Keccak512'(VN(Keccak512'(VN(HWRNG samples)))))) ⊕ /dev/(u)random

Keccak512(kb1 + entropy) > independent 512-bit key (Keccak-CTR)
Keccak512(kb2 + entropy) > independent 512-bit key (XSalsa20)
Keccak512(kb3 + entropy) > independent 512-bit key (Twofish-CTR)
Keccak512(kb4 + entropy) > independent 512-bit key (AES(GCM))

Keccak512(kb5 + entropy) > independent 512-bit key (HMAC-SHA2-512)
Keccak512(kb6 + entropy) > independent 512-bit key (SHA3-512-MAC #1)
Keccak512(kb7 + entropy) > independent 512-bit key (SHA3-512-MAC #2)
Keccak512(kb8 + entropy) > independent 512-bit key (SHA3-512-MAC #3)

where kb1..8 is a separate input by the user from the keyboard.

The Keccak512' takes 1024 bits in and outputs 512 bits. This ensures
the entropy of input is larger than output unless HWRNG produces very 
bad entropy (4 bits / byte or less)).

Usability

Added feature to launch tfcInstaller.py from Tx.py or Rx.py during local testing if lack of dependencies causes the first crypto library (XSalsa20) to causer import error.
Added option to Rx.py to print Noise packets/commands. This helps in debugging if the user is unsure why incoming packets are not displayed.
Removed deskew.c and integrated VN-whitening to genKey.py
Added prompt to overwrite keys that already exist.

OTP

Security

Added missing Keccak self-test function to startup self-test routine.

Stability

Removed unnecessary padding of one-time MAC
Added error handling and message when Rx.py runs out of keyfile.
Fixed issue where Tx.py not in local_testing mode aborted rx-side keyfile change if keyfile was not also in TxM side folder.

Usability

Added toBytes C program (by G. Vazzana) that speeds up keyfile conversion.

Common updates

Security

Replaced KeccakError class and error handling with graceful exits.
Fixed metadata leak in constant transmission mode: When sending long packets, between each packet only one command packet would be sent with the probability of 1/2. This made detection of long packets possible with statistical analysis. Tx.py now sends a random amount of command packets between each packet of a long message.

Auditability

Clarified docstrings.
Changed software to only import what is needed from modules.
Edited white space, exterminated typos, renamed variables, rewrote functions etc.
Changed rx.local.e filename globally to me.local.e. This allowed significantly cleaner code in genKey.py.
Changed references in documentation and variables from xmpp to account.

Stability

Keyfile rotation, overwriting, message encryption and keyID increase is now handled using a thread. KeyboardInterrupts etc. no longer cause errors when writing key IDs and keys.
Fixed issue where generated keys were not moved to instructive folders.
Fixed issue where the existence of keys to be overwritten is not checked from instructive folders.
Fixed display of me. header in nick change command.
Fixed issue where temporary files were not removed after keyfile generation.
Removed installation of Secure-Delete as isn't needed to run shred.
Added serial exception handling to Tx.py and Rx.py
Fixed issue where , caused problems with CSV files.
Fixed issue where illegal nick during contact creation was accepted.
Fixed issue where Tx.py did not remove temporary file if it was empty.

Usability

Added animated banner for TFC name and boolean setting print_banner to disable it.
Simplified self-test and key generation UI with acknowledgments that are printed to the same line after the task is complete.
Added a boolean option to genKey.py to disable prompt to create triplet from the key.
Added command /store {on,off} for Tx.py that allows the user to enable file reception on RxM. The command is encrypted. Rx.py lets the user know if file reception was (already) enabled/disabled.
Tx.py now shows useful printed commands when clear_input_screenis set True. The user just needs to press <enter> to continue.

TFC 0.5.4 Hotfix

Security

Fixed metadata leak where 138 or 139 char long messages were padded to 280 chars. Added check and graceful exit for padding function to ensure leaks will no longer happen.

Stability

Fixed crashes with long messages, files and multiple commands with bad parameters, rewrote contact selection functions. Fixed sliced logfile names. Fixed notification for locally received messages being discarded not showing when it should and vice versa.

TFC 0.5.4 update log

Documentation

Whitepaper

New pictures of chapter providing background.
Added more detailed analysis on leaks.
Added circuit for data diode with the flip-flop circuit.
Added picture of data diode with the circuit.
Added TFC CEV properties to abstract
Added CEV to the chapter on ciphers and self-audit.

Manual

Upgraded according to updates on software.

Software

Common

Security

Updated Keccak library from 2.0 to 3.0 which has new padding scheme.
Every function now uses isistance() input type validation.
Every safety-critical error now causes a descriptive graceful exit.

Auditability

Added docstrings for most functions with descriptions about the functionality and parameters.
Improved structure and readability of source code in general, removed typos etc.

Tx.py

Security

Common

Added threads that send a constant stream of noise commands and messages randomly (1:1 ratio) to hide metadata about when communication or command sending is actually taking place. The feature will exhaust OTP keys rapidly so the use of CEV version is recommended. Some XMPP servers might ban users for 'spamming', so using low repeat rate is recommended.
RandomSleep variable also affects the constant transmission setting.
For now, paste mode, contact changing and multicasting of messages are disabled during constant transmission to protect users against timing attacks against the system.

CEV

Added self-test functions for Keccak hash function, Keccak CTR, XSalsa20, Twofish CTR and AES-GCM
Fixed binascii.hex() NameError in cases where import of Crypto.Random.random fails: AES-GCM nonce (512 bits) is now generated using Kernel CSPRNG (/dev/urandom).
Removed code included in the libraries of Twofish and Salsa20 algorithms.
Added double assertion for Twofish CTR that checks each IV used was unique.
Changed Twofish IV generation from XORing nonce with the hash of a counter to XORing nonce with a zero-padded counter which is closer to standard.

OTP

Added self-test functions for Keccak hash function, one-time pad, and one-time MAC function.
Fixed an infinitesimally unlikely vulnerability where after reducing int('one-time key') % M521, either of the MAC keys would be zero and thus vulnerable. In the event this would occur (probability = 2/2^521), Tx.py will now exit.
When changing keys, checks that keys have been shredded and new keyfile is in place before reporting on successful key change.

Stability

Defined proper values for packet max and min size to prevent errors.

Auditability

Fixed the confusing xmpp variable to always mean the XMPP-address, without tx. prefixes.
Improved code structure significantly: the main loop is now much shorter and functions are more autonomic.

Usability

Common

Commands

/group rm without specified XMPP-addresses now overwrites and deletes group file.
/group <group name> now shows specified group members.
/msg <group name> is used to select group.
/group add <group name> <xmpp> now asks to create new group if not one already exists.
/store list command now asks RxM to display the list of temp files it has.
/aboutprovides clickable URLs (for local testing as TCB has no network connectivity.)

Others

User can now select a contact by typing the recipient XMPP to contact selection menu, tab-complete supported.
Automatically removes placement instruction for all tx.xmpp.e files.
Pressing <Enter> instead of giving nick now parses the name side from XMPP-address and capitalizes it.

Rx.py

Security

Common

Added self-test functions for Keccak hash function, Keccak-CTR XSalsa20, Twofish CTR and AES-GCM cascading crypto
Fixed failure to create a proper log of replayed packets.

CEV

Changed Twofish IV generation from XORing nonce with a hash of counter to XORing nonce with zero padded counter which is closer to standard. (Edit 03/2018: Which in hindsight was a great idea as that's the way TLS1.3 creates IV)
Removed code included in the library of Twofish and Salsa20 encryption functions.
Added double assertion for Twofish CTR that checks each IV used was unique.
Added an option to require manual acceptance of packets they key ID of which suddenly increased enough to DoS Rx.py or even crash it. When rejected, Rx.py makes an automatic event log to syslog.tfc. The user can also set the value of how large difference is required to trigger this. For constant transmission mode, this should be disabled.

OTP

Added self-test functions for Keccak hash function, one-time pad, one-time MAC and constant time MAC comparison function.

Stability

Common

Fixed crash where missing keyfile rx.xmpp.e causes an error in key ID loading.
Added clean exit when sending KeyboardInterrupt during contact nick input.
Changed UnboudLocalError to graceful exit when there is no me.xmpp.e file the moment Tx.py sends a message.
Fixed issue where malformed base64 padding caused TypeError and crash. Tampering event is now created.
Fixed issue where missing keyfile rx.xmpp.e caused get_key_id() to unnecessarily gracefully exit Rx.py.
Rx.py now prints a warning when it can't find keyfile or key ID in rxc.tfc.
Removed double warning from MAC failure.
Fixed whitespace after time stamp in syslog.tfc.

CEV

Fixed issue where MAC fail crashes on Twofish input parameter type error.

Auditability

Moved handling of encrypted commands to a function reducing the length of the main loop.

Usability

Common

Warn about missing keyfile pair. Automatically removes placement instruction for all me.xmpp.e and rx.xmpp.efiles.
Pressing <Enter> instead of giving nick now parses the name side from XMPP-address and capitalizes it.
Swapped Nick storing and loading from rx.xmpp.e to me.xmpp.e so the user can store the contact's nickname before receiving the rx.xmpp.e PSK: The latter keyfile will be added automatically to rxc.tfc.
Rx.py now reports on encoded files not yet stored when the file is rejected or improper temp file name is provided. (User can query Rx.py for a list of pending temp files with the command /store list.)
Automatically removes placement instruction for all me.xmpp.e and rx.xmpp.e keyfiles.

CEV

Hides noise messages/commands sent by Tx.py programs automatically.

OTP

Fixed issue in Rx.py OTP where a maliciously crafted packet with high keyID causes an OverflowError and crash.

NH.py

Usability

NH.py now lets the user know which account TFC uses to send messages. The user can change the account by dragging it to the topmost account in 'Manage Accounts' section in Pidgin.

genKey.py

Security

OTP

Automatically analyses whitened entropy with Ent and does graceful exit if obtained key data does not have high enough entropy (7.85 bits/byte by default)

CEV

Increased sampling time from 30000 to 50000.

Usability

genKey.py now takes parameters [FLAG] [CONTACT_XMPP] [USER_XMPP]
if the user specifies both XMPP-addresses, genKey.py generates a triplet of the key for both TxM and RxM of user and RxM of contact.
If the user does not specify one or either, genKey.py asks the user to input them manually (interactive) mode.
When key generation generates one key or triplet: tx.xmpp.e is given instruction and moved to folder move_to_TxM unless Tx.py exists in the same directory as genKey.py.
me.xmpp.e is given instruction and moved to folder move_to_RxM unless Rx.py exists in same directory as genKey.py.
rx.xmpp.e is given instruction, and it is moved to folder contact_keys?
Tx.py and Rx.py automatically remove instructions from file names, the user no longer has to do that.
Giving the output file name l or {,tx.,rx.}local{,.e} now asks if user wants to generate tx.local.e and rx.local.e files as a pair.

-tx.local.e is given instruction and moved to folder move_to_TxM unless Tx.py exists in same directory as genKey.py.

- rx.local.e is given instruction and moved to folder move_to_RxM unless Rx.py exists in the same directory as genKey.py.
Initializing software with command python genkey [FLAG] 'l' or {,tx.,rx.}local{,.e} also switches to local.
Setting boolean instructions to Falsein genKey.py disables generation of all instructions.

tfc{CEV,OTP}installer.py

Security

Installer is now provided with pre-installed SHA512 hashes of the latest version that are used to verify the integrity of files.
Created an RSA public key to sign the installer files, the public key is available at

https://cs.helsinki.fi/u/oottela/TFC-pub.key https://pgp.mit.edu/pks/lookup?op=get&search=0x4064F05A4D17DE97
Since exfiltration of private signing key is a serious issue, this requires some transparency:
1. The private key is handled only in unidirectionally separated TCB, protected with cascading crypto and undisclosed tricks.
2. Installers to be signed are moved via data diode to an RxM-like environment, from which the signature is read as a QR-code before it's decoded and uploaded to GitHub.
This will slow down the rate at which updates are published excluding cases where a vulnerability has been found.

Unfortunately, there is NO high assurance way for the user to verify TLS-MITM didn't take place and that a nation-state actor didn't change the public signing key on-the-fly to one the private pair of which was used to sign tampered files.

Auditability

Created functions for each installation subroutine.

Usability

Improved instructions that are provided.

TFC 0.4.13 update log

Added separate exit_with_message() function
Expanded group management help codes

TFC 0.4.11 update log

Split into TFC-CEV and TFC-OTP

TFC 0.4.10 update log

file-transfer
OTP: One-time MACs
Multicasting of messages
Local testing
Help menu scales to the terminal size
Attempt to relocate printable strings for easier translation

TFC 0.4.7 update log

Remove need to run TFC as superuser
Evaluate OTP keyfiles with DieHarder batter of tests
Combine TN.py and NR.py to NH.py

TFC 0.4.6 update log

UTF-8 encoding, OTP with XOR over binary data over modulo-arithmetics and printable ASCII
Long messages with multiple packets
Improved keygen

TFC 0.4.5 update log

Create installer
user manual
Swap OTP-keyfile with /newkf

TFC 0.4.4 Hotfix

Forward secrecy by overwriting of OTP keys after use.

TFC 0.4.4 update log

OTP encrypted Keccak MACs

TFC 0.4.3 update log

Counter for remaining messages

TFC 0.4.2 update log

ASCII OTP + OTP-encrypted CRC32 signature*

*In retrospect this home-brew implementation was horrible but then again, I had only started with crypto and didn't know what a MAC was.