MSC2134: Identity Hash Lookups

[Half-Shot]

To fix #2130

Rendered

Author: @anoadragon453

---

The room to discuss this MSC is #hashing-msc:amorgan.xyz.

---

FCP: #2134 (comment) ~ Travis

---

We're aware that this MSC has quite a few comments under its belt. New readers should only be concerned with the current state of the proposal (i.e what's under the Rendered link) and the unresolved comments below. ~ anoa

[ara4n]

thanks for the proposal - looks generally sane.

we need to consider salts; i’d have thought we should include a random salt in the hash and alongside the hash to make rainbow table lookups slightly harder?

[ara4n]

this should prolly be {H(email + salt), salt} to make rainbow table attacks slightly harder

[ara4n]

(although salt would have to be constant for everyone, so it doesn’t buy much)

[Half-Shot]

Would this require us to do a negotiation step where the server sends it's unique salt, and then hash it using that salt? Bit more involved but would ensure uniqueness?

(Alternatively, use the server's server_name as the salt?)

[ara4n]

the question is how bound 3pids are stored. if they are stored as hashes, then we have no choice but have a predictable salt in the hash, which doesn’t achieve much other than stop arbitrary pregenerated hash tables of mail addresses from working. instead, you’d have to pregenerate H(email + “foo”) tables specifically for matrix, where foo is defined in this MSC.. We can’t use server_name as when you’re doing a lookup you don’t know any server_names for the target.

if we store the 3pids as plain text equivalent in the IS, then the salt could be random per lookup, as the server can calculate its own hashes to compare against the uploaded ones. however, this would be pretty inefficient, plus we probably don’t want to be storing unhashed 3pids. it would mean an attacker would have to calculate hash tables for every salt (ie per lookup item) to crack the hash.

[turt2live]

Specifying a salt in the spec does at least drive away lazy attackers who want to quickly use their existing tables. It's a bit of a weak argument, but if we can force people to do even a little bit of Matrix-specific logic to exploit a vulnerability then we are in a slightly safer position.

We can also specify a SHOULD for servers to hash and salt the hashes of 3PIDs when persisting them, making rainbow tables much harder to use at the persistence level. It does a look a bit dirty (sha256(sha256(3pid), salt)), but it might be enough to ward off attacks on the persisted values.

[Half-Shot]

After much thought (and @KitsuneRal 's well made point), I don't think this MSC is going to add any substantial security or privacy to the current endpoints. The more I look at this issue, the more I realize that lookups are inherently flawed as they will always leak your contacts.

There is possibly some small gain for hashing strings with more variance such as email or social handles, but zero gain for a phone number. I would argue that for private deployments this feature is surplus to requirement, as you would be expected to trust your identity server.

So in short, I'm not sure if this MSC is the right thing to fix the problem. If anything this looks more like a problem for app developers and transparency of their UX.

[turt2live]

Specifying a salt in the spec does at least drive away lazy attackers who want to quickly use their existing tables. It's a bit of a weak argument, but if we can force people to do even a little bit of Matrix-specific logic to exploit a vulnerability then we are in a slightly safer position.

We can also specify a SHOULD for servers to hash and salt the hashes of 3PIDs when persisting them, making rainbow tables much harder to use at the persistence level. It does a look a bit dirty (sha256(sha256(3pid), salt)), but it might be enough to ward off attacks on the persisted values.

[anoadragon453]

Why not a truncated hash for extra security + smaller network load?

[anoadragon453]

I would do this, but I'm not sure what's safe to truncate by...

[anoadragon453]

I still think this proposal has much value for non-msisdn 3PIDs, and a little value for phone numbers. Imagine if you had a service where users could only use passwords that were 6 chars long. It would still be valuable not to store those passwords in plain text.

I do wonder though, could bcrypt not be used for this? Does it provide any extra security?

[anoadragon453]

I have taken over this MSC and added salt pepper and other details.

[anoadragon453]

Adding a mechanism for identity servers to use their own salt by suggestion of @mathijs:matrix.vgorcum.com.

[mvgorcum]

modulo my small comments lgtm.

[turt2live]

Implementations:

• Use the v2 (hashed) lookup for identity server queries matrix-js-sdk#1021
• Use the v2 lookup API for 3PID invites synapse#5897
• Have 'mappings' contain medium as well as address sydent#197
• Hash 3PID lookups sydent#184

[turt2live]

Spec PR: #2287

[turt2live]

Merged 🎉