Blog post about WPA crypto hardware acceleration

Author: redfast00

content/posts/0010-wpa.md

@@ -0,0 +1,335 @@
++++
+authors = ["Jasper Devreker"]
+title = "Reverse engineering WPA crypto acceleration on ESP32"
+date = "2025-03-07"
+description = "Figuring out how the hardware cryptography acceleration works"
++++
+
+This is a more technical blob post about a specific aspect of the ESP32 Wi-Fi hardware; see previous blog posts for a more general overview.
+
+In one of the previous blog posts, we talked about implementing station mode on the ESP32. We then only implemented connecting to open networks, since there is a bunch of extra work that goes into connecting to WPA protected networks. In this blog post, we'll go into what needs to be implemented to connect WPA2 networks, and how the ESP32 hardware plays into that.
+
+# Protected networks: general structure
+
+To use a WPA2 protected network, you basically have to do three things:
+basically consists of 3 parts:
+
+- key derivation / 4 way handshake (when connecting)
+- encrypting/decrypting packets (for every packet sent/received)
+- keeping the keys correct at runtime (every n minu)
+
+# Key derivation
+
+1. The Pairwise Master Key (PMK) is generated from the passphrase and SSID (`PMK = PBKDF2(HMAC−SHA1, passphrase, ssid, 4096, 256)`)
+2. The station and AP then do a 4-way handshake to derive and install a Pairwise Temporal Key between the AP and the station. This key encrypts all unicast traffic
+3. The 4 way handshake also provides the Group Temporal Key to the client. This key is used to encrypt multicast traffic. They are rekeyed every time a STA joins or leave, or every n minutes, depending on AP settings.
+
+# Hardware acceleration of encrypting/decrypting packets
+
+Every protected packet we send needs to be encrypted, and every protected packet we receive needs to be decrypted. If the ESP32 would need to do this in software, it would cost a lot of CPU time; this is why it is implemented in hardware. Instead of having to encrypt a packet before handing the encrypted result over to the radio hardware, we can just hand the plaintext packet to the hardware and tell it which "crypto key slot" index to use.
+
+The ESP32 has 25 crypto key slots. Every crypto key slot contains the nescessary information to encrypt and decrypt packets:
+
+- the cipher suite (CCMP, GCMP, ...)
+- the virtual interface index on which the packet will be decrypted  (see also the previous blog post on the RX filter; which is basically the VIF)
+- the key ID
+- the MAC address of the peer who will send encrypted packets to us
+- the key
+- (some extra bits relating to protected management frames and Signaling and Payload Protected A-MSDU's)
+
+## Encryption flow
+
+The encryption flow works as follows:
+
+1. Before sending any packets, set up the crypto slot with the correct information
+2. To have the hardware send an encrypted packet: prepare the packet as for unencrypted data, but with some exceptions:
+    - set the 'protected' bit in the frame control field
+    - after the 802.11 MAC header, but before the data, construct and insert the 8 byte CCMP header (contains the nonce and key id)
+    - add 8 bytes to the end of the packet: the hardware will overwrite these to insert the MIC (message integrity checksum)
+3. Send the packet as normally, but indicate which crypto key slot that the hardware should use in the PLCP1 register
+
+It should be noted that there are a lot of algorithms that the hardware has support for:
+
+```c
+enum wpa_alg {
+    WIFI_WPA_ALG_NONE   = 0,
+    WIFI_WPA_ALG_WEP40  = 1,
+    WIFI_WPA_ALG_TKIP   = 2,
+    WIFI_WPA_ALG_CCMP   = 3,
+    WIFI_WAPI_ALG_SMS4  = 4,
+    WIFI_WPA_ALG_WEP104 = 5,
+    WIFI_WPA_ALG_WEP    = 6,
+    WIFI_WPA_ALG_IGTK   = 7,
+    WIFI_WPA_ALG_PMK    = 8,
+    WIFI_WPA_ALG_GCMP   = 9,
+};
+```
+
+It's neat to see that they have support for SMS4, a Chinese cipher that was suggested for 802.11i, but was ultimately rejected. It lives on in [WAPI](https://en.wikipedia.org/wiki/WLAN_Authentication_and_Privacy_Infrastructure).
+
+
+## Decryption flow
+
+The decryption flow is transparent for the hardware: once you set up the key slot with the correct interface and MAC address, the hardware will automatically decrypt the packet.
+This is done based on the MAC address in the key slot and in the frame. There is a bit in the key slot that indicates whether the keyslot is for unicast or multicast frames.
+
+*Unicast: RA (addr 1) of the encrypted frame is the MAC address of module*
+
+For unicast slots, a packet will decrypt if the MAC address in the key slot matches the TA (address 2) of a frame.
+
+*Multicast: RA (addr 1) of the encrypted frame is a multicast address*
+
+Here, it does not really seem to matter what the address in the key slot is; decryption will happen as long as the RA is multicast and the packet gets through the filters
+
+As a recap:
+
+| hw key idx 0 | To DS | From DS | reception if addr 2 matches BSSID in RX filter | reception if addr 3 matches BSSID in RX filter | notes                                                                     |
+|--------------|-------|---------|--------|--------|---------------------------------------------------------------------------|
+|              | 0     | 0       | no     | yes    |                                                                           |
+|              | 0     | 1       | yes    | no     |                                                                           |
+|              | 1     | 0       | yes    | yes    | decryption works even works when addr2 and addr3 both don't match the addr in the keyslot! |
+|              | 1     | 1       | yes    | yes    | decryption even works when addr2, addr3 and addr4 both don't match the addr in the keyslot! |
+
+
+
+# Testing this
+
+To reverse engineer this part of the hardware, I wrote a Python script that generates example plaintexts and ciphertexts, also called 'test vectors' in the cryptography world. However, this had a bit of a chicken and egg problem: how do we know that our Python implementation to generate test vectors is correct? Luckily for us, there are some public test vectors: the [FreeBSD net80211 regression tests](https://web.mit.edu/freebsd/head/tools/regression/net80211/ccmp/test_ccmp.c) contain 8 CCMP test vectors.
+
+Unfortunately, none of the 8 test vectors contain a test case where there is a '4 address frame' (my term for a packet that has both from-DS and to-DS set in its Frame Control field, and as such has 4 MAC addresses). To back up a bit and explain what a '4 address frame' is: normally, Wi-Fi frames contain 3 MAC addresses. This is enough for a transmitter, receiver and an extra address for either the destination, source or BSSID. This is sufficient for the case where you have access points and stations. However, there is a special case (where a packet is both going to and coming from the distribution network; most often the case in mesh networks) where there are 4 addresses in a 802.11 frame.
+
+This 4th address, if present, is used in the encryption algorithm (more specifically, in calculating the AAD); so is critical to handle this correctly to correctly encrypt/decrypt '4 address frames'. I was worried that the hardware might not do this correctly since:
+
+- it is likely a tiny bit cheaper with regards to amount of gates used to not handle this special case
+- Espressif does not send any '4 address frames' as far as I know, let alone encrypted '4 address frames'
+- '4 address frames' seem to be pretty uncommon
+
+Using the 802.11 standard, I implemented this special case in the Python test program. This was then validated by generating a packet and decrypting it with Wireshark. I then tested encryption on the ESP32, and to my relief, the hardware *does* handle this correctly. The decryption is also correctly implemented. Good job Espressif!
+
+
+## Appendix: demonstation code
+
+See https://github.com/esp32-open-mac/esp32-open-mac/pull/24. Note that this still uses the Espressif HAL (`wDev_Insert_KeyEntry`); implementing the HAL ourselves and doing the 4 way handshake will be done in another PR.
+
+## Appendix: Python test vector script:
+
+```python
+from scapy.layers.dot11 import Dot11, Dot11QoS
+from scapy.all import hexdump, wrpcap, sendp, RadioTap
+from Crypto.Cipher import AES
+import struct
+
+def bytes_to_c_arr(data, lowercase=True):
+    return [format(b, '#04x' if lowercase else '#04X') for b in data]
+
+TESTCASE = 11
+
+transmit_if = 'wlan2mon'
+
+# These testcases come from freebsd/head/tools/regression/net80211/ccmp/test_ccmp.c
+if TESTCASE == 1:
+    plain = bytes([
+        0x08, 0x48, 0xc3, 0x2c, 0x0f, 0xd2, 0xe1, 0x28,	# /* 802.11 Header */
+        0xa5, 0x7c, 0x50, 0x30, 0xf1, 0x84, 0x44, 0x08,
+        0xab, 0xae, 0xa5, 0xb8, 0xfc, 0xba, 0x80, 0x33, 
+        
+        0xf8, 0xba, 0x1a, 0x55, 0xd0, 0x2f, 0x85, 0xae, # /* Plaintext Data */
+        0x96, 0x7b, 0xb6, 0x2f, 0xb6, 0xcd, 0xa8, 0xeb,
+        0x7e, 0x78, 0xa0, 0x50, 
+    ])
+    pn = 0xB5039776E70C
+    key = bytes.fromhex("c9 7c 1f 67 ce 37 11 85  51 4a 8a 19 f2 bd d5 2f")
+    key_id = 0
+    expected = bytes([
+        0x08, 0x48, 0xc3, 0x2c, 0x0f, 0xd2, 0xe1, 0x28,
+        0xa5, 0x7c, 0x50, 0x30, 0xf1, 0x84, 0x44, 0x08,
+        0xab, 0xae, 0xa5, 0xb8, 0xfc, 0xba, 0x80, 0x33,
+        0x0c, 0xe7, 0x00, 0x20, 0x76, 0x97, 0x03, 0xb5,
+        0xf3, 0xd0, 0xa2, 0xfe, 0x9a, 0x3d, 0xbf, 0x23,
+        0x42, 0xa6, 0x43, 0xe4, 0x32, 0x46, 0xe8, 0x0c,
+        0x3c, 0x04, 0xd0, 0x19, 0x78, 0x45, 0xce, 0x0b,
+        0x16, 0xf9, 0x76, 0x23
+    ])
+elif TESTCASE == 4:
+    plain = bytes([
+        0xa8, 0xca, 0x3a, 0x11, 0x71, 0x2a, 0x9d, 0xdf, 0x11, 0xdb,
+        0x8e, 0xf8, 0x22, 0x73, 0x47, 0x01, 0x59, 0x14, 0x0d, 0xd6,
+        0x46, 0xa2, 0xc0, 0x2f, 0x67, 0xa5,
+        0x4f, 0xad, 0x2b, 0x1c, 0x29, 0x0f, 0xa5, 0xeb, 0xd8, 0x72,
+        0xfb, 0xc3, 0xf3, 0xa0, 0x74, 0x89, 0x8f, 0x8b, 0x2f, 0xbb,
+    ])
+    pn = 0xF670A55A0FE3
+    key = bytes.fromhex('8c 89 a2 eb c9 6c 76 02  70 7f cf 24 b3 2d 38 33')
+    key_id = 0
+    expected = bytes([
+        0xa8, 0xca, 0x3a, 0x11, 0x71, 0x2a, 0x9d, 0xdf, 0x11, 0xdb,
+        0x8e, 0xf8, 0x22, 0x73, 0x47, 0x01, 0x59, 0x14, 0x0d, 0xd6,
+        0x46, 0xa2, 0xc0, 0x2f, 0x67, 0xa5, 0xe3, 0x0f, 0x00, 0x20,
+        0x5a, 0xa5, 0x70, 0xf6, 0x9d, 0x59, 0xb1, 0x5f, 0x37, 0x14,
+        0x48, 0xc2, 0x30, 0xf4, 0xd7, 0x39, 0x05, 0x2e, 0x13, 0xab,
+        0x3b, 0x1a, 0x7b, 0x10, 0x31, 0xfc, 0x88, 0x00, 0x4f, 0x35,
+        0xee, 0x3d,
+    ])
+elif TESTCASE == 7:
+    plain = bytes([
+        0x18, 0x79, 0x81, 0x46, 0x9b, 0x50, 0xf4, 0xfd, 0x56, 0xf6,
+        0xef, 0xec, 0x95, 0x20, 0x16, 0x91, 0x83, 0x57, 0x0c, 0x4c,
+        0xcd, 0xee, 0x20, 0xa0,
+        0x98, 0xbe, 0xca, 0x86, 0xf4, 0xb3, 0x8d, 0xa2, 0x0c, 0xfd,
+        0xf2, 0x47, 0x24, 0xc5, 0x8e, 0xb8, 0x35, 0x66, 0x53, 0x39,
+    ])
+    pn = 0x5EEC4073E723
+    key = bytes.fromhex('1b db 34 98 0e 03 81 24 a1 db 1a 89 2b ec 36 6a')
+    key_id = 3
+    expected = bytes([
+        0x18, 0x79, 0x81, 0x46, 0x9b, 0x50, 0xf4, 0xfd, 0x56, 0xf6,
+        0xef, 0xec, 0x95, 0x20, 0x16, 0x91, 0x83, 0x57, 0x0c, 0x4c,
+        0xcd, 0xee, 0x20, 0xa0, 0x23, 0xe7, 0x00, 0xe0, 0x73, 0x40,
+        0xec, 0x5e, 0x12, 0xc5, 0x37, 0xeb, 0xf3, 0xab, 0x58, 0x4e,
+        0xf1, 0xfe, 0xf9, 0xa1, 0xf3, 0x54, 0x7a, 0x8c, 0x13, 0xb3,
+        0x22, 0x5a, 0x2d, 0x09, 0x57, 0xec, 0xfa, 0xbe, 0x95, 0xb9,
+    ])
+elif TESTCASE == 10: # custom testcase for 4 address mode
+    plain = bytes([
+        0x08, 0x48, 0xc3, 0x2c, 0x0f, 0xd2, 0xe1, 0x28,	# /* 802.11 Header */
+        0xa5, 0x7c, 0x50, 0x30, 0xf1, 0x84, 0x44, 0x08,
+        0xab, 0xae, 0xa5, 0xb8, 0xfc, 0xba, 0x80, 0x33, 
+        0xf8, 0xba, 0x1a, 0x55, 0xd0, 0x2f, 0x85, 0xae, # /* Plaintext Data */
+        0x96, 0x7b, 0xb6, 0x2f, 0xb6, 0xcd, 0xa8, 0xeb,
+        0x7e, 0x78, 0xa0, 0x50, 
+    ])
+    pn = 0xB5039776E70C
+    key = bytes.fromhex("c9 7c 1f 67 ce 37 11 85  51 4a 8a 19 f2 bd d5 2f")
+    key_id = 0
+    parsed = Dot11(plain)
+    setattr(parsed.FCfield, 'to-DS', True)
+    setattr(parsed.FCfield, 'from-DS', True)
+    parsed.addr4 = '00:23:45:67:89:ab'
+    plain = parsed.build()
+    expected = None
+elif TESTCASE == 11:
+    # encrypt with custom mac addr
+    plain = bytes([
+        0x08, 0x48, 0xc3, 0x2c, 0x0f, 0xd2, 0xe1, 0x28,	# /* 802.11 Header */
+        0xa5, 0x7c, 0x50, 0x30, 0xf1, 0x84, 0x44, 0x08,
+        0xab, 0xae, 0xa5, 0xb8, 0xfc, 0xba, 0x80, 0x33, 
+        0xf8, 0xba, 0x1a, 0x55, 0xd0, 0x2f, 0x85, 0xae, # /* Plaintext Data */
+        0x96, 0x7b, 0xb6, 0x2f, 0xb6, 0xcd, 0xa8, 0xeb,
+        0x7e, 0x78, 0xa0, 0x50, 
+    ])
+    pn = 0
+    key = bytes.fromhex("c9 7c 1f 67 ce 37 11 85  51 4a 8a 19 f2 bd d5 2f")
+    key_id = 0
+    parsed = Dot11(plain)
+
+    broadcast = 'ff:ff:ff:ff:ff:ff'
+    ra = '00:23:45:67:89:ab'
+    bssid = 'f0:ae:a5:b8:fc:ba'
+    unrelated = 'ae:25:aa:63:1c:8e'
+
+    setattr(parsed.FCfield, 'from-DS', False)
+    setattr(parsed.FCfield, 'to-DS', True)
+
+    parsed.addr1 = broadcast
+    parsed.addr2 = unrelated
+    parsed.addr3 = unrelated
+    # parsed.addr4 = unrelated
+    # parsed.addr4 = bssid
+    parsed.FCfield.retry = False
+
+    plain = parsed.build()
+    expected = None
+else:
+    assert False, 'Testcase not found'
+
+dot11 = Dot11(plain)
+print(dot11)
+# Sanity check on scapy library
+assert (dot11.build() == plain)
+assert dot11.proto == 0, "Only PV0 supported, no 802.11ah"
+
+
+dot11_copy = dot11.copy()
+mac_1 = bytes.fromhex(dot11.addr1.replace(':', ''))
+mac_2 = bytes.fromhex(dot11.addr2.replace(':', ''))
+mac_3 = bytes.fromhex(dot11.addr3.replace(':', ''))
+mac_4 = None if dot11.addr4 is None else bytes.fromhex(dot11.addr4.replace(':', ''))
+
+plaintext_data = dot11.payload.build()
+
+print("Original packet:")
+hexdump(dot11)
+
+# mask out the bits that should be masked out
+dot11.subtype &= 0b1000
+dot11.FCfield.retry = False
+setattr(dot11.FCfield, 'pw-mgt', False)
+dot11.FCfield.MD = False
+dot11.FCfield.protected = 1
+
+assert Dot11QoS not in dot11, "QoS not implemented; see page 2493 (CCMP AAD) of 80211-2020.pdf"
+
+frame = dot11.build()
+
+# this bytes.fromhex("00 00") assumes the fragment number is 0
+assert dot11.SC & 0b1111 == 0, "Fragment number != 0 not implemented"
+aad = frame[:2] + mac_1 + mac_2 + mac_3 + bytes.fromhex("00 00") + (bytes() if mac_4 is None else mac_4)
+
+print('AAD:')
+hexdump(aad)
+
+pn_packed = struct.pack("<Q", pn)[:6]
+
+print("Packed PN:")
+hexdump(pn_packed)
+
+ccmp_header = bytes([
+    pn_packed[0],
+    pn_packed[1],
+    0, # reserved
+    (1<<5 | key_id << 6),
+    pn_packed[2],
+    pn_packed[3],
+    pn_packed[4],
+    pn_packed[5],
+    ])
+
+print("CCMP header:")
+hexdump(ccmp_header)
+
+# bytes.fromhex("00") assumes [Priority Management PV1 Zeros] are all zero
+nonce = bytes.fromhex("00") + mac_2 + struct.pack(">Q", pn)[-6:]
+
+print("CCM nonce")
+hexdump(nonce)
+
+cipher = AES.new(key, AES.MODE_CCM, nonce, mac_len=8)
+cipher.update(aad)
+msg = nonce, aad, cipher.encrypt(plaintext_data), cipher.digest()
+
+
+calculated = dot11_copy.build()[:(24 if dot11.addr4 is None else 24+6)] + ccmp_header + msg[2] + msg[3]
+
+print("on ESP32:")
+print(', '.join(bytes_to_c_arr(dot11_copy.build()[:(24 if dot11.addr4 is None else 24+6)] + ccmp_header + plaintext_data + bytes([0]*8))))
+
+
+if transmit_if is not None:
+    dot11 = Dot11(calculated)
+    dot11.FCfield.protected = True
+    sendp(RadioTap() / dot11, iface=transmit_if)
+
+if expected is None:
+    # wrpcap("out.pcap", [Dot11(calculated)])
+    pass
+else:
+    if calculated == expected:
+        print("Calculated frame matches expectation")
+        hexdump(calculated)
+    else:
+        print("Calculated does not match expected")
+        print("Calculated:")
+        hexdump(calculated)
+        print("Expected")
+        hexdump(expected)
+```