From 156a98f3f7e50e59a5cd51449a036743241c5d1b Mon Sep 17 00:00:00 2001 From: Andrew Thompson Date: Thu, 13 Feb 2020 18:07:04 -0800 Subject: [PATCH 01/12] WIP --- 0008-lorawan-routing.md | 112 ++++++++++++++++++++++++++++++++++++++++ 1 file changed, 112 insertions(+) create mode 100644 0008-lorawan-routing.md diff --git a/0008-lorawan-routing.md b/0008-lorawan-routing.md new file mode 100644 index 000000000..9565e07ca --- /dev/null +++ b/0008-lorawan-routing.md @@ -0,0 +1,112 @@ +- Start Date: 2020-02-11 +- HIP PR: +- Tracking Issue: + +# Summary +[summary]: #summary + +This HIP proposes to add bloom filter based routing tables to the blockchain ledger to aid in routing LoRaWAN join packets to the correct destination. Additionally once the device has joined, it can be assigned a DevAddr corresponding to an address allocation the OUI controls. + +# Motivation +[motivation]: #motivation + +#Why are we doing this? What use cases does it support? What problems does it solve? What is the expected outcome? + +With the addition of LoRaWAN support, we have inherited some of the routing problems inherent in the LoRaWAN network model. LoRaWAN assumes that the network has a centralized network server (or set of them) (similar to a LAN) and our model is much closer to internet routing. Additionally, it turns out many LoRaWAN devices do not allow end-user configuration of DevEUI/AppEUI/AppKey. Thus the existing model of co-opting the AppEUI to route the join packet is unsuitable and something new is needed.a + +In addition, the current scheme for DevAddr allocation (the session address, essentially) is not ideal because we map the 32 bit OUI to the DevAddr directly. This will cause problems for large OUIs at the expense of small ones. It would be good for OUIs to be able to obtain more address space for their DevAddrs if they're willing to pay for it. + +# Stakeholders +[stakeholders]: #stakeholders + +* LoRaWAN device users + +# Detailed Explanation +[detailed-explanation]: #detailed-explanation + +A bloom filter is a probabalistic data structure that allows for fast checking for key membership It never has false negatives but can have false positives depending on a configurable probability. + +A new ledger entity would be added to the blockchain, a 'routing entry'. These would have a total ordering (assigned by the order they are added to the chain) and a bloom filter that is one of a specific list of combinations of bits in the filter and # of items (addresses). Thus this bloom filter can be used to both route join requests *and* designate a size of a DevAddr allocation. + +Proposed allocation sizes are as follows: + +| Devices | Bytes on chain | +|---------|----------------| +| 100 | 497 bytes | +| 1000 | 4.85 kbytes | +| 10000 | 48.56 kbytes | +| 100000 | 485.63 kbytes | +| 1000000 | 4.74 mbytes | + +Each bloom filter is created with a 1 in 199,725,093 chance of a false positive and uses 28 hash functions. + +With the largest bloom filter, we can check for key membership in 100 microseconds on an intel core i7. The smallest bloom filter above can be checked for membership in about 70 microseconds. + +In a more complete test, 2039 OUIs holding device space for 100,052,066 devices and 10,000 populated was able to resolve the owning OUI for a device in about 0.8 seconds. + +One good piece of news is that bloom filters compress *really* well when they're mostly full or mostly empty, and they compress more the bigger they are. I've observed 2-3 orders of magnitude of compression for mostly empty blooms. + +# Drawbacks +[drawbacks]: #drawbacks + +- Why should we *not* do this? + +No matter how you slice it, this adds quite a lot of information to the ledger. Storing every possible DevEUI is 32Gb ( 2^32 * 8 bytes) of data. Storing AppEUIs or the associated routing destination just makes it worse. Bloom filters compress massively more, but they're not free. + +One flaw with this scheme will be 'route posioning' where I will be able to get join packets for devices I want to see routed to myself simply by adding that device to my bloom filter. In theory this could be used as a denial of service attack by interfering with the receive window for the legitimate join response packet. + +# Rationale and Alternatives +[alternatives]: #rationale-and-alternatives + +## Why is this design the best in the space of possible designs? + +Bloom filters are probably the most mature probabalistic filter of this type, other alternatives are cuckoo filters and exor filters, but libraries for these are less mature and not as widely available. Simply storing a complete routing table, as discussed before, is not reasonable because of size constraints. + +##What other designs have been considered and what is the rationale for not choosing them? + +Suggestions for other designs are welcome, routing arbitrary addresses without being able to use prefix routing seems hard. + +- What is the impact of not doing this? + +We will be unable to support routing join packets for arbitrary LoRaWAN join packets and thus unable to support LoRaWAN devices that are already provisioned or unable to have their credentials modified. + +# Unresolved Questions +[unresolved]: #unresolved-questions + +##What parts of the design do you expect to resolve through the HIP process before this gets merged? + +We need to nail down the permissible sizes of the bloom filters, and the false positive rate we want to tolerate. + +We need to decide if these routing table entries also contain a range of DevAddr entries to be used for routing of non-join packets. + +##What parts of the design do you expect to resolve through the implementation of this feature? + +We need to define where/how these routing entries are stored, how they're created/updated and if they can be transferred (like IP space can be traded). + +##What related issues do you consider out of scope for this HIP that could be addressed in the future independently of the solution that comes out of this HIP? + +Pricing and availability of routing table entries for organizations operating their own OUI. + +# Deployment Impact +[deployment-impact]: #deployment-impact + +##How will current users be impacted? + +All existing LoRaWAN devices will either have to have their credentials added to the routing table, or they will need new credentials. + +##How will existing documentation/knowlegebase need to be supported? + +We don't have a lot of documentation for the existing thing, so we can probably forego this. + +#Is this backwards compatible? + +It can be made to be, we can probably generate routing tables out of console's DB. + +# Success Metrics +[success-metrics]: #success-metrics + +What metrics can be used to measure the success of this design? + +##What should we measure to prove an acceptance of this by it's users? + +A user should be able to onboard any arbitrary LoRaWAN device without altering its DevEUI/AppEUI/AppKey. From ffdb7842af1df3d949244ead1aa7792cbd2fc0af Mon Sep 17 00:00:00 2001 From: Andrew Thompson Date: Wed, 19 Feb 2020 08:50:55 -0800 Subject: [PATCH 02/12] Update --- 0008-lorawan-routing.md | 263 ++++++++++++++++++++++++++++++++++------ 1 file changed, 226 insertions(+), 37 deletions(-) diff --git a/0008-lorawan-routing.md b/0008-lorawan-routing.md index 9565e07ca..a9fe1d0e4 100644 --- a/0008-lorawan-routing.md +++ b/0008-lorawan-routing.md @@ -5,16 +5,29 @@ # Summary [summary]: #summary -This HIP proposes to add bloom filter based routing tables to the blockchain ledger to aid in routing LoRaWAN join packets to the correct destination. Additionally once the device has joined, it can be assigned a DevAddr corresponding to an address allocation the OUI controls. +This HIP proposes to add [XOR filter](https://lemire.me/blog/2019/12/19/xor-filters-faster-and-smaller-than-bloom-filters/) +based routing tables to the blockchain ledger to aid in routing LoRaWAN join +packets to the correct destination. Additionally once the device has joined, it +can be assigned a DevAddr corresponding to an address allocation the OUI controls. # Motivation [motivation]: #motivation #Why are we doing this? What use cases does it support? What problems does it solve? What is the expected outcome? -With the addition of LoRaWAN support, we have inherited some of the routing problems inherent in the LoRaWAN network model. LoRaWAN assumes that the network has a centralized network server (or set of them) (similar to a LAN) and our model is much closer to internet routing. Additionally, it turns out many LoRaWAN devices do not allow end-user configuration of DevEUI/AppEUI/AppKey. Thus the existing model of co-opting the AppEUI to route the join packet is unsuitable and something new is needed.a +With the addition of LoRaWAN support, we have inherited some of the routing +problems inherent in the LoRaWAN network model. LoRaWAN assumes that the network +has a centralized network server (or set of them) (similar to a LAN) and our model +is much closer to internet routing. Additionally, it turns out many LoRaWAN devices +do not allow end-user configuration of DevEUI/AppEUI/AppKey. Thus the existing +model of co-opting the AppEUI to route the join packet is unsuitable and +something new is needed. -In addition, the current scheme for DevAddr allocation (the session address, essentially) is not ideal because we map the 32 bit OUI to the DevAddr directly. This will cause problems for large OUIs at the expense of small ones. It would be good for OUIs to be able to obtain more address space for their DevAddrs if they're willing to pay for it. +In addition, the current scheme for DevAddr allocation (the session address, +essentially) is not ideal because we map the 32 bit OUI to the DevAddr directly. +This will cause problems for large OUIs at the expense of small ones. It would +be good for OUIs to be able to obtain more address space for their DevAddrs if +they're willing to pay for it. # Stakeholders [stakeholders]: #stakeholders @@ -24,83 +37,258 @@ In addition, the current scheme for DevAddr allocation (the session address, ess # Detailed Explanation [detailed-explanation]: #detailed-explanation -A bloom filter is a probabalistic data structure that allows for fast checking for key membership It never has false negatives but can have false positives depending on a configurable probability. - -A new ledger entity would be added to the blockchain, a 'routing entry'. These would have a total ordering (assigned by the order they are added to the chain) and a bloom filter that is one of a specific list of combinations of bits in the filter and # of items (addresses). Thus this bloom filter can be used to both route join requests *and* designate a size of a DevAddr allocation. - -Proposed allocation sizes are as follows: - -| Devices | Bytes on chain | -|---------|----------------| -| 100 | 497 bytes | -| 1000 | 4.85 kbytes | -| 10000 | 48.56 kbytes | -| 100000 | 485.63 kbytes | -| 1000000 | 4.74 mbytes | - -Each bloom filter is created with a 1 in 199,725,093 chance of a false positive and uses 28 hash functions. - -With the largest bloom filter, we can check for key membership in 100 microseconds on an intel core i7. The smallest bloom filter above can be checked for membership in about 70 microseconds. - -In a more complete test, 2039 OUIs holding device space for 100,052,066 devices and 10,000 populated was able to resolve the owning OUI for a device in about 0.8 seconds. - -One good piece of news is that bloom filters compress *really* well when they're mostly full or mostly empty, and they compress more the bigger they are. I've observed 2-3 orders of magnitude of compression for mostly empty blooms. +A XOR filter is a probabalistic data structure that allows for fast checking for +key membership It never has false negatives but can have false positives. XOR +filters come in 2 flavors, xor8 and xor16. xor8 has a false positive rate of +about 0.3 percent and xor16 has a false positive rate of about 0.002%. We +propose that the space tradeoff (2x) for xor16 is worth it because it results in +much less ambiguous routing for large numbers of devices. + +XOR filters require inputs be hashed to a 64 bit integer before being stored or +compared against the XOR filter. We propose the use of +[xxhash](http://cyan4973.github.io/xxHash/)'s 64 bit mode. This is one of the +fastest hash functions available and seems to provide good distribution for the +XOR filter. Both the DevEUI and the AppEUI (totaling 128 bits together) would be +hashed to a 64 bit key, this should help in cases of DevAddr collisions. + +A new ledger entity would be added to the blockchain, a 'routing entry'. These +would have a total ordering (assigned by the order they are added to the chain) +and an XOR filter built from the set of active DevEUI/AppEUIs for the owning OUI. + +XOR16 filters for a range of sizes are presented below for illustration purposes: + +| Devices | Bytes on chain | +|-----------|----------------| +| 100 | 322 bytes | +| 1,000 | 2.48 kbytes | +| 10,000 | 24.09 kbytes | +| 100,000 | 240.21 kbytes | +| 1,000,000 | 2.34 mbytes | + +With the largest XOR filter, we can check for key membership in about 14 +milliseconds on an raspberry pi 3b+ in 32 bit mode. The smallest xor filter can +be checked in well under a millisecond. + +In a more complete test, 10,000 OUIs holding device space for 50 million devices +was able route an address to its destination in 1.1 seconds, again on a +raspberry pi 3b+ in 32 bit mode. 64 bit mode is expected to be slightly faster. +The same test on a pi 4 is about 200 milliseconds faster, and the same test on +an intel i7-7700HQ runs in about 96 milliseconds. + +Some more benchmarks are presented below. The memory and database sizes are +approximate. + +On an Intel i7-7700HQ running Void linux: + +``` +running xor8_xxhash with 10000 OUIs and 8000000 Devices +Attempting 1000 random lookups +Average memory 0.21Mb, max 0.51Mb, min 0.10Mb +Approximate database size 9.99Mb +Average errors 39.300, max 59, min 20 +Average lookup 0.056s, max 0.150s, min 0.034s +Lookup misses 0 + + +running xor16_xxhash with 10000 OUIs and 8000000 Devices +Attempting 1000 random lookups +Average memory 0.54Mb, max 0.96Mb, min 0.23Mb +Approximate database size 23.76Mb +Average errors 0.155, max 2, min 0 +Average lookup 0.077s, max 0.202s, min 0.054s +Lookup misses 0 + + +running bloom with 10000 OUIs and 8000000 Devices +Attempting 1000 random lookups +Average memory 0.55Mb, max 1.02Mb, min 0.24Mb +Approximate database size 38.65Mb +Average errors 0.036, max 2, min 0 +Average lookup 0.101s, max 0.218s, min 0.068s +Lookup misses 0 +``` + +On a Raspberry Pi 3b+ running 32 bit Raspbian: + +``` +running xor8_xxhash with 10000 OUIs and 8000000 Devices +Attempting 1000 random lookups +Average memory 0.12Mb, max 0.25Mb, min -0.06Mb +Approximate database size 9.99Mb +Average errors 39.300, max 59, min 20 +Average lookup 0.360s, max 0.426s, min 0.278s +Lookup misses 0 + + +running xor16_xxhash with 10000 OUIs and 8000000 Devices +Attempting 1000 random lookups +Average memory 0.27Mb, max 0.84Mb, min 0.08Mb +Approximate database size 23.76Mb +Average errors 0.155, max 2, min 0 +Average lookup 0.533s, max 0.629s, min 0.487s +Lookup misses 0 + + +running bloom with 10000 OUIs and 8000000 Devices +Attempting 1000 random lookups +Average memory 0.14Mb, max 0.88Mb, min 0.02Mb +Approximate database size 38.65Mb +Average errors 0.030, max 1, min 0 +Average lookup 0.700s, max 0.838s, min 0.651s +Lookup misses 1000 +``` + +On a Raspberry Pi 4 running 32 bit Rasbian: + +``` +running xor8_xxhash with 10000 OUIs and 8000000 Devices +Attempting 1000 random lookups +Average memory 0.15Mb, max 0.26Mb, min -0.02Mb +Approximate database size 9.99Mb +Average errors 39.300, max 59, min 20 +Average lookup 0.269s, max 0.386s, min 0.227s +Lookup misses 0 + + +running xor16_xxhash with 10000 OUIs and 8000000 Devices +Attempting 1000 random lookups +Average memory 0.24Mb, max 0.37Mb, min 0.08Mb +Approximate database size 23.76Mb +Average errors 0.155, max 2, min 0 +Average lookup 0.391s, max 0.436s, min 0.355s +Lookup misses 0 + + +running bloom with 10000 OUIs and 8000000 Devices +Attempting 1000 random lookups +Average memory 0.20Mb, max 0.40Mb, min 0.02Mb +Approximate database size 38.65Mb +Average errors 0.030, max 1, min 0 +Average lookup 0.536s, max 0.583s, min 0.500s +Lookup misses 1000 +``` + +As we can see, xor16-xxhash seems to be the best combination of speed, size and +accuracy. Additionally the bloom filter seems to have some portability issue +(likely related to the 32/64 bit switch) which would need to be resolved. + +Finally, a more ambitious test, on the same 3 machines, in the same order: + +``` +running xor16_xxhash with 10000 OUIs and 50000000 Devices +Attempting 1000 random lookups +Average memory 0.53Mb, max 1.58Mb, min 0.04Mb +Approximate database size 118.54Mb +Average errors 0.179, max 2, min 0 +Average lookup 0.096s, max 0.137s, min 0.079s +Lookup misses 0 + +running xor16_xxhash with 10000 OUIs and 50000000 Devices +Attempting 1000 random lookups +Average memory -0.01Mb, max 1.58Mb, min -0.09Mb +Approximate database size 118.54Mb +Average errors 0.179, max 2, min 0 +Average lookup 1.112s, max 1.322s, min 0.945s +Lookup misses 0 + +running xor16_xxhash with 10000 OUIs and 50000000 Devices +Attempting 1000 random lookups +Average memory 0.17Mb, max 1.99Mb, min 0.07Mb +Approximate database size 118.54Mb +Average errors 0.179, max 2, min 0 +Average lookup 0.916s, max 1.009s, min 0.864s +Lookup misses 0 +``` + +The benchmark code can be found +[here](https://github.com/Vagabond/bloom_router). These runs were generated by +the command in the README. The benchmark was first run on the Intel machine to +generate the routing tables, run a second time to use the generated routing +tables, and then the tables were copied to the Pi to run the test with the same +databases (generating the tables can be quite RAM intensive, so it is unsuitable +to be done on the Pi). # Drawbacks [drawbacks]: #drawbacks - Why should we *not* do this? -No matter how you slice it, this adds quite a lot of information to the ledger. Storing every possible DevEUI is 32Gb ( 2^32 * 8 bytes) of data. Storing AppEUIs or the associated routing destination just makes it worse. Bloom filters compress massively more, but they're not free. +No matter how you slice it, this adds quite a lot of information to the ledger. +However, storing every possible DevEUI in a list would be 32Gb ( 2^32 * 8 bytes) +of data. Storing AppEUIs or the associated routing destination just makes it +worse. XOR filters compress massively more, but they're not free. -One flaw with this scheme will be 'route posioning' where I will be able to get join packets for devices I want to see routed to myself simply by adding that device to my bloom filter. In theory this could be used as a denial of service attack by interfering with the receive window for the legitimate join response packet. +One flaw with this scheme will be 'route posioning' where I will be able to get +join packets for devices I want to see routed to myself simply by adding that +device to my bloom filter. In theory this could be used as a denial of service +attack by interfering with the receive window for the legitimate join response +packet. # Rationale and Alternatives [alternatives]: #rationale-and-alternatives ## Why is this design the best in the space of possible designs? -Bloom filters are probably the most mature probabalistic filter of this type, other alternatives are cuckoo filters and exor filters, but libraries for these are less mature and not as widely available. Simply storing a complete routing table, as discussed before, is not reasonable because of size constraints. +XOR16 filters using xxhash64 were evaluated against bloom filters using a 1 in +200 million false positive rate, XOR8 filters using erlang:phash2 and xxhash64 and +xor16 filters using erlang:phash2. XOR16 with xxhash64 provided the best all +around speed and memory usage, and were very competitive for lookup times. +phash2 has too many collisions (it's only a 60 bit hash) for this application, +so it was rejected. + +Other alternatives include cuckoo filters, but they're expected to be slower +than xor16+xxhash64 and require a more complicated implementation. Simply +storing a complete routing table, as discussed before, is not reasonable +because of size constraints. ##What other designs have been considered and what is the rationale for not choosing them? -Suggestions for other designs are welcome, routing arbitrary addresses without being able to use prefix routing seems hard. +Suggestions for other designs are welcome, routing arbitrary addresses without +being able to use prefix routing seems hard. - What is the impact of not doing this? -We will be unable to support routing join packets for arbitrary LoRaWAN join packets and thus unable to support LoRaWAN devices that are already provisioned or unable to have their credentials modified. +We will be unable to support routing join packets for arbitrary LoRaWAN +join packets and thus unable to support LoRaWAN devices that are +already provisioned or unable to have their credentials modified. # Unresolved Questions [unresolved]: #unresolved-questions ##What parts of the design do you expect to resolve through the HIP process before this gets merged? -We need to nail down the permissible sizes of the bloom filters, and the false positive rate we want to tolerate. +We need to nail down the permissible sizes of the XOR filters we want to store. -We need to decide if these routing table entries also contain a range of DevAddr entries to be used for routing of non-join packets. +We need to decide if these routing table entries also contain a range of +DevAddr entries to be used for routing of non-join packets. ##What parts of the design do you expect to resolve through the implementation of this feature? -We need to define where/how these routing entries are stored, how they're created/updated and if they can be transferred (like IP space can be traded). +We need to define where/how these routing entries are stored, how they're +created/updated and if they can be transferred (like IP space can be traded). ##What related issues do you consider out of scope for this HIP that could be addressed in the future independently of the solution that comes out of this HIP? -Pricing and availability of routing table entries for organizations operating their own OUI. +Pricing and availability of routing table entries for organizations operating +their own OUI. # Deployment Impact [deployment-impact]: #deployment-impact ##How will current users be impacted? -All existing LoRaWAN devices will either have to have their credentials added to the routing table, or they will need new credentials. +All existing LoRaWAN devices will either have to have their credentials added +to the routing table, or they will need new credentials. ##How will existing documentation/knowlegebase need to be supported? -We don't have a lot of documentation for the existing thing, so we can probably forego this. +We don't have a lot of documentation for the existing thing, so we can probably +forego this. #Is this backwards compatible? -It can be made to be, we can probably generate routing tables out of console's DB. +It can be made to be, we can probably generate routing tables out of console's +DB, for the devices we know the DevEUI for. # Success Metrics [success-metrics]: #success-metrics @@ -109,4 +297,5 @@ What metrics can be used to measure the success of this design? ##What should we measure to prove an acceptance of this by it's users? -A user should be able to onboard any arbitrary LoRaWAN device without altering its DevEUI/AppEUI/AppKey. +A user should be able to onboard any arbitrary LoRaWAN device without altering +its DevEUI/AppEUI/AppKey. From 02738389dad74a60f724103febc1f4f8b20ea72b Mon Sep 17 00:00:00 2001 From: Andrew Thompson Date: Wed, 19 Feb 2020 08:53:06 -0800 Subject: [PATCH 03/12] Fix subsections --- 0008-lorawan-routing.md | 16 ++++++++-------- 1 file changed, 8 insertions(+), 8 deletions(-) diff --git a/0008-lorawan-routing.md b/0008-lorawan-routing.md index a9fe1d0e4..a1816cc09 100644 --- a/0008-lorawan-routing.md +++ b/0008-lorawan-routing.md @@ -241,12 +241,12 @@ than xor16+xxhash64 and require a more complicated implementation. Simply storing a complete routing table, as discussed before, is not reasonable because of size constraints. -##What other designs have been considered and what is the rationale for not choosing them? +## What other designs have been considered and what is the rationale for not choosing them? Suggestions for other designs are welcome, routing arbitrary addresses without being able to use prefix routing seems hard. -- What is the impact of not doing this? +## What is the impact of not doing this? We will be unable to support routing join packets for arbitrary LoRaWAN join packets and thus unable to support LoRaWAN devices that are @@ -255,19 +255,19 @@ already provisioned or unable to have their credentials modified. # Unresolved Questions [unresolved]: #unresolved-questions -##What parts of the design do you expect to resolve through the HIP process before this gets merged? +## What parts of the design do you expect to resolve through the HIP process before this gets merged? We need to nail down the permissible sizes of the XOR filters we want to store. We need to decide if these routing table entries also contain a range of DevAddr entries to be used for routing of non-join packets. -##What parts of the design do you expect to resolve through the implementation of this feature? +## What parts of the design do you expect to resolve through the implementation of this feature? We need to define where/how these routing entries are stored, how they're created/updated and if they can be transferred (like IP space can be traded). -##What related issues do you consider out of scope for this HIP that could be addressed in the future independently of the solution that comes out of this HIP? +## What related issues do you consider out of scope for this HIP that could be addressed in the future independently of the solution that comes out of this HIP? Pricing and availability of routing table entries for organizations operating their own OUI. @@ -275,12 +275,12 @@ their own OUI. # Deployment Impact [deployment-impact]: #deployment-impact -##How will current users be impacted? +## How will current users be impacted? All existing LoRaWAN devices will either have to have their credentials added to the routing table, or they will need new credentials. -##How will existing documentation/knowlegebase need to be supported? +## How will existing documentation/knowlegebase need to be supported? We don't have a lot of documentation for the existing thing, so we can probably forego this. @@ -295,7 +295,7 @@ DB, for the devices we know the DevEUI for. What metrics can be used to measure the success of this design? -##What should we measure to prove an acceptance of this by it's users? +## What should we measure to prove an acceptance of this by it's users? A user should be able to onboard any arbitrary LoRaWAN device without altering its DevEUI/AppEUI/AppKey. From c2f3ce61466b003731bb967959ca8b6e7706fca5 Mon Sep 17 00:00:00 2001 From: Andrew Thompson Date: Wed, 19 Feb 2020 08:53:43 -0800 Subject: [PATCH 04/12] Fix --- 0008-lorawan-routing.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/0008-lorawan-routing.md b/0008-lorawan-routing.md index a1816cc09..a32832f53 100644 --- a/0008-lorawan-routing.md +++ b/0008-lorawan-routing.md @@ -211,7 +211,7 @@ to be done on the Pi). # Drawbacks [drawbacks]: #drawbacks -- Why should we *not* do this? +## Why should we *not* do this? No matter how you slice it, this adds quite a lot of information to the ledger. However, storing every possible DevEUI in a list would be 32Gb ( 2^32 * 8 bytes) From 48a8b8f01c53717a555b75fac83b064b09f37fd2 Mon Sep 17 00:00:00 2001 From: Andrew Thompson Date: Wed, 19 Feb 2020 08:55:51 -0800 Subject: [PATCH 05/12] Fix --- 0008-lorawan-routing.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/0008-lorawan-routing.md b/0008-lorawan-routing.md index a32832f53..b48bf5a21 100644 --- a/0008-lorawan-routing.md +++ b/0008-lorawan-routing.md @@ -285,7 +285,7 @@ to the routing table, or they will need new credentials. We don't have a lot of documentation for the existing thing, so we can probably forego this. -#Is this backwards compatible? +## Is this backwards compatible? It can be made to be, we can probably generate routing tables out of console's DB, for the devices we know the DevEUI for. From 04bdd366dc2c8f9a46bcafc644ed2aa2185913a5 Mon Sep 17 00:00:00 2001 From: Andrew Thompson Date: Thu, 20 Feb 2020 15:32:57 -0800 Subject: [PATCH 06/12] Update 0008-lorawan-routing.md Co-Authored-By: Louis Thiery --- 0008-lorawan-routing.md | 2 ++ 1 file changed, 2 insertions(+) diff --git a/0008-lorawan-routing.md b/0008-lorawan-routing.md index b48bf5a21..dbe428fc3 100644 --- a/0008-lorawan-routing.md +++ b/0008-lorawan-routing.md @@ -4,7 +4,9 @@ # Summary [summary]: #summary +Many LoRaWAN sensors come pre-provisioned and as such AppEUI and AppKey cannot be updated. [Our current LoRaWAN implementation](https://github.com/helium/HIP/commit/8e233ac37b9c39c3083be014d7f4b25aa40b1a18) requires updating these fields for a device to work on the Helium network. +We would like to implement a scheme that takes AppEUI, AppKey, and DevEUI as fixed parameters, but still enables hotspots to route data to the proper endpoint. This HIP proposes to add [XOR filter](https://lemire.me/blog/2019/12/19/xor-filters-faster-and-smaller-than-bloom-filters/) based routing tables to the blockchain ledger to aid in routing LoRaWAN join packets to the correct destination. Additionally once the device has joined, it From 002f9dee19aba141c744ef5802ea00f454d44b05 Mon Sep 17 00:00:00 2001 From: Andrew Thompson Date: Thu, 20 Feb 2020 15:34:30 -0800 Subject: [PATCH 07/12] Break DevAddr routing into a seperate HIP --- ...routing.md => 0008-lorawan-join-routing.md | 17 +- 0009-lorawan-devaddr-routing.md | 161 ++++++++++++++++++ 2 files changed, 165 insertions(+), 13 deletions(-) rename 0008-lorawan-routing.md => 0008-lorawan-join-routing.md (93%) create mode 100644 0009-lorawan-devaddr-routing.md diff --git a/0008-lorawan-routing.md b/0008-lorawan-join-routing.md similarity index 93% rename from 0008-lorawan-routing.md rename to 0008-lorawan-join-routing.md index dbe428fc3..8967d9be7 100644 --- a/0008-lorawan-routing.md +++ b/0008-lorawan-join-routing.md @@ -9,8 +9,8 @@ Many LoRaWAN sensors come pre-provisioned and as such AppEUI and AppKey cannot b We would like to implement a scheme that takes AppEUI, AppKey, and DevEUI as fixed parameters, but still enables hotspots to route data to the proper endpoint. This HIP proposes to add [XOR filter](https://lemire.me/blog/2019/12/19/xor-filters-faster-and-smaller-than-bloom-filters/) based routing tables to the blockchain ledger to aid in routing LoRaWAN join -packets to the correct destination. Additionally once the device has joined, it -can be assigned a DevAddr corresponding to an address allocation the OUI controls. +packets to the correct destination. A related HIP describes the routing of +packets once the device has joined. # Motivation [motivation]: #motivation @@ -25,12 +25,6 @@ do not allow end-user configuration of DevEUI/AppEUI/AppKey. Thus the existing model of co-opting the AppEUI to route the join packet is unsuitable and something new is needed. -In addition, the current scheme for DevAddr allocation (the session address, -essentially) is not ideal because we map the 32 bit OUI to the DevAddr directly. -This will cause problems for large OUIs at the expense of small ones. It would -be good for OUIs to be able to obtain more address space for their DevAddrs if -they're willing to pay for it. - # Stakeholders [stakeholders]: #stakeholders @@ -51,7 +45,7 @@ compared against the XOR filter. We propose the use of [xxhash](http://cyan4973.github.io/xxHash/)'s 64 bit mode. This is one of the fastest hash functions available and seems to provide good distribution for the XOR filter. Both the DevEUI and the AppEUI (totaling 128 bits together) would be -hashed to a 64 bit key, this should help in cases of DevAddr collisions. +hashed to a 64 bit key, this should help in cases of DevEUI collisions. A new ledger entity would be added to the blockchain, a 'routing entry'. These would have a total ordering (assigned by the order they are added to the chain) @@ -261,13 +255,10 @@ already provisioned or unable to have their credentials modified. We need to nail down the permissible sizes of the XOR filters we want to store. -We need to decide if these routing table entries also contain a range of -DevAddr entries to be used for routing of non-join packets. - ## What parts of the design do you expect to resolve through the implementation of this feature? We need to define where/how these routing entries are stored, how they're -created/updated and if they can be transferred (like IP space can be traded). +created/updated, etc. ## What related issues do you consider out of scope for this HIP that could be addressed in the future independently of the solution that comes out of this HIP? diff --git a/0009-lorawan-devaddr-routing.md b/0009-lorawan-devaddr-routing.md new file mode 100644 index 000000000..c9e4a0063 --- /dev/null +++ b/0009-lorawan-devaddr-routing.md @@ -0,0 +1,161 @@ +- Start Date: 2020-02-20 +- HIP PR: +- Tracking Issue: + +# Summary +[summary]: #summary + +This HIP proposes a routing scheme for LoRaWAN packets post-join, via DevAddr. A +block of DevAddr addresses will be obtainable for an OUI via a blockchain +transaction. This range of addresses will be assigned to devices joined to this +OUI and hotspots will be able to do fast and simple 'range routing' to route +packets to their destination OUI. These address ranges should be transferrable +and splittable (down to some minimum size) to allow for address reconfiguration +in the future. A related HIP describes the routing of LoRaWAN join packets. + +# Motivation +[motivation]: #motivation + +The Helium network is a multi-tenant network, we envision multiple +organizations controlling their own OUIs and devices without any central +authority. Thus we need an unambiguous routing mechanism so that hotspots know +where to send device packets. This routing scheme should not require frequent +updates or extensive coordination. + +The current strategy is to alias devices by assigning them the OUI as their +DevAddr and brute forcing the Message Integrity Code of the packet against all +known active session keys. While this is relatively fast (about 30 microseconds +on an Intel i7-7700HQ), an OUI with a very large number of active devices will +have to do this, as a linear search, a lot. We consider this will be a +scalability problem in the future. + +# Stakeholders +[stakeholders]: #stakeholders + +Anyone routing device traffic on the Helium network. + +# Detailed Explanation +[detailed-explanation]: #detailed-explanation + +- Introduce and explain new concepts. + +- It should be reasonably clear how the proposal would be implemented. + +- Provide representative examples that show how this proposal would be commonly + used. + +- Corner cases should be dissected by example. + +We'd introduce new transactions to obtain/transfer/split 'address blocks', +similar to IP address space. These blocks should be available in some fixed +sizes that are neatly divisible into smaller sizes (eg powers of 2, as in IP +address blocks). The size of the allocation would affect the price required to +obtain them. Allocations would consist of {Start and Size} (eg {0, 512} or +{2048, 4096} etc). The blocks would be allocated contiguously, starting from 0, +serialized into a total ordering by the blockchain transaction order. + +A new class of ledger entries would be added, in a new column family. These +entries would have the big-endian starting address as the key: +`<>` and the corresponding value would be the +owning OUI: `<>`. Big endian is used because we +can take advantage of rocksdb's lexiographic sorting properties more effectively. + +When a packet needs to be routed, the Hotspot simply needs to seek to the +DevAddr as big endian in the rocksdb column family. The iterator can then be +used to find the previous key. That key will be the start of the address +allocation, and the value will be the OUI to route to. This should be extremely +fast and efficent, and can be augmented with a cache if needed. + +A range routing prototype has been added to the bloom_router repo and some +benchmarks have been done: + +Intel i7-7700HQ +``` +running with 1000 OUIs and 8000000 Devices +Attempting 1000 random lookups +Average memory 0.25Mb, max 0.29Mb, min -0.04Mb +Approximate database size 0.02Mb +Average lookup 0.004s, max 0.1s, min 0.00001s +Lookup misses 0 +``` + +Raspberry Pi 4b +``` +running with 1000 OUIs and 8000000 Devices +Attempting 1000 random lookups +Average memory 0.17Mb, max 0.19Mb, min 0.00Mb +Approximate database size 0Mb +Average lookup 0.04s, max 0.5s, min 0.00004s +Lookup misses 0 +``` + +Raspberry Pi 3b+ +``` +running with 1000 OUIs and 8000000 Devices +Attempting 1000 random lookups +Average memory 0.08Mb, max 0.10Mb, min -0.06Mb +Approximate database size 0Mb +Average lookup 0.03s, max 0.4s, min 0.0001s +Lookup misses 0 +``` + +The good news is that, on average, this is orders of magnitude faster, on +average, than the XOR routing scheme for joins, but it is still not free. + +# Drawbacks +[drawbacks]: #drawbacks + + + +# Rationale and Alternatives +[alternatives]: #rationale-and-alternatives + +This is your chance to discuss your proposal in the context of the whole design +space. This is probably the most important section! + +- Why is this design the best in the space of possible designs? + +- What other designs have been considered and what is the rationale for not + choosing them? + +- What is the impact of not doing this? + +# Unresolved Questions +[unresolved]: #unresolved-questions + +- What parts of the design do you expect to resolve through the HIP process + before this gets merged? + +- What parts of the design do you expect to resolve through the implementation + of this feature? + +- What related issues do you consider out of scope for this HIP that could be + addressed in the future independently of the solution that comes out of this + HIP? + +# Deployment Impact +[deployment-impact]: #deployment-impact + +Describe how this design will be deployed and any potential imapact it may have on +current users of this project. + +- How will current users be impacted? + +- How will existing documentation/knowlegebase need to be supported? + +- Is this backwards compatible? + + - If not, what is the procedure to migrate? + +# Success Metrics +[success-metrics]: #success-metrics + +What metrics can be used to measure the success of this design? + +- What should we measure to prove a performance increase? + +- What should we measure to prove an improvement in stability? + +- What should we measure to prove a reduction in complexity? + +- What should we measure to prove an acceptance of this by it's users? From e5f60968f0e8af7b87bb4b660d994875caea027a Mon Sep 17 00:00:00 2001 From: Andrew Thompson Date: Thu, 20 Feb 2020 16:19:55 -0800 Subject: [PATCH 08/12] Add more deets --- 0009-lorawan-devaddr-routing.md | 14 +++++++++++++- 1 file changed, 13 insertions(+), 1 deletion(-) diff --git a/0009-lorawan-devaddr-routing.md b/0009-lorawan-devaddr-routing.md index c9e4a0063..1d0d4e044 100644 --- a/0009-lorawan-devaddr-routing.md +++ b/0009-lorawan-devaddr-routing.md @@ -29,6 +29,17 @@ on an Intel i7-7700HQ), an OUI with a very large number of active devices will have to do this, as a linear search, a lot. We consider this will be a scalability problem in the future. +Even a small amount of addresses could be used, in combination with geographic +information, to disambiguate devices very efficently. The maximum amount of +addresses an organization would need would likely be roughly the number of +distinct devices they expect to have in a geographic area at once. For example, +8 addresses would be able to disambiguate between thousands of devices as long +as no more than 8 of those devices were in the same geographic region at once. +Devices moving between regions regularly would probably require falling back on +the brute force lookup, or those devices could be re-addressed by forcing a +re-join (or they may re-join automatically after a period of being out of Helium +network coverage). + # Stakeholders [stakeholders]: #stakeholders @@ -105,7 +116,8 @@ average, than the XOR routing scheme for joins, but it is still not free. # Drawbacks [drawbacks]: #drawbacks - +This is yet another piece of data to store in the ledger, although the size is +quite small (64 bits per address block). It also involves seeking on the disk. # Rationale and Alternatives [alternatives]: #rationale-and-alternatives From 7385035a449c6319a035a650f7afcdeec65aa7f7 Mon Sep 17 00:00:00 2001 From: Andrew Thompson Date: Fri, 21 Feb 2020 16:08:24 -0800 Subject: [PATCH 09/12] Updates --- 0009-lorawan-devaddr-routing.md | 24 +++++++++++++++++++++++- 1 file changed, 23 insertions(+), 1 deletion(-) diff --git a/0009-lorawan-devaddr-routing.md b/0009-lorawan-devaddr-routing.md index 1d0d4e044..4911e30f7 100644 --- a/0009-lorawan-devaddr-routing.md +++ b/0009-lorawan-devaddr-routing.md @@ -13,6 +13,9 @@ packets to their destination OUI. These address ranges should be transferrable and splittable (down to some minimum size) to allow for address reconfiguration in the future. A related HIP describes the routing of LoRaWAN join packets. +Range routing refers to finding which range an address falls into, eg 5 is +between 0 and 9. + # Motivation [motivation]: #motivation @@ -127,11 +130,24 @@ space. This is probably the most important section! - Why is this design the best in the space of possible designs? +Mapping all devices in an OUI to a single address has aliasing problems, prefix +routing is harder to sequentially allocate and manage. I'm not sure what else is +suitable. + - What other designs have been considered and what is the rationale for not choosing them? +As mentioned above, aliasing all devices to one address (the OUI number) and +prefix routing are the only two candidates I can think of. + +Range addressing also allows us to punt on the 33.5 million DevAddr problem for +a while (32 bits of DevAddr minus 7 bits of 'NetID' in the lorawan spec). + - What is the impact of not doing this? +We end up with thousands or more devices aliased to a single address and have to +perform expensive brute force lookups to resolve DevAddrs to DevEUIs. + # Unresolved Questions [unresolved]: #unresolved-questions @@ -153,11 +169,17 @@ current users of this project. - How will current users be impacted? +Existing DevAddr addressing will have to be replaced the next time the devices +authenticate to the network. + - How will existing documentation/knowlegebase need to be supported? +This HIP should provide the foundation for the LoRaWAN routing documentation. + - Is this backwards compatible? - - If not, what is the procedure to migrate? +This change should be fairly transparent, we just need to force all the existing +devices to re-authenticate. # Success Metrics [success-metrics]: #success-metrics From 5b7430cebe07985de7412e60fd15831546b3a1e7 Mon Sep 17 00:00:00 2001 From: Andrew Thompson Date: Fri, 21 Feb 2020 16:21:00 -0800 Subject: [PATCH 10/12] Review comments --- 0008-lorawan-join-routing.md | 13 ++++++++++--- 1 file changed, 10 insertions(+), 3 deletions(-) diff --git a/0008-lorawan-join-routing.md b/0008-lorawan-join-routing.md index 8967d9be7..8633f4d84 100644 --- a/0008-lorawan-join-routing.md +++ b/0008-lorawan-join-routing.md @@ -47,9 +47,10 @@ fastest hash functions available and seems to provide good distribution for the XOR filter. Both the DevEUI and the AppEUI (totaling 128 bits together) would be hashed to a 64 bit key, this should help in cases of DevEUI collisions. -A new ledger entity would be added to the blockchain, a 'routing entry'. These -would have a total ordering (assigned by the order they are added to the chain) -and an XOR filter built from the set of active DevEUI/AppEUIs for the owning OUI. +A list of XOR filters would be attached to the OUI record in the ledger that +would contain the routing filters for that OUI. A single filter would suffice +for small OUIs, only larger OUIs whose amount of devices could exceed the +maximum filter size would require more than one routing filter. XOR16 filters for a range of sizes are presented below for illustration purposes: @@ -204,6 +205,12 @@ tables, and then the tables were copied to the Pi to run the test with the same databases (generating the tables can be quite RAM intensive, so it is unsuitable to be done on the Pi). +The process of generating the filter is roughly as follows: the complete list of +{DevEUI, AppEUI} pairs are hashed into a 64 bit keys and then fed into the XOR +filter generator. The filter then does some bit twiddling to set some bits in a +list of 'fingerprints'. A fairly readable example can be found +[here](https://github.com/juanbono/xor-filter/blob/master/src/lib.rs). + # Drawbacks [drawbacks]: #drawbacks From 544d8fda53911952b49fccafdd3e05e9c1aaa207 Mon Sep 17 00:00:00 2001 From: Andrew Thompson Date: Fri, 21 Feb 2020 16:37:57 -0800 Subject: [PATCH 11/12] Less average --- 0009-lorawan-devaddr-routing.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/0009-lorawan-devaddr-routing.md b/0009-lorawan-devaddr-routing.md index 4911e30f7..d178d9117 100644 --- a/0009-lorawan-devaddr-routing.md +++ b/0009-lorawan-devaddr-routing.md @@ -113,8 +113,8 @@ Average lookup 0.03s, max 0.4s, min 0.0001s Lookup misses 0 ``` -The good news is that, on average, this is orders of magnitude faster, on -average, than the XOR routing scheme for joins, but it is still not free. +The good news is that, on average, this is orders of magnitude fasterthan the +XOR routing scheme for joins, but it is still not free. # Drawbacks [drawbacks]: #drawbacks From 8ae86f7265b475a2b8981c3f17aa30b6ca7bb1e7 Mon Sep 17 00:00:00 2001 From: lthiery Date: Mon, 4 May 2020 10:24:16 -0700 Subject: [PATCH 12/12] update from range to subnet --- 0000-template.md | 35 ++++++++++-------- 0009-lorawan-devaddr-routing.md | 64 ++++++++++++--------------------- README.md | 1 + 3 files changed, 44 insertions(+), 56 deletions(-) diff --git a/0000-template.md b/0000-template.md index 35d6075bf..f4491b766 100644 --- a/0000-template.md +++ b/0000-template.md @@ -1,38 +1,45 @@ -- Start Date: +- Start Date: 2020-02-18 - HIP PR: - Tracking Issue: # Summary [summary]: #summary -One paragraph explanation of the proposal. +LongFi is not a full protocol from the ground up, but instead a blockchain layer on top of LoRaWAN. This allows any off-the-shelf LoRaWAN device to connect to the Helium network if you can update its AppKey and AppEui. # Motivation [motivation]: #motivation -Why are we doing this? What use cases does it support? What problems does it -solve? What is the expected outcome? +There are many LoRaWAN compatible devices already out there and LoRaWAN already has many desirable protocol features (ACK, downlink, FCC certified, international definition). In order to accelerate adoption of the Helium network and to lower technical barriers, LongFi is no longer a distinct protocol from LoRaWAN but instead a layering of some blockchain components on top of LoRaWAN. + # Stakeholders [stakeholders]: #stakeholders -* Who is affected by this HIP? - -* How are we soliciting feedback on this HIP from these stakeholders? Note that - they may not be watching the HIPs repository or even aren't directly active in - the Rust Async Ecosystem working group. +* LoRaWAN device users # Detailed Explanation [detailed-explanation]: #detailed-explanation -- Introduce and explain new concepts. +This initial implementation of LongFi on LoRaWAN focuses on a single method of end-device activation: Over-the-Air Activation (OTAA). Activation by Personalization (ABP) is currently not supported. + +In OTAA, DevEUI, AppEUI, and AppKey are all the LoRaWAN primitives used in the Join Request. The LongFi primitives of OUI and Device_ID are mapped into AppEUI: + +``` +___________________________________________ +| AppEui (32 bit) | -- It should be reasonably clear how the proposal would be implemented. +| OUI (16 bit) | Device_ID (16 bit) | +‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾ +``` -- Provide representative examples that show how this proposal would be commonly - used. +The AppEui (unique device identifier) and AppKey are used to accomplish a join, at which point the device is assigned a netid, a devaddr, and arkey, and nwkey +07:53:01.544 -> netid: 3302728 +07:53:01.544 -> devaddr: 1000000 +07:53:01.544 -> artKey: E8-FB-23-2B-CA-41-D4-32-E0-F5-54-24-5F-FC-8A-DF +07:53:01.544 -> nwkKey: C3-E6-6F-B-B1-84-1E-4A-79-6F-27-E-E9-74-D6-3B +07:53:01.544 -> 733431: EV_TXCOMPLETE (includes waiting for RX windows) -- Corner cases should be dissected by example. # Drawbacks [drawbacks]: #drawbacks diff --git a/0009-lorawan-devaddr-routing.md b/0009-lorawan-devaddr-routing.md index d178d9117..2a122bf6e 100644 --- a/0009-lorawan-devaddr-routing.md +++ b/0009-lorawan-devaddr-routing.md @@ -6,15 +6,17 @@ [summary]: #summary This HIP proposes a routing scheme for LoRaWAN packets post-join, via DevAddr. A -block of DevAddr addresses will be obtainable for an OUI via a blockchain -transaction. This range of addresses will be assigned to devices joined to this -OUI and hotspots will be able to do fast and simple 'range routing' to route -packets to their destination OUI. These address ranges should be transferrable -and splittable (down to some minimum size) to allow for address reconfiguration -in the future. A related HIP describes the routing of LoRaWAN join packets. - -Range routing refers to finding which range an address falls into, eg 5 is -between 0 and 9. +subnet of DevAddr addresses will be obtainable for an OUI via a blockchain +transaction. This subnet of addresses will be assigned to devices at the discretion +of OUI's Router (aka: the Network Server). Hotspots will be able to do fast and +simple 'subnet routing' to route packets to their destination OUI. These subnets +should be transferrable and splittable (down to some minimum size) to allow for +address reconfiguration in the future. A related HIP describes the routing of LoRaWAN +join packets. + +Subnets are bought in numbers of bits and blocks ranging from 4-bits to 16-bits are +possible (ie: 16 - 65,536 addresses). Furthermore, location-based aliasing is +encouraged, allowing geographically disparate devices to share the same address. # Motivation [motivation]: #motivation @@ -61,15 +63,13 @@ Anyone routing device traffic on the Helium network. - Corner cases should be dissected by example. We'd introduce new transactions to obtain/transfer/split 'address blocks', -similar to IP address space. These blocks should be available in some fixed +similar to IP address subnets. These blocks should be available in some fixed sizes that are neatly divisible into smaller sizes (eg powers of 2, as in IP address blocks). The size of the allocation would affect the price required to -obtain them. Allocations would consist of {Start and Size} (eg {0, 512} or -{2048, 4096} etc). The blocks would be allocated contiguously, starting from 0, -serialized into a total ordering by the blockchain transaction order. +obtain them. A new class of ledger entries would be added, in a new column family. These -entries would have the big-endian starting address as the key: +entries would have the starting address as the key: `<>` and the corresponding value would be the owning OUI: `<>`. Big endian is used because we can take advantage of rocksdb's lexiographic sorting properties more effectively. @@ -85,36 +85,16 @@ benchmarks have been done: Intel i7-7700HQ ``` -running with 1000 OUIs and 8000000 Devices +running with 1000 OUIs and 10000000 Devices Attempting 1000 random lookups -Average memory 0.25Mb, max 0.29Mb, min -0.04Mb +Average memory 0.36Mb, max 0.42Mb, min 0.06Mb Approximate database size 0.02Mb -Average lookup 0.004s, max 0.1s, min 0.00001s -Lookup misses 0 -``` - -Raspberry Pi 4b -``` -running with 1000 OUIs and 8000000 Devices -Attempting 1000 random lookups -Average memory 0.17Mb, max 0.19Mb, min 0.00Mb -Approximate database size 0Mb -Average lookup 0.04s, max 0.5s, min 0.00004s -Lookup misses 0 +Average lookup 0.00002s, max 0.001s, min 0.00001s ``` -Raspberry Pi 3b+ -``` -running with 1000 OUIs and 8000000 Devices -Attempting 1000 random lookups -Average memory 0.08Mb, max 0.10Mb, min -0.06Mb -Approximate database size 0Mb -Average lookup 0.03s, max 0.4s, min 0.0001s -Lookup misses 0 -``` -The good news is that, on average, this is orders of magnitude fasterthan the -XOR routing scheme for joins, but it is still not free. +This is orders of magnitude faster than the XOR routing scheme for joins, but +it is still not entirely free. # Drawbacks [drawbacks]: #drawbacks @@ -131,8 +111,8 @@ space. This is probably the most important section! - Why is this design the best in the space of possible designs? Mapping all devices in an OUI to a single address has aliasing problems, prefix -routing is harder to sequentially allocate and manage. I'm not sure what else is -suitable. +routing is harder to sequentially allocate and manage. Subnet routing is selected +above range routing due to it being widely familiar. - What other designs have been considered and what is the rationale for not choosing them? @@ -140,7 +120,7 @@ suitable. As mentioned above, aliasing all devices to one address (the OUI number) and prefix routing are the only two candidates I can think of. -Range addressing also allows us to punt on the 33.5 million DevAddr problem for +Subnet addressing also allows us to punt on the 33.5 million DevAddr problem for a while (32 bits of DevAddr minus 7 bits of 'NetID' in the lorawan spec). - What is the impact of not doing this? diff --git a/README.md b/README.md index 4bb3c4569..6b95d77ee 100644 --- a/README.md +++ b/README.md @@ -1,2 +1,3 @@ +[![Build Status](https://travis-ci.com/helium/lorawan-sniffer.svg?token=35YrBmyVB8LNrXzjrRop&branch=master)](https://travis-ci.com/helium/lorawan-sniffer) # HIP Helium Improvement Proposals