NimBLE esp32 server bonding to a 2nd client breaks bond info from the 1st client? #782

benjie-git · 2024-12-06T05:49:57Z

I'm setting up an ESP32-S3 as a peripheral/server connecting to one client at a time. It uses bonding, but I'm having the following problem: My first client can connect, and bonds/pairs fine, and can re-connect fine without re-pairing, but then after I disconnect, and pair to a 2nd client, the 1st client can no longer connect properly.

When the first client is a Mac, after another device pairs and disconnects, the Mac can still partially connect, but receives no data. I believe that it is failing to authenticate.
When the first client is an iPhone, after another device pairs and disconnects, the iPhone is no longer able to authenticate, and so quickly disconnects.

I'm developing in Arduino IDE 2.3.3, with esp32 3.0.7, and NimBLE-Arduino 1.4.2.

    NimBLEServer *pServer = NimBLEDevice::createServer();
    NimBLEDevice::setSecurityAuth(true, true, true);
    NimBLEDevice::setSecurityIOCap(BLE_HS_IO_NO_INPUT_OUTPUT);
    ...
    NimBLEAdvertising *pAdvertising = pServer->getAdvertising();
    pAdvertising->setAppearance(GENERIC_HID);
    pAdvertising->addServiceUUID(serviceUUID);
    pAdvertising->start();

my server callbacks include:

    void onConnect(NimBLEServer *pServer, ble_gap_conn_desc* desc)
    {
        pServer->updateConnParams(desc->conn_handle, 6, 15, 1, 300);
        NimBLEDevice::startSecurity(desc->conn_handle);
    }

Am I missing something that I need to do to convince the esp32 to keep bonding info for multiple clients?

Thank you for your help!

The text was updated successfully, but these errors were encountered:

h2zero · 2024-12-06T15:28:32Z

What is happening is that you have reached the limit of the bonds / cccds available, when this happens the oldest bond is deleted from NVS to make room for the new one. This is adjustable here:

NimBLE-Arduino/src/nimconfig.h

Line 119 in 6e89afc

    
           /** @brief Un-comment to change the number of devices allowed to store/bond with */

I would suggest also fully erasing the flash before uploading to clear all of the stored bonds and start again.

benjie-git · 2024-12-06T19:07:44Z

Thanks, I updated src/nimconfig.h to set CONFIG_BT_NIMBLE_MAX_BONDS to 5 and CONFIG_BT_NIMBLE_MAX_CCCDS to 20. But it is still not working. Do I need to do anything else to rebuild the NimBLE library? It seems like Arduino IDE rebuilds it.

What I'm seeing now is that once I have bonded from my iPhone, and then from my mac, I can still connect from my iPhone without re-pairing, but after connecting from my phone, connections from my mac are broken, and can not read or get notifications.

Thanks for your help, and for any further ideas you might have.

h2zero · 2024-12-06T19:42:01Z

Did you erase the flash? Also close and reopen the IDE to clear the data.

benjie-git · 2024-12-06T19:57:18Z

Thanks! I did erase the flash, and restarted the IDE, and updated to Arduino 2.3.4 and NimBLE-Arduino 1.4.3, but no change. Reconnecting to the iPhone breaks the bond/pair connection to the mac, and re-pairing to the mac breaks the iPhone bond.

I can try to put together a complete, minimal example showing the problem.

h2zero · 2024-12-06T21:23:07Z

I just tested with my phone and an esp32 client and have no issues switching connection back and forth and reading data works as expected with no pairing prompt.

benjie-git · 2024-12-06T22:39:56Z

Thank you for your help. I'm running my ESP32 as a composite HID device that contains just one child device which presents as an XBox game controller. It's based off of the ESP32-BLE-CompisiteHID project, but I've modified it to simplify and remove some unneeded device classes. I don't think I actually need to be using a CompositeHID, so I think I'll try unwrapping the controller device, and using that directly. If that still doesn't work for me, I'll add a minimal project showing the problem, so we can discuss the same exact code. Thanks again.

benjie-git · 2024-12-09T20:40:52Z

This issue I'm seeing is actually that after creating a 2nd bond, and then connecting back to the first device again, the first device sees only empty services with no characteristics.

Similar to some comments in issue #651

h2zero · 2024-12-09T22:39:58Z

@benjie-git please try flashing your device with this: https://github.com/Edzelf/ESP32-Show_nvs_keys after each bond so we can see what the NVS state is.

sanastasiou · 2024-12-10T00:00:51Z

@h2zero I wouldn’t want esp to delete anything by itself if some limits are reached. Is there a way to disallow bonding of new device once we know that we have X number of bonded devices?

h2zero · 2024-12-10T00:16:42Z

The low level callback is global and overridable, have a look at NimBLEDevice::init

benjie-git · 2024-12-10T19:14:43Z

@h2zero Here's the output from Show_nvs_keys after bonding to one device, and then after bonding to a 2nd device.
Keys 000 and 001 look like preferences data.

D: Partition nvs found, 28672 bytes
D: Namespace ID of ESP32Radio is 255
D: Key 000: ControllerESP32
D: Key 001: BleCompisiteHID
D: Key 002: phy
D: Key 003: cal_data
D: Key 064: cal_data
D: Key 065: cal_mac
D: Key 067: cal_mac
D: Key 068: cal_version
D: Key 069: nimble_bond
D: Key 070: pairedAddresses
D: Key 072: pairedAddresses
D: Key 073: our_sec_1
D: Key 077: our_sec_1
D: Key 078: peer_sec_1
D: Key 082: peer_sec_1
D: Key 083: cccd_sec_1
D: Key 085: cccd_sec_1
D: Key 086: cccd_sec_2
D: Key 088: cccd_sec_2


D: Partition nvs found, 28672 bytes
D: Namespace ID of ESP32Radio is 255
D: Key 000: ControllerESP32
D: Key 001: BleCompisiteHID
D: Key 002: phy
D: Key 003: cal_data
D: Key 064: cal_data
D: Key 065: cal_mac
D: Key 067: cal_mac
D: Key 068: cal_version
D: Key 069: nimble_bond
D: Key 073: our_sec_1
D: Key 077: our_sec_1
D: Key 078: peer_sec_1
D: Key 082: peer_sec_1
D: Key 083: cccd_sec_1
D: Key 085: cccd_sec_1
D: Key 086: cccd_sec_2
D: Key 088: cccd_sec_2
D: Key 089: cccd_sec_3
D: Key 091: cccd_sec_3
D: Key 092: pairedAddresses
D: Key 094: pairedAddresses
D: Key 095: our_sec_2
D: Key 099: our_sec_2
D: Key 100: peer_sec_2
D: Key 104: peer_sec_2
D: Key 105: cccd_sec_4
D: Key 107: cccd_sec_4
D: Key 108: cccd_sec_5
D: Key 110: cccd_sec_5

h2zero · 2024-12-10T21:41:07Z

All looks good there, both bonds are stored...

h2zero · 2024-12-10T21:47:16Z

I've been able to partially reproduce this with an esp32c6 and using esp idf with esp-nimble-cpp. what I found is that this appears to be an apple bug, in order to get the services to show again I had to turn bluetooth off and on again, then connect.

sanastasiou · 2024-12-10T21:49:20Z

@h2zero in my case it was an android phone which was connecting and was showing no characteristics.. however that was a plain esp32.. With a C3 I could not reproduce the issue.

benjie-git · 2024-12-10T21:50:40Z

@h2zero Thanks. I've been restarting Bluetooth on both the mac and the iPhone too, but unfortunately that hasn't helped.
Do you think this might be an issue specific to BLE HID devices?

h2zero · 2024-12-10T21:52:09Z

There seems to be a trend with the HID devices, though I am currently testing with an HID device firmware at the moment as well.

h2zero · 2024-12-11T01:44:57Z

It would seem we need a common code to use for troubleshooting this, here is what I am using now:

#include <NimBLEDevice.h>
#include <NimBLEHIDDevice.h>
#include <HIDTypes.h>

#define HID_REPORT_ID 0x01

static const uint8_t HID_REPORT_DESC[] =
{
	// CONSUMER BUTTON
	USAGE_PAGE(1),		0x0C,			// USAGE_PAGE (Consumer)
	USAGE(1),		0x01,			// USAGE (Consumer Control)
	COLLECTION(1),		0x01,			// COLLECTION (Application)
	REPORT_ID(1),		HID_REPORT_ID,		// REPORT_ID (1)
	LOGICAL_MINIMUM(1),	0x00,			// LOGICAL_MINIMUM (0)
	LOGICAL_MAXIMUM(1),	0x01,			// LOGICAL_MAXIMUM (1)
	REPORT_SIZE(1),		0x01,			// REPORT_SIZE (1)
	REPORT_COUNT(1),	0x08,			// REPORT_COUNT (8)
	USAGE(1),		0xE9,			// Volume Increase
	USAGE(1),		0xEA,			// Volume Decrease
	USAGE(1),		0xE2,			// Volume Mute
	USAGE(1),		0x40,			// Menu
	USAGE(1),		0xCD,			// Play
	USAGE(1),		0xB7,			// Stop
	USAGE(1),		0xB5,			// Next Track
	USAGE(1),		0xB6,			// Prev Track
	HIDINPUT(1),		0x02,			// INPUT (Data,Var,Abs,No Wrap,Linear,Preferred State,No Null Position)
	END_COLLECTION(0)				// END COLLECTION
};

class ServerCallbacks : public NimBLEServerCallbacks {
    void onConnect(NimBLEServer* pServer, NimBLEConnInfo& connInfo) override {
        printf("Client address: %s\n", connInfo.getAddress().toString().c_str());

        /**
         *  We can use the connection handle here to ask for different connection parameters.
         *  Args: connection handle, min connection interval, max connection interval
         *  latency, supervision timeout.
         *  Units; Min/Max Intervals: 1.25 millisecond increments.
         *  Latency: number of intervals allowed to skip.
         *  Timeout: 10 millisecond increments.
         */
        pServer->updateConnParams(connInfo.getConnHandle(), 24, 48, 0, 18);
        NimBLEDevice::startSecurity(connInfo.getConnHandle());
    }

    void onDisconnect(NimBLEServer* pServer, NimBLEConnInfo& connInfo, int reason) override {
        printf("Client disconnected - start advertising\n");
        NimBLEDevice::startAdvertising();
    }
} scb;

extern "C" void app_main()
{
	printf("Starting app_main\n");

	NimBLEDevice::init("HIDTestDev");
	NimBLEDevice::setSecurityAuth(true, false, false);

	NimBLEServer *pServer = NimBLEDevice::createServer();
    pServer->setCallbacks(&scb);

	NimBLEHIDDevice * pHidDevice = new NimBLEHIDDevice(pServer);
	pHidDevice->setManufacturer("DeviceManuTest");
	pHidDevice->setPnp(0x02, 0x0B0B, 0x0C0C, 0x0101);
	pHidDevice->setHidInfo(0x00, 0x01);
	pHidDevice->getInputReport(HID_REPORT_ID);
	pHidDevice->setReportMap((uint8_t*)HID_REPORT_DESC, sizeof(HID_REPORT_DESC));
	pHidDevice->startServices();

	NimBLEAdvertising *pAdvertising = NimBLEDevice::getAdvertising();
	pAdvertising->setAppearance(0x03C1);	//HID Keyboard icon
	pAdvertising->addServiceUUID(pHidDevice->getHidService()->getUUID());
	pAdvertising->start();
}

Other than the Apple weirdness it seems to work perfectly with 2 paired devices. What does happen though is that when HID devices are automatically reconnected to whichever device is fastest to establish, preference is given to the last successfully connected peer. In this case both the Mac and iPhone are trying to connect when the device starts advertising, the last one connected to it wins, the other stays in a state of connection but unconfirmed so no access to the services is provided.

Details: A device advertises, 2 devices looking for it try to connect, the device that last connected to the advertiser is given preference by the BLE stack, which makes sense. The way BLE connections work is there is the initial connect command sent, then the confirmation via command, unfortunately this is problem as there is no confirmation event, the only confirmation is sending a command and getting a response. So if the 2 devices in question send the connect command at the same time the last device to successfully connect will connect and the other device will sit in a "connected", but not confirmed state, so no services etc. are available.

Testing has shown me that the only way for this to work is to have the connection parameters set with loose enough connection parameters that 2 devices can remain connected, which reduces performance. Also connecting 1 HID device to multiple peers seems problematic at best, which device is being controlled?

The problem here I would suggest is that, because this is and HID device, Apple, and probably every other manufacturer, once bonded will always look for the device in order to connect as fast as possible and in the background, before any apps connect. On my phone the BLE stack in the phone connected before the app could even get a scan result from it, low level action.

benjie-git · 2024-12-11T02:25:01Z

Thank you @h2zero
I did a quick conversion of your example to NimBLE v1.4.3, and to an Arduino sketch, included below. This example works fine for me, so I'll try to counter with an example that shows my issue.

#include <NimBLEDevice.h>
#include <NimBLEHIDDevice.h>
#include <HIDTypes.h>


#define HID_REPORT_ID 0x01

static const uint8_t HID_REPORT_DESC[] =
{
    // CONSUMER BUTTON
    USAGE_PAGE(1),		0x0C,			// USAGE_PAGE (Consumer)
    USAGE(1),		0x01,			// USAGE (Consumer Control)
    COLLECTION(1),		0x01,			// COLLECTION (Application)
    REPORT_ID(1),		HID_REPORT_ID,		// REPORT_ID (1)
    LOGICAL_MINIMUM(1),	0x00,			// LOGICAL_MINIMUM (0)
    LOGICAL_MAXIMUM(1),	0x01,			// LOGICAL_MAXIMUM (1)
    REPORT_SIZE(1),		0x01,			// REPORT_SIZE (1)
    REPORT_COUNT(1),	0x08,			// REPORT_COUNT (8)
    USAGE(1),		0xE9,			// Volume Increase
    USAGE(1),		0xEA,			// Volume Decrease
    USAGE(1),		0xE2,			// Volume Mute
    USAGE(1),		0x40,			// Menu
    USAGE(1),		0xCD,			// Play
    USAGE(1),		0xB7,			// Stop
    USAGE(1),		0xB5,			// Next Track
    USAGE(1),		0xB6,			// Prev Track
    HIDINPUT(1),		0x02,			// INPUT (Data,Var,Abs,No Wrap,Linear,Preferred State,No Null Position)
    END_COLLECTION(0)				// END COLLECTION
};

class ServerCallbacks : public NimBLEServerCallbacks {
    void onConnect(NimBLEServer *pServer, ble_gap_conn_desc* desc) {
        printf("Client address: %s\n", ((std::string)(NimBLEAddress(desc->peer_ota_addr))).c_str());
        pServer->updateConnParams(desc->conn_handle, 24, 48, 0, 18);
        NimBLEDevice::startSecurity(desc->conn_handle);
    }

    void onDisconnect(NimBLEServer *pServer, ble_gap_conn_desc* desc) {
        printf("Client disconnected - start advertising\n");
        NimBLEDevice::startAdvertising();
    }
} scb;


void setup() {
    printf("Starting app_main\n");

    NimBLEDevice::init("HIDTestDev");
    NimBLEDevice::setSecurityAuth(true, false, false);

    NimBLEServer *pServer = NimBLEDevice::createServer();
    pServer->setCallbacks(&scb);

    NimBLEHIDDevice * pHidDevice = new NimBLEHIDDevice(pServer);
    pHidDevice->manufacturer("DeviceManuTest");
    pHidDevice->pnp(0x02, 0x0B0B, 0x0C0C, 0x0101);
    pHidDevice->hidInfo(0x00, 0x01);
    pHidDevice->inputReport(HID_REPORT_ID);
    pHidDevice->reportMap((uint8_t*)HID_REPORT_DESC, sizeof(HID_REPORT_DESC));
    pHidDevice->startServices();

    NimBLEAdvertising *pAdvertising = NimBLEDevice::getAdvertising();
    pAdvertising->setAppearance(0x03C1);	//HID Keyboard icon
    pAdvertising->addServiceUUID(pHidDevice->hidService()->getUUID());
    pAdvertising->start();

    // Add battery level, as a quick way to check that characteristics are readable
    pHidDevice->setBatteryLevel(100);
    pHidDevice->batteryLevel()->notify();
}


void loop() {
    delay(10);
}

benjie-git · 2024-12-12T23:39:50Z

@h2zero After a bit of cleanup based on your example code, I'm realizing that the ESP32's multiple bonds do seem fine. It turns out that a different part of my testing process is to blame for most of my problem. Turning the Mac's main Bluetooth switch off, and then later back on, seems to break something about the Mac's pairing info. I was turning that off to release the connection from the Mac, so that I could connect from the iPhone without rushing. But I'm testing now without turning off the Mac's Bluetooth, and I'm able to connect back and forth between the Mac and the iPhone, and the bond continues to work fine, and all services populate with characteristics.

The remaining problem is minor. It's just that turning off the Mac's bluetooth entirely seems to corrupt something about the connection, and no amount of rebooting the ESP32 or connecting and disconnecting seems to help, until I re-pair. I think the bond itself is actually ok, because it connects and onAuthenticationComplete() still gets called, with sec_state.encrypted == true. Maybe some service info is getting cached wrong on the mac?

I am not seeing this problem with your example HID code -- only with my own. So I feel that there is likely still room for improvement on my end.

h2zero · 2024-12-13T00:13:18Z

Glad you're making progress, strange issue for sure.

benjie-git · 2024-12-17T01:21:41Z

Wow, I think my main remaining issue was just setting my connection params too tight.

I was using:

server->updateConnParams(desc->conn_handle, 6, 7, 0, 80);

and calling notify on my HID input characteristic every 8ms, and was having lots of trouble with reconnection.

When I switch to:

server->updateConnParams(desc->conn_handle, 12, 24, 0, 80);

and sending updates every 15ms, those weird problems seem to have resolved themselves. I haven't run any overnight tests yet, but it's looking fine so far.

h2zero · 2024-12-17T02:53:07Z

@benjie-git Yes, connection parameters are SUPER important. Hope it works well.

benjie-git · 2024-12-17T18:56:07Z

Another related question: My HID descriptor is large at 283 bytes.
It looks to me like the m_reportMapChr characteristic in NimBLEHIDDevice.cpp gets set up with a default max Characteristic size of 20. Does this need to be re-created with a bigger max size in NimBLEHIDDevice::setReportMap?

Edit: I'll start a new issue about this one.

sanastasiou · 2024-12-26T18:55:59Z

Wow, I think my main remaining issue was just setting my connection params too tight.

I was using:
server->updateConnParams(desc->conn_handle, 6, 7, 0, 80);
and calling notify on my HID input characteristic every 8ms, and was having lots of trouble with reconnection.

When I switch to:
server->updateConnParams(desc->conn_handle, 12, 24, 0, 80);
and sending updates every 15ms, those weird problems seem to have resolved themselves. I haven't run any overnight tests yet, but it's looking fine so far.

Sorry to be hijacking the thread. Where exactly do u call this? Upon connection or during a connection is established?

benjie-git · 2024-12-26T19:57:54Z

@sanastasiou I'm calling this in my onConnect() callback

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

NimBLE esp32 server bonding to a 2nd client breaks bond info from the 1st client? #782

NimBLE esp32 server bonding to a 2nd client breaks bond info from the 1st client? #782

benjie-git commented Dec 6, 2024

h2zero commented Dec 6, 2024 •

edited

Loading

benjie-git commented Dec 6, 2024

h2zero commented Dec 6, 2024

benjie-git commented Dec 6, 2024

h2zero commented Dec 6, 2024

benjie-git commented Dec 6, 2024

benjie-git commented Dec 9, 2024 •

edited

Loading

h2zero commented Dec 9, 2024

sanastasiou commented Dec 10, 2024

h2zero commented Dec 10, 2024

benjie-git commented Dec 10, 2024 •

edited

Loading

h2zero commented Dec 10, 2024

h2zero commented Dec 10, 2024

sanastasiou commented Dec 10, 2024

benjie-git commented Dec 10, 2024

h2zero commented Dec 10, 2024

h2zero commented Dec 11, 2024 •

edited

Loading

benjie-git commented Dec 11, 2024

benjie-git commented Dec 12, 2024

h2zero commented Dec 13, 2024

benjie-git commented Dec 17, 2024

h2zero commented Dec 17, 2024

benjie-git commented Dec 17, 2024 •

edited

Loading

sanastasiou commented Dec 26, 2024

benjie-git commented Dec 26, 2024

NimBLE esp32 server bonding to a 2nd client breaks bond info from the 1st client? #782

NimBLE esp32 server bonding to a 2nd client breaks bond info from the 1st client? #782

Comments

benjie-git commented Dec 6, 2024

h2zero commented Dec 6, 2024 • edited Loading

benjie-git commented Dec 6, 2024

h2zero commented Dec 6, 2024

benjie-git commented Dec 6, 2024

h2zero commented Dec 6, 2024

benjie-git commented Dec 6, 2024

benjie-git commented Dec 9, 2024 • edited Loading

h2zero commented Dec 9, 2024

sanastasiou commented Dec 10, 2024

h2zero commented Dec 10, 2024

benjie-git commented Dec 10, 2024 • edited Loading

h2zero commented Dec 10, 2024

h2zero commented Dec 10, 2024

sanastasiou commented Dec 10, 2024

benjie-git commented Dec 10, 2024

h2zero commented Dec 10, 2024

h2zero commented Dec 11, 2024 • edited Loading

benjie-git commented Dec 11, 2024

benjie-git commented Dec 12, 2024

h2zero commented Dec 13, 2024

benjie-git commented Dec 17, 2024

h2zero commented Dec 17, 2024

benjie-git commented Dec 17, 2024 • edited Loading

sanastasiou commented Dec 26, 2024

benjie-git commented Dec 26, 2024

h2zero commented Dec 6, 2024 •

edited

Loading

benjie-git commented Dec 9, 2024 •

edited

Loading

benjie-git commented Dec 10, 2024 •

edited

Loading

h2zero commented Dec 11, 2024 •

edited

Loading

benjie-git commented Dec 17, 2024 •

edited

Loading