Scan peripherals and filter by service UUID

[joelviel]

Hi,

Is it possible to scan peripherals and only consider those with a specific service UUID?
BLE addresses change often if using Android peripheral, what is your solution to detect compliant peripherals with your central in that case?

I am currenty using NodeJS noble ( noble.startScanning(uuid) ) but I am looking for alternative

Thanks

[bsiever]

By OS:

• The Linux backend appears to allow this via an optional filters dictionary (I haven't tested it).
• PR Updates to CoreBluetooth backend #209 should add identical support for CoreBluetooth too. (I have tested some of it)
• I don't know much about the Windows backend, but it doesn't look like it does filtering yet.

The BlueZ documentation includes a section on SetDiscoveryFilters that gives the keywords allowed and their meaning. See: https://www.mideiros.net/doc/system/packages/bluez/dbus-apis/adapter-api.txt (Search for SetDiscoveryFilter)

Here's an example of scanning in the CoreBluetooth version:

    scanner = bleak.BleakScanner(filters={"UUIDs":["1d93af38-9239-11ea-bb37-0242ac130002"], "DuplicateData":False})
    scanner.register_detection_callback(detection_callback)
    await scanner.start()
    await asyncio.sleep(3.0)
    await scanner.stop()
    devices = await scanner.get_discovered_devices()

[dlech]

Is there a compelling reason (like performance) that the filtering should be done at the OS level?

I've been thinking it would be nice to have an async iterator in the scanner. This could be used to implement filters in Python and it would not be restricted to the lowest common denominator of OS capabilities. (Off topic: this feature would also nicely replace register_detection_callback())

For example, get the first device with matching service UUID (ignore that it doesn't match the current API - this is more like pseudo-code):

MY_UUID = 'xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx'

async with BleakScanner() as scanner:
    # this has an iteration every time a new advertisement is seen
    async for advertisement in scanner.scan(timeout=5.0):
        if advertisement.service_uuids and MY_UUID in advertisement.service_uuids:
            my_device_id = advertisement.device_id
            # Breaking out of the for loop stops scanning. If there were any advertisements
            # already received after this one, they are ignored.
            break
    else:
        # The for loop completed without `break` so we must have reached timeout.
        print('device not found!')
        exit(1)

    my_device = await scanner.connect(my_device_id)
    ...

Or for all matching devices:

MY_UUID = 'xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx'

device_ids = []

async with BleakScanner() as scanner:
    async for advertisement in scanner.scan(timeout=5.0):
        if not advertisement.service_uuids:
            continue
        if MY_UUID not in advertisement.service_uuids:
            continue
        device_ids.append(advertisement.device_id)

    if not device_ids:
        print('no devices found!')
        exit(1)

    devices = await asyncio.gather(*(scanner.connect(x) for x in device_ids))
    ...

[hbldh]

As @bsiever says, there is OS level scan filtering available:

• In Linux, this is set via the filters keyword in the discovery method or via filters keyword in the constructor or set_scanning_filter method on the BleakScannerBlueZDBus.
• In Windows, it is available via filters keyword in the constructor or set_scanning_filter method on the BleakScannerDotNet class.
• I will merge parts of Updates to CoreBluetooth backend #209 shortly, among other things the OS level scanning filter methods. It will be available on the corresponding objects in the Core Bluetooth backend.

@dlech My sole reason for leaving the scan filtering on the OS level is so that I would not have to support more code. Since it is already available there, I thought that it would be sufficient with providing access to those methods and leave it at that. The BleakScanner object is also meant for people to be able to implement their own filtering solutions, with user defined callbacks in which I thought one could evaluate if a detected device should be filtered or not.

Interesting that you put the connect method on the Scanner object. I have this morning thought that it would have been better to merge the scanning to the BleakClient and use that for everything, removing the costly double implementations of most things...

[bsiever]

@hbldh I thought that scanning may merit merging into the client too, but I haven't looked at all three clients. On the CoreBluetooth end this may make it easier to remove the singleton/cbapp.

It may also be worth considering people who are using bleak for observers that purely scan and process advertisements --- I don't know if there's a downside to folding scanning into clients for that use-case.

[dhalbert]

One issue with bluez scanning is that the kernel support always turns on de-duplication of advertisements. This makes it difficult or impossible to use multiple advertisements to transmit changing data in a connectionless way. See https://marc.info/?l=linux-bluetooth&m=158225950522806&w=2. We are lobbying to allow control over this.

For now, we have gotten around this problem by using hcitool or bluetoothctl to initiate scanning, and hcidump to look at the raw data. However, this requires adding privileges to those commands. See
https://github.com/adafruit/Adafruit_Blinka_bleio#support-for-duplicate-advertisement-scanning-on-linux
for more details, and
https://github.com/adafruit/Adafruit_Blinka_bleio/blob/master/_bleio/adapter_.py
for some code, particularly _use_hcitool() and _start_scan_hcitool().

I do not expect bleak to provide this hack, but the restriction might be worth noting in the docs.

[dlech]

Interesting that you put the connect method on the Scanner object. I have this morning thought that it would have been better to merge the scanning to the BleakClient and use that for everything, removing the costly double implementations of most things...

Don't put any importance on that. I was just being lazy and didn't look up the existing API. 😄

[hbldh]

@dhalbert Even if bleak provided the hcitool / hcidump methods in python, it would still require some setcap solution in the system as well, doesn't it? A core idea of bleak initially was to provide a "pip only" solution to achieving BLE capability in Python, and I feel reluctant to add things that require tinkering outside of the Python program.

Please open a issue where this BlueZ scanning discussion can be had!

[dhalbert]

@dhalbert Even if bleak provided the hcitool / hcidump methods in python, it would still require some setcap solution in the system as well, doesn't it? A core idea of bleak initially was to provide a "pip only" solution to achieving BLE capability in Python, and I feel reluctant to add things that require tinkering outside of the Python program.

Right, I agree about that. We only did what we had to, after this unpleasant discovery. The setcap solution is not elegant or desired; the right thing is for bluez kernel support to be fixed.

[joelviel]

Thank you for your numerous responses

I think on Windows the filters "UUIDs" when calling BleakScanner has no effect.
I am surprised about connecting performance (BleakClient.connect()), it takes always more than 4.5 seconds when I test, is it normal? (It is usually faster with other solutions)

import asyncio
import bleak
import uuid
import time

my_service_uuid = "0000fff0-0000-1000-8000-00805f9b34fb"
my_characteristic_uuid = '0000fff5-0000-1000-8000-00805f9b34fb'
my_peripheral_name = 'dtcBLE'
scanner = bleak.BleakScanner(filters={"UUIDs":[my_service_uuid], "DuplicateData":False})

def notification_handler(sender, data):
    print(f"{sender}: {data}")
    
def detection_callback(*args):
    print('LTPPP', args[1].Advertisement.LocalName)

async def run():
    
    scanner.register_detection_callback(detection_callback)
    target_peripheral_addr = None
    while(target_peripheral_addr is None):
        
        await scanner.start()
        await asyncio.sleep(0.7)
        await scanner.stop()
        devices = await scanner.get_discovered_devices()
        
        #devices = await bleak.discover() // slower alternative to detect peripherals
        
        for d in devices:
            print(d.address, d.name, d.metadata)
            if d.name==my_peripheral_name and my_service_uuid in d.metadata['uuids']:
                target_peripheral_addr = d.address
            else: print ('Target peripheral not found')
    
    print("compliant peripheral found!")
    t0 = time.clock()
    client = bleak.BleakClient(target_peripheral_addr, loop=loop)
    await client.connect()
    print('perf', time.clock() -t0)
    
loop = asyncio.get_event_loop()
loop.run_until_complete(run())

[hbldh]

@joelviel No, the UUIDs filter option does not exist in Windows. Look at the docstring for the Windows backend Scanner: https://github.com/hbldh/bleak/blob/master/bleak/backends/dotnet/scanner.py#L51-L58

You have to do something like

from System import Guid
from Windows.Devices.Bluetooth.Advertisement import BluetoothLEAdvertisementFilter, BluetoothLEAdvertisement 

adv_filter = BluetoothLEAdvertisementFilter()
adv_filter.Advertisement = BluetoothLEAdvertisement()
adv_filter.Advertisement.ServiceUuids = [Guid.Parse("0000fff0-0000-1000-8000-00805f9b34fb")]

bleak.BleakScanner(AdvertisementFilter=adv_filter)

first.

I haven't tried the code above, merely writing it from documentation. Try it out.

Evidently, scanning filters need better documentation and some examples...