Skip to content

Commit

Permalink
Merge pull request #17 from Jakeler/bleak-backend
Browse files Browse the repository at this point in the history
Switch to Bleak backend
  • Loading branch information
Jakeler authored Apr 14, 2021
2 parents c6562e1 + 293b5d3 commit 0f3bb3d
Show file tree
Hide file tree
Showing 22 changed files with 646 additions and 367 deletions.
3 changes: 0 additions & 3 deletions .gitmodules
Original file line number Diff line number Diff line change
@@ -1,3 +0,0 @@
[submodule "bluepy-repo"]
path = bluepy-repo
url = https://github.com/edwios/bluepy.git
207 changes: 143 additions & 64 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,116 +2,186 @@
A tool to connect Bluetooth 4.0+ Low Energy to UART modules and normal PCs/laptops/RaspberryPi.
It fulfills the same purpose as `rfcomm bind` for the old Bluetooth 2.0, creating a virtual serial port in `/dev/pts/x`, which makes it usable with any terminal or application.

### Installation
## Installation
### Standard (via Python Package Index)
The software is written completely in Python and packaged as module, so it can be easily installed with pip:
```console
$ pip install ble-serial
```
pip install ble-serial
pip install git+https://github.com/edwios/bluepy.git@10f1cee90afb416f139949b86b491e4cfa98c886
```
If you are wondering why the second command is required: It depends on the bluepy library, but unfortunately there are [bugs](https://github.com/IanHarvey/bluepy/issues/253) in the original version and there was no development since a year, so it is important to specifically install this fork with a few fixes.

Now you should have 2 new scripts: `ble-scan` and the main `ble-serial`.

Note: To be able to do device scans without using `sudo` or root, you must grant the `bluepy-helper` binary additional [capabilities/permissions](https://github.com/IanHarvey/bluepy/issues/313#issuecomment-428324639). Follow the steps outlined below:
On Linux you ready now and can directly jump to the usage section!

Find bluepy-helper (typically ~/.local/lib/python3.6/site-packages/bluepy/bluepy-helper).
Give it permissions so you don't have to run scripts with sudo:
```sh
sudo setcap 'cap_net_raw,cap_net_admin+eip' bluepy-helper`
### From source (for developers)
You can clone the repository with:
```console
$ git clone https://github.com/Jakeler/ble-serial.git
```

### Finding devices
First make sure the bluetooth adapter is enabled, for example with `bluetoothctl power on`, then the scan function can be used (note: root is required for this step, if you have not added the capabilities above):
Then switch branches, make changes etc...
Make sure the dependencies are installed, I recommend to use a virtualenv like this:
```console
$ python -m venv ble-venv
$ source ble-venv/bin/activate
$ pip install -r requirements.txt
```
# ble-scan

The package can be either started directly with `-m`:
```console
$ python -m ble_serial ARGUMENTS # Main tool = ble-serial
$ python -m ble_serial.scan # BLE scan = ble-scan
$ python -m ble_serial.setup_com0com # Windows only setup = ble-setup
```

Or installed with `pip` from the current directory:
```console
$ pip install .
```
and started as usual.

### Additional steps for Windows
Windows does not have a builtin feature to create virtual serial ports (like Linux does), so it is required to install a additional driver. I decided to use the open source `com0com` Null-modem emulator, downloaded from [here](https://sourceforge.net/projects/signed-drivers/files/com0com/v3.0/) as signed version. This is required because unsigned drivers can not be installed anymore. Note that on latest Windows 10 you likely still have to disable secure boot for it to work.

ble-serial includes the `ble-setup` script to make the `com0com` configuration easier:
```console
> ble-setup.exe -h
usage: ble-setup [-h] [--install-path INSTALL_PATH]

Setup required COM port pair

optional arguments:
-h, --help show this help message and exit
--install-path INSTALL_PATH
Installation directory of the null modem emulator (default: C:/Program Files (x86)/com0com/)
```
Discovered device: 20:91:48:4c:4c:54 -> UT61E - JK
...
Found 2 devices!

Device 20:91:48:4c:4c:54 (public), RSSI=-58 dB
01: Flags = 06
ff: Manufacturer = 484d2091484c4c54
16: 16b Service Data = 00b000000000
02: Incomplete 16b Services = 0000ffe0-0000-1000-8000-00805f9b34fb
09: Complete Local Name = UT61E - JK
0a: Tx Power = 00
It will request administrator privileges (if it does not already have it) and setup the port in another CMD window:
```
Changing into C:/Program Files (x86)/com0com/
> Checking port list for BLE
CNCA0 PortName=-
CNCB0 PortName=-
BLE port does not exist
> Checking port list for COM9
CNCA0 PortName=-
CNCB0 PortName=-
> Trying to create port pair
CNCA1 PortName=COM9
CNCB1 PortName=BLE
ComDB: COM9 - logged as "in use"
Setup done!
Device ...
Hit any key to close
```
The output is a list of the recognized nearby devices. After the MAC address it prints out the device name, if it can be resolved.
As you can see it created the `BLE`<->`COM9` pair. ble-serial will internally connect to `BLE`, users can then send/receive the data on `COM9`

Otherwise there exist multiple proprietary serial port emulators, these should work too. Just manually create a pair that includes a port named `BLE`.

## Usage
### Finding devices
First make sure the bluetooth adapter is enabled, for example with `bluetoothctl power on`, then the scan function can be used:
```console
$ ble-scan

Started BLE scan

20:91:48:4C:4C:54 (RSSI=-56): UT61E - JK
...

If there are no devices found it might help to increase the scan time. All discoverable devices must actively send advertisements, to save power the interval of this can be quite long, so then try for example 30 seconds.
Finished BLE scan
```
The output is a list of the recognized nearby devices. After the MAC address and signal strength it prints out the device name, if it can be resolved.

If there are no devices found it might help to increase the scan time. All discoverable devices must actively send advertisements, the interval of this can be quite long to save power, so try for example 30 seconds in this case.
```console
$ ble-scan -h
usage: ble-scan [-h] [-t SEC] [-d ADDR]

Scanner for BLE devices and service/characteristics.

optional arguments:
-h, --help show this help message and exit
-t SEC, --scan-time SEC
Duration of the scan in seconds (default: 5.0)
-d, --deep-scan Try to connect to the devices and read out the service/characteristic UUIDs (default: False)
-d ADDR, --deep-scan ADDR
Try to connect to device and read out service/characteristic UUIDs
(default: None)
```
On Bluetooth 2.0 there was a "serial port profile", with 4.0 BLE there is unfortunately no standardized mode anymore, every chip manufacturer chooses their own ID to implement the features.

On Bluetooth 2.0 there was a "serial port profile", with 4.0 - 5.2 (BLE) there is unfortunately no standardized mode anymore, every chip manufacturer chooses their own ID to implement the features.
```py
'0000ff02-0000-1000-8000-00805f9b34fb', # LithiumBatteryPCB adapter
'0000ffe1-0000-1000-8000-00805f9b34fb', # TI CC245x (HM-10, HM-11)
```
Some usual IDs are included in ble-serial, these will be tried automatically if nothing is specified.
You might skip this part and start directly with the connection.

Otherwise to find the correct ID, use the deep scan option, it will go through the devices and shows all provided interfaces. This scan can take long, especially if there are many devices in the area, so only use it if you want to find the right write characteristic ID.
```
# ble-scan -d
```
```
Device ...
...
Service: 00001800-0000-1000-8000-00805f9b34fb
Characteristic: 00002a00-0000-1000-8000-00805f9b34fb READ
Characteristic: 00002a01-0000-1000-8000-00805f9b34fb READ
Characteristic: 00002a02-0000-1000-8000-00805f9b34fb READ WRITE
Characteristic: 00002a03-0000-1000-8000-00805f9b34fb READ WRITE
Characteristic: 00002a04-0000-1000-8000-00805f9b34fb READ
Service: 00001801-0000-1000-8000-00805f9b34fb
Characteristic: 00002a05-0000-1000-8000-00805f9b34fb INDICATE
Service: 0000ffe0-0000-1000-8000-00805f9b34fb
Characteristic: 0000ffe1-0000-1000-8000-00805f9b34fb READ WRITE NO RESPONSE NOTIFY
```
Now in addition to the previous output there are all characteristics listed, grouped into services. The characteristics in the first service starting with `00002a` are not interesting in this case, because they are standard values (for example the device name), if you want to know more look at [this list](https://gist.github.com/sam016/4abe921b5a9ee27f67b3686910293026#file-allgattcharacteristics-java-L57).
Otherwise to find the correct ID, use the deep scan option. It expects a device MAC address, connects to it and reads out all services/characteristic/descriptors:
```console
$ ble-scan -d 20:91:48:4C:4C:54
Started deep scan of 20:91:48:4C:4C:54

SERVICE 00001801-0000-1000-8000-00805f9b34fb (Handle: 12): Generic Attribute Profile
CHARACTERISTIC 00002a05-0000-1000-8000-00805f9b34fb (Handle: 13): Service Changed ['indicate']
DESCRIPTOR 00002902-0000-1000-8000-00805f9b34fb (Handle: 15): Client Characteristic Configuration
SERVICE 0000ffe0-0000-1000-8000-00805f9b34fb (Handle: 16): Vendor specific
CHARACTERISTIC 0000ffe1-0000-1000-8000-00805f9b34fb (Handle: 17): Vendor specific ['read', 'write-without-response', 'notify']
DESCRIPTOR 00002902-0000-1000-8000-00805f9b34fb (Handle: 19): Client Characteristic Configuration
DESCRIPTOR 00002901-0000-1000-8000-00805f9b34fb (Handle: 20): Characteristic User Description

After the (U)ID the permissions are listed. We are searching for a characteristic that allows writing = sending to the device, the only candidate in here is `0000ffe1-0000-1000-8000-00805f9b34fb` (spoiler: a HM-11 module again).
Completed deep scan of 20:91:48:4C:4C:54
```
Now the interesting parts are the characteristics, grouped into services. The ones belows the first service starting with `00002` are not interesting in this case, because they are standard values (for example the device name), if you want to know more look at [this list](https://gist.github.com/sam016/4abe921b5a9ee27f67b3686910293026#file-allgattcharacteristics-java-L57).

After the ID, handle and type the permissions are listed in []. We are searching for a characteristic that allows writing = sending to the device, the only candidate in here is `0000ffe1-0000-1000-8000-00805f9b34fb` (spoiler: a HM-11 module again).
Same procedure with the read characteristic, this modules handle read and write through the same characteristic, but some other chips split it.


### Connecting a device
The `ble-serial` tool itself has a few more options:
```
```console
$ ble_serial -h
usage: __main__.py [-h] [-v] -d DEVICE [-t {public,random}] [-i ADAPTER] [-m MTU] [-w WRITE_UUID] [-l FILENAME] [-b]
[-p PORT] [-r READ_UUID]

Create virtual serial ports from BLE devices.

optional arguments:
-h, --help show this help message and exit
-v, --verbose Increase verbosity to log all data going through (default: False)
-d DEVICE, --dev DEVICE
BLE device address to connect (hex format, can be seperated by colons) (default: None)
-t {public,random}, --address-type {public,random}
BLE address type, either public or random (default: public)
-i ADAPTER, --interface ADAPTER
BLE host adapter number to use (default: 0)
BLE host adapter number to use (default: hci0)
-m MTU, --mtu MTU Max. bluetooth packet data size in bytes used for sending (default: 20)
-w WRITE_UUID, --write-uuid WRITE_UUID
The GATT chracteristic to write the serial data, you might use "scan.py -d" to find it out (default: None)
The GATT characteristic to write the serial data, you might use "ble-scan -d" to find it out
(default: None)
-l FILENAME, --log FILENAME
Enable optional logging of all bluetooth traffic to file (default: None)
-b, --binary Log data as raw binary, disable transformation to hex. Works only in combination with -l (default: False)
-b, --binary Log data as raw binary, disable transformation to hex. Works only in combination with -l (default:
False)
-p PORT, --port PORT Symlink to virtual serial port (default: /tmp/ttyBLE)
-r READ_UUID, --read-uuid READ_UUID
The GATT characteristic to subscribe to notifications to read the serial data (default: None)

```
Only the device address is always required:
```
```console
$ ble-serial -d 20:91:48:4c:4c:54
```
```
21:02:55.823 | INFO | virtual_serial.py: Slave created on /tmp/ttyBLE -> /dev/pts/8
21:02:56.410 | INFO | interface.py: Connected device 20:91:48:4c:4c:54
21:02:56.909 | INFO | interface.py: Receiver set up
21:02:56.909 | INFO | __main__.py: Running main loop!
18:36:09.255 | INFO | linux_pty.py: Slave created on /tmp/ttyBLE -> /dev/pts/7
18:36:09.255 | INFO | ble_interface.py: Receiver set up
18:36:09.258 | INFO | ble_interface.py: Trying to connect with 20:91:48:4C:4C:54
18:36:12.291 | INFO | ble_interface.py: Device 20:91:48:4C:4C:54 connected
18:36:12.637 | INFO | main.py: Running main loop!
```
This log shows a successful start, the virtual serial port was opened on `/dev/pts/8`, the number at the end changes, depending on how many pseudo terminals are already open on the system. In addition it creates automatically a symlink to `/tmp/ttyBLE`, so you can easily access it there always on the same file, the default can be changed with the `-p`/`--port` option.

Expand All @@ -129,18 +199,27 @@ $ ble-serial -d 20:91:48:4c:4c:54 -r 0000ffe1-0000-1000-8000-00805f9b34fb
```

Also there is an option to log all traffic on the link to a text file:
```
```console
$ ble-serial -d 20:91:48:4c:4c:54 -l demo.txt
cat demo.txt
```
```
...
$ cat demo.txt
2019-12-09 21:15:53.282805 <- BLE-OUT: 48 65 6c 6c 6f 20 77 6f 72 6c 64
2019-12-09 21:15:53.491681 -> BLE-IN: b0 b0 b0 b0 b0 b0 3b b0 b0 b0 ba b0 0d 8a
2019-12-09 21:15:53.999795 -> BLE-IN: b0 b0 b0 b0 b0 b0 3b b0 b0 b0 ba b0 0d 8a
```
Per default it is transformed to hex bytes, use `-b`/`--binary` to log raw data.

You can use `-v` to increase the log verbosity to DEBUG:
```console
18:31:25.136 | DEBUG | ble_interface.py: Received notify from 17: bytearray(b'\xb0\xb0\xb0\xb0\xb0\xb0;\xb0\xb0\xb0\xba\xb0\r\x8a')
18:31:25.136 | DEBUG | linux_pty.py: Write: bytearray(b'\xb0\xb0\xb0\xb0\xb0\xb0;\xb0\xb0\xb0\xba\xb0\r\x8a')

18:31:25.373 | DEBUG | linux_pty.py: Read: b'hello world'
18:31:25.373 | DEBUG | ble_interface.py: Sending b'hello world'
```
This will log all traffic going through. Note that everything shows up two times, because it goes through the ble module and then into the serial port and vice versa.

As always, i hope it was helpful. If you encounter problems, please use the issue tracker on [GitHub](https://github.com/Jakeler/ble-serial/issues).

### Known limitations
* Higher bitrates: 9600 bit/s is well tested and works fine. 19200 and higher can cause data loss on longer transmissions.
* Chromium 73+ based applications, including NW.js/electron desktop apps, for example current Betaflight/INAV Configurator: Connection to the virtual serial port (pty) fails. This is because of explicit whitelisting in chromium.
8 changes: 8 additions & 0 deletions ble_serial/__init__.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
import platform

if platform.system() == 'Linux':
from ble_serial.ports.linux_pty import UART as platform_uart
elif platform.system() == 'Windows':
from ble_serial.ports.windows_com0com import COM as platform_uart
else:
raise Exception('Platform not supported!')
69 changes: 2 additions & 67 deletions ble_serial/__main__.py
Original file line number Diff line number Diff line change
@@ -1,69 +1,4 @@
import logging, sys, argparse, time
from ble_serial.virtual_serial import UART
from ble_serial.interface import BLE_interface
from ble_serial.fs_log import FS_log, Direction
from bluepy.btle import BTLEDisconnectError

def main():
parser = argparse.ArgumentParser(formatter_class=argparse.ArgumentDefaultsHelpFormatter,
description='Create virtual serial ports from BLE devices.')

parser.add_argument('-v', '--verbose', dest='verbose', action='store_true',
help='Increase verbosity to log all data going through')
parser.add_argument('-d', '--dev', dest='device', required=True,
help='BLE device address to connect (hex format, can be seperated by colons)')
parser.add_argument('-t', '--address-type', dest='addr_type', required=False, choices=['public', 'random'], default='public',
help='BLE address type, either public or random')
parser.add_argument('-i', '--interface', dest='adapter', required=False, default='0',
help='BLE host adapter number to use')
parser.add_argument('-w', '--write-uuid', dest='write_uuid', required=False,
help='The GATT chracteristic to write the serial data, you might use "scan.py -d" to find it out')
parser.add_argument('-l', '--log', dest='filename', required=False,
help='Enable optional logging of all bluetooth traffic to file')
parser.add_argument('-b', '--binary', dest='binlog', required=False, action='store_true',
help='Log data as raw binary, disable transformation to hex. Works only in combination with -l')
parser.add_argument('-p', '--port', dest='port', required=False, default='/tmp/ttyBLE',
help='Symlink to virtual serial port')
parser.add_argument('-r', '--read-uuid', dest='read_uuid', required=False,
help='The GATT characteristic to subscribe to notifications to read the serial data')
args = parser.parse_args()

logging.basicConfig(
format='%(asctime)s.%(msecs)03d | %(levelname)s | %(filename)s: %(message)s',
datefmt='%H:%M:%S',
level=logging.DEBUG if args.verbose else logging.INFO
)

try:
uart = UART(args.port)
bt = BLE_interface(args.device, args.addr_type, args.adapter, args.write_uuid, args.read_uuid)
if args.filename:
log = FS_log(args.filename, args.binlog)
bt.set_receiver(log.middleware(Direction.BLE_IN, uart.write_sync))
uart.set_receiver(log.middleware(Direction.BLE_OUT, bt.send))
else:
bt.set_receiver(uart.write_sync)
uart.set_receiver(bt.send)
logging.info('Running main loop!')
uart.start()
while True:
bt.receive_loop()
except BTLEDisconnectError as e:
logging.warning(f'Bluetooth connection failed')
except KeyboardInterrupt:
logging.info('Keyboard interrupt received')
except Exception as e:
logging.error(f'Unexpected Error: {e}')
finally:
logging.warning('Shutdown initiated')
if 'uart' in locals():
uart.stop()
if 'bt' in locals():
bt.shutdown()
if 'log' in locals():
log.finish()
logging.info('Shutdown complete.')
exit(0)
from ble_serial.main import launch

if __name__ == '__main__':
main()
launch()
Loading

0 comments on commit 0f3bb3d

Please sign in to comment.