low baudrate output

[lestofante]

Describe the bug

When writing to the BLE serial i can only read ~200 byes/s, while when reading i can easily get over 50.000B/s (and my BT firmware seems to be the bottleneck

Log messages
I use a python script with pyserial, ser1.write(data) where data is 512 bytes (MTU exchanges is 247 and from my smartphone i can exceed 110.000 B/s).
I can see a virtual output speed of 72MB/s for write, but seems like this data is parked in a buffer that then get flushed WAY slower:

17:00:47.935 | DEBUG | ble_interface.py: Sending b'nnnnnnnnoooooooooopp'
17:00:48.025 | DEBUG | ble_interface.py: Sending b'ppppppppqqqqqqqqqqrr'
17:00:48.118 | DEBUG | ble_interface.py: Sending b'rrrrrrrrsssssssssstt'
17:00:48.208 | DEBUG | ble_interface.py: Sending b'ttttttttuuuuuuuuuuvv'
17:00:48.295 | DEBUG | ble_interface.py: Sending b'vvvvvvvvzzzzzzzzzzyy'
17:00:48.385 | DEBUG | ble_interface.py: Sending b'yyyyyyyy000000000011

Noticing the potential huge amount of data getting buffred may be the issue, i put delay between write, to force the bitrate to remain around 921600 (my target), and it did not work

I find interesting the library is sending way less data than it should per packet that it should (~20 instead of 247, that sound like the standard MTU)
and way less packet/s than it should (10Hz instead of 500Hz)

Setup (please complete the following information):

• OS: Arch
• Bluetooth Hardware: ID 2550:8761 Realtek Bluetooth Radio
• BlueZ Version: 5.70
• Python Version: 3.11.5
• ble-serial and dependency versions:

ble-serial        2.7.1
bleak             0.21.1
coloredlogs       15.0.1
dbus-fast         2.11.1
humanfriendly     10.0
pip               23.2.1
setuptools        65.5.0
typing_extensions 4.8.0

Additional Context

full test code:

#!/usr/bin/python

import serial
import time
import os
import sys

data = b'aaaaaaaaaabbbbbbbbbbccccccccccddddddddddeeeeeeeeeeffffffffffgggggggggghhhhhhhhhhiiiiiiiiiillllllllllmmmmmmmmmmnnnnnnnnnnooooooooooppppppppppqqqqqqqqqqrrrrrrrrrrssssssssssttttttttttuuuuuuuuuuvvvvvvvvvvzzzzzzzzzzyyyyyyyyyy000000000011111111112222222222\r\n3333333333444444444455555555556666666666777777777788888888889999999999!!!!!!!!!!@@@@@@@@@@##########$$$$$$$$$$%%%%%%%%%%^^^^^^^^^^&&&&&&&&&&**********(((((((((())))))))))__________++++++++++==========]]]]]]]]]]{{{{{{{{{{""""""""""..........,,,,,,,,,,\r\n'

ser1 = serial.Serial(sys.argv[1], timeout=0.1, baudrate=int(sys.argv[2]))
ser2 = serial.Serial(sys.argv[3], timeout=0.1, baudrate=int(sys.argv[4]))

data_out = 0
data_in = 0
check_index = 0
correct = 0
loops = 0
while True:
    loops += 1
    start = time.time()

    print(f"writing {len(data)}")
    complete = False
    ser1.write(data)
    data_out += len(data)

    duration = time.time() - start
    print(f"data out: {data_out} data in: {data_in}, correct {correct} B/s: {len(data) / duration} {len(data)} {loops}")

    time.sleep(0.0054-duration)

    duration = time.time() - start
    print(f"B/s: {len(data) / duration}")

print("end")

edit:
on reception, the library seems to work ok. I can generate 120.000/150.000 bytes/s (confirmed on android smartphone with blueftuit) and i can receive 80.000 on PC with this library (antenna on the pc is not optimal placed) both using TCP and SERIAL mode.
Also tested the TCP mode foroutput, and i got the 200bytes/s max too.

[Jakeler]

Try to set your custom 247 byte MTU with the argument:

  -m MTU, --mtu MTU     Max. bluetooth packet data size in bytes used for sending (default: 20)

In my testing it did about 21 kbit/s (2000 B/s) with a adapter that did not support higher MTUs, so about 100 packets/s. Similar performance for sending and receiving, probably the BLE device was bottlenecking.
You get almost 2 orders of magnitude more receive speed, so you BLE device seems much more capable.

I don't think buffering is the issue, it will write constantly if there is data.

Noticed you have the latest bleak 0.21.1, it changed the default to use write with response, if supported. More reliable but obviously slower. Maybe we should make this configurable via args too. Try:

pip install bleak==0.20.2

[lestofante]

with -m 247 I get between 500 and 750 bytes/s so still quite bad.
using bleak==0.20.2 i get error using -m 247 but work with 240, i get 4296bytes/s and 3500bytes/s with default 20mtu

using -v i can see the data is as expected, and does not seems to slow down the output compared to without

the PC is a ryzen 3600 so i doubt is a issue of cpu bottleneck either

I am very surprised about this asynchronous data rate

[Jakeler]

In case you haven't already seen: the latest version (3.0.0) does not require a bleak downgrade anymore and there is a parameter for writing with response.

Also I analyzed the performance further. Looking directly at the interface from BlueZ:

sudo dbus-monitor --system "destination=org.bluez" "sender=org.bluez"

Found out that ble-serial + bleak actually finishes fast. 1 MB is transferred in less than 4 secs and the heavy dbus traffic stops, means 250 000 bytes/s = 2M baud!
But then there is a buffer in bluez (visible in the memory allocation) - the sending through the bluetooth interface is much slower:

Created by Wireshark 4.4.2

Average bytes/s
5.210

Average bits/s
41 k

Minus overhead also around 4000 bytes/s payload.

Average pps
255.3

Average packet size, B
20

This is interesting too, because I technically set 64 byte MTU.
So it's splitting the packets at 20 bytes anyway and the receiver reassembles it

Receiving as client is faster:

Average bytes/s
15 k

Average bits/s
125 k

The other side is here ble-serial server (peripheral role).

Closing this as not planned, since the bottleneck is definitely not this library.
Maybe BlueZ could be optimized or it's really some hardware limitation.