Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

added MacOS native gui version #19

Merged
merged 60 commits into from
Feb 11, 2021
Merged

Conversation

semvis123
Copy link
Contributor

@semvis123 semvis123 commented Oct 28, 2020

Gui is working and semi disabled because the bluetooth connector isn't working.

Gui is working and semi disabled because the bleutooth connector isn't working.
@semvis123 semvis123 changed the title added macos base, bluetooth connector not working added MacOS base, bluetooth connector not working Oct 28, 2020
@semvis123 semvis123 changed the title added MacOS base, bluetooth connector not working added MacOS base, bluetooth connector not fully working Nov 4, 2020
Copy link
Owner

@Plutoberth Plutoberth left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi, I'm aware that you're not done with this yet, but I just thought to provide some comments. Thanks for working on this PR!

Client/macos/MacOSGUI.cpp Outdated Show resolved Hide resolved
Client/macos/MacOSBluetoothConnector.mm Outdated Show resolved Hide resolved
Client/macos/MacOSBluetoothConnector.mm Outdated Show resolved Hide resolved
Client/macos/MacOSBluetoothConnector.mm Outdated Show resolved Hide resolved
@Plutoberth Plutoberth added the enhancement New feature or request label Nov 13, 2020
Client/macos/main.mm Outdated Show resolved Hide resolved
@semvis123
Copy link
Contributor Author

semvis123 commented Nov 15, 2020

I'm having some trouble writing the recv function, I thought something like this should work:
// in the rfcommChannelData function
printf("%d\n", dataLength);
delegateCPP->receivedBytes = (char*)dataPointer;
delegateCPP->receivedLength = dataLength;

// in the recv function
while(receivedLength != length){
[[NSRunLoop currentRunLoop] runUntilDate:[NSDate dateWithTimeIntervalSinceNow:1]];
}
strcpy(buf, receivedBytes);
return length;

But it doesn't, when I print the receivedlength variable it gives 6, although the expected byte length is 2048 (right?).
I edited the kinda working async bluetoothconnector for this.
bt connector recv test.zip

*edit readability

@semvis123 semvis123 mentioned this pull request Nov 17, 2020
@semvis123
Copy link
Contributor Author

semvis123 commented Nov 19, 2020

I changed the receive function once again, this time I am receiving correct data, at least I was - now I am getting negative numbers, for example: -1 85 2 0 -18 16 -70 46 (I don't know what caused it 🙁 ).
*edit checked today and they are back to normal: 62 0 0 0 0 0 27 0 weird...

But I got another problem, when I block the receive function (recv) to wait for new data, it also blocks the other receive function (rfcommChannelData) too, so it waits infinitely.

bt connector test version.zip

@Plutoberth
Copy link
Owner

Plutoberth commented Nov 20, 2020

I changed the receive function once again, this time I am receiving correct data, at least I was - now I am getting negative numbers, for example: -1 85 2 0 -18 16 -70 46 (I don't know what caused it 🙁 ).
*edit checked today and they are back to normal: 62 0 0 0 0 0 27 0 weird...

But I got another problem, when I block the receive function (recv) to wait for new data, it also blocks the other receive function (rfcommChannelData) too, so it waits infinitely.

bt connector test version.zip

For me it looks like a lot of those problems are caused by the async stuff adding complications. The sync APIs are always simpler, and I think that they will solve a lot of our problems (especially things like unneeded blocking).

The response that you got (with the 62) looks OK, but you didn't catch the end (indicated by the end marker, 62).

I don't like the threaded logic, like in recv with the promises and receivedBytes. It isn't clean and it might've caused the previous issues.

It's kinda difficult for me to test this myself, since I don't have a Mac, but I can try to dive into the docs for openRFCOMMChannelSync myself if it doesn't work out.

P.S. In general, if you need to add something like

while(receivedLength != length){
[[NSRunLoop currentRunLoop] runUntilDate:[NSDate dateWithTimeIntervalSinceNow:1]];
}

in multi-threaded code, there's always a better way to do it.

@varenc
Copy link

varenc commented Dec 17, 2020

Just want to say I compiled this macOS client and gave it a try. I know this is a WIP and while couldn't actually get it to work, it's very promising! Thanks for making this @semvis123. After connecting my headphones over bluetooth they would then showed up in the app instantly. I'd connect, but then the app hangs when I tried to change ambient sound settings. Also when the headphones were connected to my mac and my phone at the same time, with the phone as secondary, I could see that the phone's connection would instantly disappear when I pressed the connect button in this app. So it definitely was succeeding at connecting at some level.

But now weirdly after doing this a few times I've somehow gotten into a state that whenever I press the connect button I get a crash. Unclear what changed. Here's an abbreviated crash report if that's helpful. Looks like something GUI related? 🤷‍♂️

Process:               sonyheadphonesclient [65611]
Path:                  /private/tmp/*/sonyheadphonesclient.app/Contents/MacOS/SonyHeadphonesClient
Identifier:            com.semvis123.SonyHeadphonesClient
Version:               0
Code Type:             X86-64 (Native)
Parent Process:        ??? [1]
Responsible:           sonyheadphonesclient [65611]
User ID:               502

Date/Time:             2020-12-16 19:49:08.013 -0800
OS Version:            Mac OS X 10.15.7 (19H15)
Report Version:        12
Bridge OS Version:     4.6 (17P6610)
Anonymous UUID:        6B776252-4A2E-4EF7-BB88-A2ED3ACE6799

Sleep/Wake UUID:       FB221ACE-371A-48DE-83A3-1AFCAA41FF33

Time Awake Since Boot: 140000 seconds
Time Since Wake:       16000 seconds

System Integrity Protection: enabled

Crashed Thread:        15

Exception Type:        EXC_CRASH (SIGABRT)
Exception Codes:       0x0000000000000000, 0x0000000000000000
Exception Note:        EXC_CORPSE_NOTIFY

Application Specific Information:
terminating with uncaught exception of type char const*
abort() called

Thread 0:: Dispatch queue: com.apple.main-thread
0   libsystem_kernel.dylib        	0x00007fff71ce8e4e semaphore_timedwait_trap + 10
1   libdispatch.dylib             	0x00007fff71b4eb6b _dispatch_sema4_timedwait + 76
2   libdispatch.dylib             	0x00007fff71b4ef97 _dispatch_semaphore_wait_slow + 58
3   com.apple.QuartzCore          	0x00007fff4361a77b -[CAMetalLayer nextDrawable] + 837
4   com.semvis123.SonyHeadphonesClient	0x000000010df52057 EnterGUIMainLoop(BluetoothWrapper) + 1271
5   com.semvis123.SonyHeadphonesClient	0x000000010df51150 main + 224
6   libdyld.dylib                 	0x00007fff71ba7cc9 start + 1

No idea if this is helpful at this stage or not. If it's just noise, feel free to ignore. Cheers!

@semvis123
Copy link
Contributor Author

I have been busy lately so haven't made much progress.
@varenc
The hanging of the app has to do with the recv function not working yet, it is located in one of the macOS Bluetooth files that I added.

About the state in which the app will crash immediately after connect button is pressed. I don't know if I had any issues like that but what I do know is that if stuff acts weirdly, powering off and on your headphones can help.

Btw you might have noticed that the app can take quite some resources on macOS, this is likely because it uses imgui, I will try to make a native ui version as well.

@semvis123
Copy link
Contributor Author

image

Native non functional ui, the connect button could use the native bluetooth device selector.

openRFCOMMChannelSync doesn't appear to have a synchronous/blocking receive. So this function must be made manually, the current function doesn't work because it also blocks the callback of the AsyncCommDelegate (rfcommChannelData), could someone with more C++ / Objc++ experience take a look at this?

bt connector test version.zip

@Plutoberth
Copy link
Owner

Plutoberth commented Dec 18, 2020

The photo looks awesome, but is this implemented not using Dear Imgui?

Additionally, it's never recommended to post code in a zip file. Just push the changes to your branch so everyone could see them here - much easier to collaborate this way.

@semvis123
Copy link
Contributor Author

@Plutoberth This is indeed without dear imgui, instead I use a storyboard using Xcode, because I don't think that it's possible to create a native feeling/looking ui using dear imgui.

Also I posted a zip file because those changes are a draft version specifically for Bluetooth, but I could maybe create a second branch, thanks for your advice.

@Plutoberth
Copy link
Owner

@semvis123 okay. Ideally I prefer to use as little platform specific code as possible, but since this is a toy project whatever works for you is good.

Putting it in a second branch and even a draft PR is okay for even very WIP versions :)

@semvis123
Copy link
Contributor Author

Got the receive function partly working now, it only crashes after a few times🤔

@Plutoberth
Copy link
Owner

Got the receive function partly working now, it only crashes after a few times🤔

What's the cause of the crash?

@semvis123
Copy link
Contributor Author

@Plutoberth EXC_BAD_ACCESS / segmentation fault. (Currently line 145 of MacOSBluetoothConnector.mm)

Could be that I understand the purpose of the function incorrectly, the function currently is supposed to wait until there is enough new data received based on length, is this correct or should it return as soon as it receives new data?(2048 bytes of data seems much to me, especially because it only seems to receive 32 bytes at a time right?)

It currently receives ±4 times and after that forth time it fails to check if it needs more data.

@semvis123
Copy link
Contributor Author

semvis123 commented Jan 16, 2021

Great🎉
Newly compiled version
Is there some application icon that I can add to this?
PS: May I also add such a donation button for the macOS version xD

@Plutoberth
Copy link
Owner

Originally I created some application icon (basically a hand-drawn version of the Sony Headphones app), but decided not to use it because I didn't want to get a C&D from Sony (because it seems like the kind of company that would do that). If you want to create something yourself - go ahead. About the donations - seeing as I reversed the app, created the original version, review PRs, etc, I think that it's fair that all donations go to me. I'm not expecting to get a significant sum from this, if anything; If I get over $100 over the lifetime of the project, which is exceedingly unlikely IMO, I'll work something out so that contributors get their fair share. Otherwise it's just not worth the time.

@semvis123
Copy link
Contributor Author

semvis123 commented Jan 16, 2021

Hmm I'm not very great at creating icons, and without icon would also be fine. We can just skip the icon.

About the donations ...

Yeah that's totally fair in my opinion, I was just joking :)

*edit oops wrong button 🤣

@semvis123 semvis123 closed this Jan 16, 2021
@semvis123 semvis123 reopened this Jan 16, 2021
@Plutoberth
Copy link
Owner

OK. So I'll wait until varenc confirms that it works and then I'll merge it and create a new release.

@Miigon
Copy link

Miigon commented Jan 17, 2021

😄Just tested the macOS version.
Some work needs to be done before it can compile properly and run.

Native GUI version:

  • Add following to info.plist
	<key>NSPrincipalClass</key>
	<string>NSApplication</string>

Otherwise this error occurs when trying to run it:

No Info.plist file in application bundle or no NSPrincipalClass in the Info.plist file, exiting

imGui version:

Make these changes to makefile

  • replace every single ../imgui/examples/ with ../imgui/backends/ (examples is not the correct folder)
  • (line 16) replace ../imgui/imgui_demo.cpp with ../imgui/imgui_tables.cpp (imgui_demo.cpp isn't required, but imgui_tables.cpp is)

After these changes, both version compiles and functions correctly. 🎉Everything, including Focus on Voice, works.

(ps. My bluetooth keyboard also showed up in the device list of the imGui version, I'm not sure if it's supposed to happen or not.)

@semvis123
Copy link
Contributor Author

Thanks for testing👍

  • Add following to info.plist
	<key>NSPrincipalClass</key>
	<string>NSApplication</string>

Didn't came across this one, but should be fixed now.

imGui version:

Make these changes to makefile

  • replace every single ../imgui/examples/ with ../imgui/backends/ (examples is not the correct folder)
  • (line 16) replace ../imgui/imgui_demo.cpp with ../imgui/imgui_tables.cpp (imgui_demo.cpp isn't required, but imgui_tables.cpp is)

This project still uses an older version of imgui where there is no backends folder, would be happy to switch to a newer version, but since the native gui version will be merged I don't think that it is necessary.

(ps. My bluetooth keyboard also showed up in the device list of the imGui version, I'm not sure if it's supposed to happen or not.)

This is expected behavior, but I could change it to filter on SERVICE_UUID but I think that it shouldn't be necessary.
Btw on the topic of SERVICE_UUID, I saw that it is double defined, in IBluetoothConnector.h and Constants.h.

@semvis123 semvis123 changed the title added MacOS base, bluetooth connector fully working added MacOS native gui version Jan 17, 2021
@whazor
Copy link

whazor commented Feb 1, 2021

Thanks for all the work, the mac client looks promising! I tried to increasing ambient sound level and focus on voice but could not get it to work. The following messages might be relevant for debugging:

2021-02-01 10:50:17.353 SonyHeadphonesClient[11577:33169473] Error: [AppleBluetoothHIDDeviceGen2][_setEventService:]Couldn't copy BD_ADDR
2021-02-01 10:50:19.887 SonyHeadphonesClient[11577:33170367] [establishKernelConnection] Received an error from IOServiceOpen() - 0xe00002c7.  NULLing out io_service_t.
2021-02-01 10:50:19.887 SonyHeadphonesClient[11577:33170367] instantiateOnDevice for regular

@semvis123
Copy link
Contributor Author

semvis123 commented Feb 1, 2021

@whazor interesting, have you tried leaving the device selector menu open for ~30 seconds, you will find a pop-up asking for Bluetooth permission.

Please take a look at this link:
https://www.macrumors.com/how-to/reset-mac-bluetooth-module/

The problem doesn't seem to be caused by the client itself, but it looks like a general problem.
Are other Bluetooth devices also experiencing problems?
Have you tried restarting your Mac?

@whazor
Copy link

whazor commented Feb 1, 2021

Resetting the Bluetooth indeed worked. Thanks!

@semvis123
Copy link
Contributor Author

Looks like I forgot to upload the newest compiled version (the one that is also compatible with older macOS versions).
So here it is, the newest compiled version

* improved isconnected function, it now also checks if the channel is open
* fixed the timeout in the connect
- statusbar button added
    - when pressed it changes between ambient sound and noise cancelling
    - still uses placeholder images (couldn't find great ones that are available on older macos versions)
@semvis123
Copy link
Contributor Author

semvis123 commented Feb 4, 2021

I added a statusbar button, when clicked it toggles ambient sound mode.
image This icon means that ambient sound is enabled.
image This icon means that noise cancelling is enabled.
if someone finds better icons it would be great, just used those because they are built-in and compatible with older macOS versions. The statusbar button remains visible after the window is closed, to remove it just quit the program.

I also improved the isconnected function, it now also checks if the channel is open.
And lastly I fixed the timeout in the connect, now it actually does something😅

Latest compiled version with statusbar toggle

@AmgC36
Copy link

AmgC36 commented Feb 4, 2021

I added a statusbar button, when clicked it toggles ambient sound mode.
image This icon means that ambient sound is enabled.
image This icon means that noise cancelling is enabled.
if someone finds better icons it would be great, just used those because they are built-in and compatible with older macOS versions. The statusbar button remains visible after the window is closed, to remove it just quit the program.

I also improved the isconnected function, it now also checks if the channel is open.
And lastly I fixed the timeout in the connect, now it actually does something😅

Latest compiled version with statusbar toggle

Just to check, does this work with WH-1000XM4 as well? Anytime I connect to them once your app is open and then move the slider, the app immediately quits

@semvis123
Copy link
Contributor Author

I added a statusbar button, when clicked it toggles ambient sound mode.
image This icon means that ambient sound is enabled.
image This icon means that noise cancelling is enabled.
if someone finds better icons it would be great, just used those because they are built-in and compatible with older macOS versions. The statusbar button remains visible after the window is closed, to remove it just quit the program.
I also improved the isconnected function, it now also checks if the channel is open.
And lastly I fixed the timeout in the connect, now it actually does something😅
Latest compiled version with statusbar toggle

Just to check, does this work with WH-1000XM4 as well? Anytime I connect to them once your app is open and then move the slider, the app immediately quits

Hmm that's weird, I don't have a wh-1000-xm4 to test it, but could you send the logs by opening the executable within the terminal. (path-to-headphones-client.app)/Contents/MacOS/SonyHeadphonesClient

@AmgC36
Copy link

AmgC36 commented Feb 4, 2021

I added a statusbar button, when clicked it toggles ambient sound mode.
image This icon means that ambient sound is enabled.
image This icon means that noise cancelling is enabled.
if someone finds better icons it would be great, just used those because they are built-in and compatible with older macOS versions. The statusbar button remains visible after the window is closed, to remove it just quit the program.
I also improved the isconnected function, it now also checks if the channel is open.
And lastly I fixed the timeout in the connect, now it actually does something😅
Latest compiled version with statusbar toggle

Just to check, does this work with WH-1000XM4 as well? Anytime I connect to them once your app is open and then move the slider, the app immediately quits

Hmm that's weird, I don't have a wh-1000-xm4 to test it, but could you send the logs by opening the executable within the terminal. (path-to-headphones-client.app)/Contents/MacOS/SonyHeadphonesClient

Here you go:

SonyHeadphonesClient\ 2.app/Contents/MacOS/SonyHeadphonesClient ; exit;
2021-02-04 20:25:13.414 SonyHeadphonesClient[1780:32627] *** Assertion failure in -[TerminateEnabledModalPanel setInitialFirstResponder:], /AppleInternal/BuildRoot/Library/Caches/com.apple.xbs/Sources/AppKit/AppKit-1894.60.100/AppKit.subproj/NSWindow.m:6187
2021-02-04 20:25:13.437 SonyHeadphonesClient[1780:32627] Error: [AppleBluetoothHIDDeviceGen2][_setEventService:]Couldn't copy BD_ADDR
2021-02-04 20:25:13.456 SonyHeadphonesClient[1780:32627] Finding image for "AirPods" (known non-Apple device) with major:0x4 minor:0x1 vendorID:0x5D6 productID:0xA, vendorIDSource:1, colorID:0xFF - isMac:0 isIPad:0 isIPhone:0: "SoundPref.icns"
2021-02-04 20:25:13.460 SonyHeadphonesClient[1780:32627] Finding image for "Bluetooth 3.0 Keyboard" (known non-Apple device) with major:0x5 minor:0x10 vendorID:0x4E8 productID:0x7021, vendorIDSource:2, colorID:0xFF - isMac:0 isIPad:0 isIPhone:0: "KeyboardIcon_2.icns"
2021-02-04 20:25:13.461 SonyHeadphonesClient[1780:32627] Finding image for "Echo Show-9AN" (known non-Apple device) with major:0x4 minor:0x5 vendorID:0x1949 productID:0x1200, vendorIDSource:2, colorID:0xFF - isMac:0 isIPad:0 isIPhone:0: "SoundPref.icns"
2021-02-04 20:25:13.463 SonyHeadphonesClient[1780:32627] Finding image for "WH-1000XM4" (known non-Apple device) with major:0x4 minor:0x1 vendorID:0x54C productID:0xD58, vendorIDSource:2, colorID:0xFF - isMac:0 isIPad:0 isIPhone:0: "SoundPref.icns"
2021-02-04 20:25:13.464 SonyHeadphonesClient[1780:32627] Finding image for "Echo Dot-37W" (known non-Apple device) with major:0x4 minor:0x5 vendorID:0x1949 productID:0x1200, vendorIDSource:2, colorID:0xFF - isMac:0 isIPad:0 isIPhone:0: "SoundPref.icns"
2021-02-04 20:25:13.466 SonyHeadphonesClient[1780:32627] Finding image for "JBL GO" (known non-Apple device) with major:0x4 minor:0x1 vendorID:0xFFFF productID:0x0, vendorIDSource:0, colorID:0xFF - isMac:0 isIPad:0 isIPhone:0: "SoundPref.icns"
2021-02-04 20:25:13.469 SonyHeadphonesClient[1780:32627] Finding image for "Bluetooth Mouse" (known non-Apple device) with major:0x5 minor:0x20 vendorID:0xA5C productID:0x4503, vendorIDSource:2, colorID:0xFF - isMac:0 isIPad:0 isIPhone:0: "Mouse.icns"
2021-02-04 20:25:16.654 SonyHeadphonesClient[1780:32671] [establishKernelConnection] Received an error from IOServiceOpen() - 0xe00002c7. NULLing out io_service_t.
2021-02-04 20:25:16.655 SonyHeadphonesClient[1780:32671] instantiateOnDevice for regular
libc++abi.dylib: terminating with uncaught exception of type RecoverableException: Invalid checksum!
Abort trap: 6
logout
Saving session...
...copying shared history...
...saving history...truncating history files...
...completed.
Deleting expired sessions...10 completed.

[Process completed]

@templeman
Copy link

@AmgC36 I have similar results with the XM4. I've opened issue #27 regarding adding WH-1000XM4 support.

@Plutoberth
Copy link
Owner

Plutoberth commented Feb 7, 2021

Hi again,

I'm going to make some changes to the master branch today (probably), including merging the Linux version. I'll notify you when that happens, and then could you re-test everything works and recompile? After that I'll (finally haha) merge and create a new release. It's just very problematic for me to make changes on the macOS version, so I wanna get a few out of the way right now.

P.S. The statusbar feature is very cool!

Copy link
Owner

@Plutoberth Plutoberth left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I ended up only making changes to the windows specific parts, so I think that we can go ahead and merge this now. LMK if it's OK by you.

@Plutoberth
Copy link
Owner

Also, you can open an issue to create proper icons for the statusbar - someone might step up.

@semvis123
Copy link
Contributor Author

Did some little improvements and cleanup, merging should be okay now👍
Latest version with proper error handling

@Plutoberth
Copy link
Owner

Plutoberth commented Feb 11, 2021

Merging. Closes #6. Thanks for your help!!

@Plutoberth Plutoberth merged commit c03ccd4 into Plutoberth:master Feb 11, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

Successfully merging this pull request may close these issues.

7 participants