Skip to content
This repository has been archived by the owner on Aug 15, 2024. It is now read-only.

Support XEP-0167 (voice calls) #135

Open
jlaba opened this issue Jun 26, 2022 · 28 comments
Open

Support XEP-0167 (voice calls) #135

jlaba opened this issue Jun 26, 2022 · 28 comments

Comments

@jlaba
Copy link

jlaba commented Jun 26, 2022

Support at least for audio calls via XEP-0167 would be nice. Is it already on the roadmap?

@geobra
Copy link
Owner

geobra commented Jun 27, 2022

Agree. I also want to have that :-)! But this is more difficult then it seems to be. And given the current man-power, it does not look like this feature will be implemented in the near future...

@jlaba
Copy link
Author

jlaba commented Jul 5, 2022

Can you outline, what needs to be done?
May be I find some time to start the implementation.

@geobra
Copy link
Owner

geobra commented Jul 12, 2022

I have not looked into the details yet. But this is a starting point:
https://xmpp.org/extensions/xep-0166.html

@jlaba
Copy link
Author

jlaba commented Jul 15, 2022

Thanks! I have read through the things a little bit. I understood, that a minimal implementation needs

  • XEP-0166 - Jingle Basics
  • XEP-0167 - Jingle RTP Sessions (Voice or Video Calls)
  • XEP-0176 or XPE-0177 - (both solutions to handle the data session)

In swiften there seems to be no support for this extensions. Since swiften seems to be quite unmaintained, may be a switch over to qxmpp library would be worth thinking about. Following https://doc.qxmpp.org/qxmpp-dev/xep.html Qxmpp supports the XEP feature list of shmoose like this:

  • XEP-0184: Message Delivery Receipts - ok
  • XEP-0199: XMPP Ping - ok
  • XEP-0363: HTTP File Upload - ok
  • XEP-0333: Chat Markers - ok
  • XEP-0313: Message Archive Management - ok
  • XEP-0359: Unique and Stable Stanza IDs - ok
  • XEP-0045: Multi-User Chat - ok
  • XEP-0280: Message Carbons - ok
  • XEP-0198: Stream Management - ok
  • XEP-0384: OMEMO Encryption - only partially yet

and it also supports

  • XEP-0166: Jingle
  • XEP-0167: Jingle RTP Sessions
  • XEP-0176: Jingle ICE-UDP Transport Method

Of course this is no shortterm solution, but neverthless: What is your opinion about it?

@geobra
Copy link
Owner

geobra commented Jul 15, 2022

Agree. There is not much action on Swiften. But to switch to qxmpp is a major rewrite of a lot of the code base. Maybe this step is necessary for new features. Currently, I don't have the resources to do that.

If you want to help, maybe there is a dual approach.
First, test if qxmpp is build- and runable on SFOS. Then, try to use qxmpp side by side with the swiften framework in shmoose. By this, shmoose could use qxmpp for new features and step by step rewrite the 'old' code to also use qxmpp.

@jlaba
Copy link
Author

jlaba commented Jul 16, 2022

I have tried to build it. v1.1.0 works with Qt5.6. v1.2.0 already requires Qt5.7.
Since v1.1.0 contains already the class QXmppCallManager it should work with this verison.

I have built it with:

../sailfishOS/bin/sfdk cmake . -- -DBUILD_TESTS=OFF -DBUILD_DOCUMENTATION=OFF -DBUILD_EXAMPLES=OFF
../sailfishOS/bin/sfdk make

v1.1.0 not supports the following XEPs. But shmoose is only affected by XEP-0359.

  • Starting with v1.2.0 (requires >Qt5.7.0; ABI Change)
    • XEP-0077: In-band registration
    • XEP-0231: Bits of Binary
  • Starting with v1.3.0 (requires >Qt5.7.0; complys with the XMPP Compliance Suites 2020)
    • XEP-0245: The /me Command
    • XEP-0357: Push Notifications: Enable/disable IQ
    • XEP-0359: Unique and Stable Stanza IDs
    • XEP-0428: Fallback Indication
  • Starting with v.1.4.0 (requires >Qt5.7.0; requires C++17; compliance suites for 2021 - reached without changes)
    • XEP-0224: Attention manager
    • MIX-MISC XEP-0407: Invitation message extension

@geobra
Copy link
Owner

geobra commented Jul 18, 2022

I did not expect to get a newer Qt version soon at SFOS. And with Qt 6, they changed their licensing model. And Jolla probably has to pay license fees, if they want to switch to that v6 version.

This having said, going with qxmpp seems a dead-end on SFOS. Maybe we should switch to nemo-mobile as a target OS ;-). Even if we go with v1.1.0, if there is a security risk, we can't just update to a newer version.

From a technical point of view...
With something like "swiften::client->onDataRead" you can get all the received xml as a string stream. Is it possible to forward that to qxmpp?

@jlaba
Copy link
Author

jlaba commented Jul 23, 2022

This problem with the license / Qt was new for me. So I read only a little bit about it. From my point of view, Qt could be also in the future an alternative. Because either Sailfish OS will solve the licensing issues and update to a newer version or it will switch to another library, but than it would be no problem to ship Qt as an additional rpm. I personally would put my coins on the first one.

Assuming, either way will be the future, the problem is, getting security fixes and enhancements for qxmpp meanwhile.

@ron282
Copy link
Contributor

ron282 commented Aug 10, 2022

Hi, I've been able to get a start of an implementation supporting XEP-0166/XEP-0167/XEP-0176 with Swiften. Swiften already supports Jingle. I added classes to get RTP transport description and ICE-UDP. This is at an embryonic stage. Therefore to make it work I had to make a small patch to Swiften. I'm able to get debug traces when a call is received.

@geobra
Copy link
Owner

geobra commented Aug 11, 2022

Sounds good. It's a first step. Please get us updated :-).

@ron282
Copy link
Contributor

ron282 commented Aug 14, 2022

Work is in progress. It takes time to write all the classes in the same manner as Swiften, and then a Stun client, and then to connect to speaker and to microphone. I'd like to get it work with Conversations: SRTP - DTLS (XEP-0320) has to be supported. So it adds some complexity.

@geobra
Copy link
Owner

geobra commented Sep 3, 2022

Just a thought. But maybe it could also be an option to have a look, what is needed to get qxmpp to build also with the Qt version Sailfish supports. Maybe there are only a few places in the qxmpp code where a decision needs to be placed about the target Qt version and add some code for that version. The guys from qxmpp had to agree to accept those changes as a merge request. Someone could ask them ;-).

But with this, in my opinion, it would make sense to start a new SFOS xmpp client from scratch. To refactor shmoose away from swiften to only use qxmpp is a major task. I can't handle this with the amount of available time. And it would bring the possibility to have a test-driven development from the very beginning. Shmoose is now at a point, where every change in the code could break something. And the (integration) test coverage of about 60% is not enough. It needs a test for every class, which is not there at the moment.

@geobra
Copy link
Owner

geobra commented Sep 10, 2022

For those of you (@jlaba ?) who want to play around with qxmpp, if have cloned the repo into qxmpp-sfos and inserted a little patch file to make it build on sfos. Use it with:

patch -p1 <sfos.patch

@ron282
Copy link
Contributor

ron282 commented Sep 11, 2022

You mean patch -p1 <sfos.diff :)

@ron282
Copy link
Contributor

ron282 commented Sep 11, 2022

I compiled easily the QXMPP library and I ran a couple of examples successfully.
I tried to compile the OMEMO extension but I had few issues that may need additional lines in the sfos.diff because of differences in feature sets between Qt 5.6 and Qt 5.14:

  • fromSecsSinceEpoch(a) has to be replaced by fromMSecsSinceEpoch(a*1000ul)
  • toSecsSinceEpoch() has to be replaced by toMSecsSinceEpoch()/1000
  • Qt::SkipEmptyParts replaced by QString::SkipEmptyParts
  • Qt::KeepEmptyParts replaced by QString::KeepEmptyParts
  • QOverload by static_cast...
  • qAsConst by Q_FOREACH

But I'm blocked when I need to use the libomemo-c library in Qxmpp. I never know how to proceed to make the library accessible to Qxmpp for compiling...

@geobra
Copy link
Owner

geobra commented Sep 12, 2022

That was also my next step on my todo list, to test the omemo build ;-). Do you have a patch with your changes?
I can have a look how to get it linked against libomemo-c.

@geobra
Copy link
Owner

geobra commented Sep 13, 2022

Ok, I built libomemo-c for sfos and patched the qxmpp cmake files to use this library.
On my side, there is an issue about qca-qt5. This library does not seem to be official provided from jolla.
I asked this guy [1] for also providing the -devel packages.

[1] https://openrepos.net/content/ruedigergad/qca-qt5

@geobra
Copy link
Owner

geobra commented Sep 14, 2022

Maybe it make sense to just build it [1] by our own. I will try it...

[1] https://github.com/KDE/qca

@geobra
Copy link
Owner

geobra commented Sep 15, 2022

I build qca 2.2.1 and libomemo-c from master.

Now, I also get a number of error messages in qxmpp with omemo enabled. The omemo stuff is implemented by using even more modern c++ then SFOS can handle. Let me think about if it is worth to follow this path and patch away the modern constructs by old workarounds...

@geobra
Copy link
Owner

geobra commented Sep 15, 2022

If you want to give it a try, I have updated the diff file. CMakeLists of qxmpp is updated to build against the local builds. Use this steps (out of my head, adjust as needed ;-))

mkdir xmpp && cd xmpp

wget https://github.com/KDE/qca/archive/refs/tags/v2.2.1.tar.gz
tar -xvf ...
cd qca-2.2.1
mkdir build && cd build
set TESTS to off in CMakeLists.txt
sb2 -t SailfishOS-4.2.0.21-aarch64 cmake ..
sb2 -t SailfishOS-4.2.0.21-aarch64 make
cd ..

https://github.com/dino/libomemo-c
cd libomemo-c
add 'set(CMAKE_POSITION_INDEPENDENT_CODE ON)' to CMakeLists.txt
mkdir build && cd build
sb2 -t SailfishOS-4.2.0.21-aarch64 cmake ..
sb2 -t SailfishOS-4.2.0.21-aarch64 make
cd ..

git clone https://github.com/geobra/qxmpp-sfos
cd qxmpp-sfos
patch -p1 < sfos.diff
sb2 -t SailfishOS-4.2.0.21-aarch64 cmake -DBUILD_EXAMPLES=false -DBUILD_TESTS=false -DBUILD_OMEMO=true ..
sb2 -t SailfishOS-4.2.0.21-aarch64 make

@geobra
Copy link
Owner

geobra commented Sep 16, 2022

Done.

With the latest commit of the sfos.diff, qxmpp with enabled omemo builds for sfos.

Where to go from here with Shmoose, @ron282 ? As @jlaba correct pointed out, Swiften did not get new features or bug fixes for quite a while now. The patches to qxmpp are small and clear. But can become bigger and more complex in the future. From my point of view, there are some options:

  • Stay with Swiften and implement all new features within Shmoose context
  • Change to qxmpp
    -- develop a patch for each new release, to get it build on sfos
    -- try to get the patches for sfos to qxmpp mainline (if they have interest in)

And if we want to switch to qxmpp

  • restart from scratch with a new xmpp client for sfos
  • try to refactor the existing code base of shmoose to use qxmpp
  • try to get Kaidan build for sfos and 'just' exhange the kirigami ui with sfos silicia ui

Lots of stuff to decide...

@ron282
Copy link
Contributor

ron282 commented Sep 16, 2022

That's a great news. I look at the qxmmp api and it looks clean and much more easier to integrate with Shmoose. I looked also at Swiften for audio calls and I spent some time trying to figure out what needs to be done. This is quite complex. I'd rather go with adapting Shmoose for qxmpp but I let you decide! If I can help i would be happy to help for that.

@ron282
Copy link
Contributor

ron282 commented Sep 18, 2022

I did a Shmoose version (qxmpp branch) which is using qxmpp for roster. It works.

@geobra
Copy link
Owner

geobra commented Sep 22, 2022

Okay, lets try it with a simple prove of concept first. With a goal to have the roster and be able to send simple 1to1 messages. Thats it, no other stuff or XEPs. With this, we will get a feeling about the effort.

I will create a new project, based on the Shmoose Master. And exclude most of the classes from the build. Based on that, try to exchange swiften with qxmpp. Then we can start by exchanging the API calls and get back the actual functionality.

Give me some days to prepare this.

@geobra
Copy link
Owner

geobra commented Oct 1, 2022

Just a quick feedback. Excluding stuff and getting the MVP compiling is ongoing. Had to think about how to structure it best. But in the end, the architecture will not change much and qxmpp will be included in a similar way as swiften was...

@geobra
Copy link
Owner

geobra commented Oct 2, 2022

Here is a cloned repo from Shmoose.
https://github.com/geobra/shmong

It assumes to have the build qxmpp lib ready, parallel to the shmong directory.
The shmong repo is cleaned up as much as possible. It builds, runs with a LD_LIBRARY_PATH pointing to qxmpp build. It can connect to a xmpp server via the Desktop build and should be able to send messages. For that to work, the refactored RosterController must be ported. The persistence should be stable, but that needs to be checked.

But @ron282 ...

During the work of refactoring the last days, I noticed that I did not have the time to really actively work on this. In the old times I started shmoose, about 6 years ago, I was more in programming then today. So, if we want to go to try this change with qxmpp, I can assist you with hints on how things work. The main coding stuff would be on you.

Saying that, think about if you want to 'own' the new xmpp client as a project, hosted on your github account. Or if I should add you as a main developer to the Shmong repo.

What is your opinion about that all?

@geobra
Copy link
Owner

geobra commented Oct 4, 2022

Hi @ron282

I manually merged a lot of your code from your harbour-shmoose -> qxmpp branch into shmong. The Desktop client is now working with this. The roster is processed and displayed. And it is possible to send and receive 1to1 messages.

Maybe we should stop now discussing here and open a new thread in an appropriate repository ;-).

@ron282
Copy link
Contributor

ron282 commented Oct 5, 2022

Hi @geobra , that's very good news! On my side i'm trying now since 3 weeks to create .spec files to compile libomemo, qca and qxmpp. I'm using sfdk to compile. I'm not a cmake expert so it's a slopy learning curve. I'm sucessfull with libomemo but i'm blocked with qca. Now you have something running i will have a look. This is more intrresting! Ok to.continue silewhere the discussion but we will have to come back later. Everything started because of calls!

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants