Pushing Images / Files

[superchase9000]

Is there any current or planned support for pushing files?

[immanuelfodor]

Great idea! I haven't got any use case for it yet that's why it wasn't implemented but it can be definitely done: https://matrix-nio.readthedocs.io/en/latest/examples.html#sending-an-image

The only thing need to be figured out is how to get the file and its stats from the post message without creating a temp file (if it's possible, not to spam the container's filesystem nor have the responsibility of cleanup).

[immanuelfodor]

Two links for future reference:

• https://github.com/Tinche/aiofiles#tempfile
• https://us-pycon-2019-tutorial.readthedocs.io/aiohttp_file_uploading.html#handle-uploaded-image

[superchase9000]

@immanuelfodor Nice! Thanks for looking into this. This is wonderful project.

[neilwoodwards]

Fantastic project well done. Is there a way to send json strings without quotations surrounding it?

[immanuelfodor]

Thanks!

Could you please provide an example what would you want to achieve? My first intuition is that the JSON should be parseable by Python. Second guess is that curl can be parameterized with a data file, so the JSON could be passed this way without extra quotes.

[neilwoodwards]

Could you please provide an example what would you want to achieve? My first intuition is that the JSON should be parseable by Python. Second guess is that curl can be parameterized with a data file, so the JSON could be passed this way without extra quotes.

When I send curl -d "world" http://localhost:8000/post/token it returns world: '' Rather, I just want a single string with not quotations or colons like world.

Also, when I send a webhook from iOS (Shortcuts) it appears as message: coming to you from shortcuts instead of coming to you from shortcuts.

Just trying to send strings so that its just the string when it appears in element.io

My .env file has the following:

MESSAGE_FORMAT=yaml USE_MARKDOWN=False ALLOW_UNICODE=True DISPLAY_APP_NAME=False

[immanuelfodor]

Oh, I see what you mean by extra quotes. It's just a coincidence that it is a valid YAML according to the Python YAML parser. The built-in parser appends the extra quotes as the world key doesn't have a value, hence the world: ''. It would be interesting to see what happens if you set the message format env to raw (I can't try this right now) as you want to send in simple strings.

[immanuelfodor]

As for the iOS Shortcuts, it probably sends a JSON like this:

{
    "message": "coming to you from shortcuts"
} 

Or a form-data payload like this:

message=coming to you from shortcuts

So I think this one is normal (feature, not bug), the gateway doesn't alter the payload it receives by stripping out elements (keys or values). What you want to achieve here is some custom functionality based on sender token or payload structure (to remove the message key).

[neilwoodwards]

message=coming to you from shortcuts

So I think this one is normal (feature, not bug), the gateway doesn't alter the payload it receives by stripping out elements (keys or values). What you want to achieve here is some custom functionality based on sender token or payload structure (to remove the message key).

Ah ok. That explains it. Thank you. Do you have anything you can link me regarding to stripping out the JSON elements?

[immanuelfodor]

Oh, it's just plain old Python, you can alter the message anywhere between the payload decode and the send message function call: https://github.com/immanuelfodor/matrix-encrypted-webhooks/blob/main/src/WebhookServer.py#L55

If you'll use the gateway for only Shortcuts, you can replace the message formatting with data = data.message here, so the send function will get the value string only instead of the full string JSON dump:

matrix-encrypted-webhooks/src/WebhookServer.py

Line 78 in 4fda6c0

data = self._format_message(message_format, allow_unicode, data)

[neilwoodwards]

Receives this (which is fine):
2022-11-13 10:32:35,216 | INFO | module:root | Received raw data: "Test"

But then prints this out:
2022-11-13 10:32:35,428 | INFO | module:root | @user in Room | Test\n...\n

Where is this new information coming from?

[immanuelfodor]

The second log line is the message sent to Matrix by the bridge received back in the bridge as a new Matrix message event. The bridge receives a webhook -> the bridge logs the payload -> the bridge sends it as a message to the configured room -> Matrix sends the message event (back) to the bridge -> the bridge logs the message event. The last two steps also happen when a regular user sends a message in the room, and it could provide a capability later to manage the bridge from the room via messages/commands.

Do you see the \n...\n part of the message in Matrix as well? So does your message appear as

Test
...

in your Matrix client as well?

[neilwoodwards]

Do you see the \n...\n part of the message in Matrix as well? So does your message appear as

Test
...

in your Matrix client as well?

Yep. It comes up as that. See here: https://ibb.co/hV5GJw4

Is there a way so it's just Test?

[immanuelfodor]

Well, I tested it, and I can't see any extra ... around the message.

$ curl -d "world" https://example.com/post/TOKEN
{"success": true}

See the image for the Matrix output:

Even the logs are fine:

...
2022-12-16 11:30:26,470 | INFO | module:root | We synced, token: XXXXXXXXX_XXXXXXXXXX_XXXXXXXXXXX
2022-12-16 11:30:27,242 | INFO | module:root | We synced, token: XXXXXXXXX_XXXXXXXXXX_XXXXXXXXXXX
2022-12-16 11:30:29,194 | INFO | module:root | Received raw data: world
2022-12-16 11:30:29,217 | INFO | module:aiohttp.access | 10.42.1.40 [16/Dec/2022:11:30:29 +0000] "POST /post/TOKEN HTTP/1.1" 200 180 "-" "curl/7.74.0"
2022-12-16 11:30:29,243 | INFO | module:root | @WEBHOOK_USER in ROOM | world

The settings I used were the same as yours:

  MESSAGE_FORMAT: "raw"
  USE_MARKDOWN: "False"
  DISPLAY_APP_NAME: "False"
  ALLOW_UNICODE: "True"

[neilwoodwards]

Thanks @immanuelfodor

What about sending JSON webhooks? When I send:

{ "message": "Hello World" }

I receive the following:

The settings I am using for this is:

MESSAGE_FORMAT: "json"
USE_MARKDOWN: "False"
DISPLAY_APP_NAME: "False"
ALLOW_UNICODE: "True"

Is there a way to only extract the Hello World?

[immanuelfodor]

Yes, the answer is at #2 (comment) 😄

[immanuelfodor]

Note to self:

Regarding the original idea of pushing images and files, Apprise is considering a Matrix notification channel overhaul at caronc/apprise#795 . When they implement this, the custom E2E client written in this repo could be replaced by Apprise, just the same way we use that in the RocketChat gateway (https://github.com/immanuelfodor/rocketchat-push-gateway).

We depend on two issues of Apprise to be completed before it can be integrated: caronc/apprise#356 for attachment support and caronc/apprise#305 for E2E support. When this happens, the Matrix webhook bridge's API can mainly stay the same and the project could become a wrapper of Apprise to provide the necessary (containerized) REST API to accept webhooks.