Allow sending of binary WebSocket messages in StompSubProtocolHandler [SPR-12475]

[spring-projects-issues]

Adam Granger opened SPR-12475 and commented

The changes made on #16907
(4a29e16#diff-7bc1370febf168db39f9b3a608f68fe8) only adds support for receiving websocket messages from a client.

I would like to see support for binary messages from server to client - the method handleMessageToClient(...). Currently this only supports the one case, TextMessage

byte[] bytes = this.stompEncoder.encode(stompAccessor.getMessageHeaders(), (byte[]) message.getPayload());
session.sendMessage(new TextMessage(bytes));

---

Affects: 4.1.2

Reference URL: http://stackoverflow.com/questions/26213819/is-it-possible-to-send-binary-data-with-stomp-over-websockets-using-spring-webso

Issue Links:

• Allow binary WebSocket messages in StompSubProtocolHandler [SPR-12301] #16907 Allow binary WebSocket messages in StompSubProtocolHandler

Referenced from: commits 670974d, 51367de

1 votes, 7 watchers

[spring-projects-issues]

Rossen Stoyanchev commented

The question is how decide on Text vs BinaryMessage? Perhaps we could check the content-type header. Does the message have one?

[spring-projects-issues]

Rossen Stoyanchev commented

Adam Granger, to avoid an incomplete solution (like with #16907) I am going to set the fix version to 4.1.4 to allow more time to properly test and verify a potential solution. Or if we manage it this week, we can still fold it in 4.1.3.

I've created a branch called "binary-messages" in my own fork which contains a proposed fix. Would you be able to clone and install that in your local Maven repository to try out? If you do something like the below, you should end up with 4.1.3.BUILD-SNAPSHOT installed in your local maven repo to work against.

$ mkdir spring-framework
$ cd spring-framework
$ git clone git@github.com:rstoyanchev/spring-framework.git
$ git checkout binary-messages
$ ./gradlew install -x javadoc

Alternatively perhaps you could provide a small repro project (see here) and I can work with you on that.

Basically my expectation is you would create a custom MessageConverter to serialize to JSON and then encode to Base64. For example extend MappingJackson2MessageConverter and override convertToInternal as follows:

public Object convertToInternal(Object payload, MessageHeaders headers) {
    return Base64Utils.encode((byte[]) super.convertToInternal(payload, headers));
}

You'd also need to set the content-type to "application/octet-stream" probably by overriding getDefaultContentType .

[spring-projects-issues]

Adam Granger commented

Thanks for looking at this, I was hoping for "true" binary websocket messages, i.e. sending of raw byte data. The current framework already allows sending of Base64 encoded data via TextMessage.

The class org.springfraamework.web.socket.WebSocketSession current supports sending of a arbitary byte[] data via sendMessage(new BinaryMessage(byte [] data) )

I'll have a go at cloning and building locally.

[spring-projects-issues]

Rossen Stoyanchev commented

Yes you're right about Base64. In that case do you see it as an application-wide switch, i.e. when enabled all client outbound messages are sent as binary messages? I just don't see how else a decision can be made whether to send text or binary.

[spring-projects-issues]

Adam Granger commented

The issue seems to be in the interface between spring messaging and web sockets.

• Spring messaging consumes a message of type org.springframework.messaging.Message via classes such as SimpMessagingTemplate.send(message).
• StompSubProtocolHandler expects to see messages with byte[] payload so has no way to tell whether they are true byte [] content or an encoded bytes of a String.

The only solution I can see is some how marking each Message or its contents as "real binary". There are a number of possible ways to do this, but I'm guessing using the Message.getHeaders() is the accepted way? So should be be "just" a case of coming up with a sensible header and switching on that in handleMessageToClient to decide to send BinaryMessage or TextMessage ?

Ideally I'd like to send a mixture of text and binary messages depending on the situation. The application I'm working on uses a combination of binary data at high data rates and more structured data at lower data rates. Using binary for structured data adds significant complexity at sending/receiving end which is why I'd like to leave that as JSON, and only use Binary when I absolutely have to for performance reasons.

[spring-projects-issues]

Rossen Stoyanchev commented

The only solution I can see is some how marking each Message or its contents as "real binary".

There is already a mechanism to tell if the body contains binary data (i.e. the absence of content-type or equal to "application/octet-stream"). A STOMP frame consists of command, headers, and body. The spec says "commands and headers are encoded in UTF-8" (see here) and "if the content-type header is set, its value MUST be a MIME type which describes the format of the body. Otherwise, the receiver SHOULD consider the body to be a binary blob." (see here). Do we really need a separate header to decide? It's fully in your control whether to send a message with text or binary body.

Looking at stomp.js, it appears capable of dealing with a binary WebSocket messages (see here).

The solution currently proposed reflects the above. Is it not working for you?

[spring-projects-issues]

Adam Granger commented

Sorry for the confusion, I've looked at your fork and looks spot-on correct , yes I agree using the same header as the stomp spec makes complete sense.

Unfortunately I've not managed to pull your repo for a test - I work behind a corporate proxy and can't seem to get git clone to work with http. Anyway I'm back at home for the weekend now, I'll have a go again on Monday.

Thanks again.

[spring-projects-issues]

Rossen Stoyanchev commented

I presume you tried "git clone https://github.com/rstoyanchev/spring-framework.git"?

In any case we have time for 4.1.4. Let me know.

[spring-projects-issues]

Adam Granger commented

I can confirm I've tested this with my project. I can see binary messages are received in Chrome when the header "contentType" is set to "application/octet-stream".

Note I had to set "contentType", not "content-type" as your comment above suggests. It seems MessageHeaderAccessor.getContentType() uses MessageHeaders.CONTENT_TYPE ("contentType") not StompHeaderAccessor.STOMP_CONTENT_TYPE_HEADER ("content-type"). Is that correct?

[spring-projects-issues]

Rossen Stoyanchev commented

Okay, thanks for confirming. We'll roll that into 4.1.4 as planned. And yes MessageHeaderAccessor#setContentType is the right one to use.

[spring-projects-issues]

Ganapathy Pichumani commented

Hi Adam,

Im trying to send a audio blob using spring-stomp/sockjs and i have added the contentType to Stomp header. But it doesnt work.

Do i have to change the content-type on sock? I appreciate your help

[spring-projects-issues]

Ganapathy Pichumani commented

Hi Adam, Issue seems to be with stomp-sockjs - workaournd - base64 encode the audio array buffer during exchange...

not very efficient.

[spring-projects-issues]

Ivan Jacobs commented

Hey is it possible to point to an working example. It is not clear how exactly to achieve the sending of binary using stomp-sockjs .

[spring-projects-issues]

Rossen Stoyanchev commented

SockJS does not support sending binary messages. The goal of this ticket was to make sure you can send binary messages with a STOMP endpoint over WebSocket. If you look in the StompSubProtocolHandler you'll see it decides whether to send the message as a binary or a text WebSocket message.

[spring-projects-issues]

Ivan Jacobs commented

Thank you for this answer. What is the alternative to send binary from client to back end ?
As converting to encoded base64 string and sending is quite network consuming.