Support for new Steam chat

[JustArchi]

Today Steam got new chat update that is trying to mimic Discord in majority of aspects. I'm wondering if we could hope for at least basic at first, SK2 support for it.

I took a quick look and the good news are that we can use good old Steam protocol for that, which makes it fall into SK2 scope, but the bad news are that it might require some extra work:

Currently NHA2 is missing relevant bits, it looks like #477 proposed a fix, we might need something like that to fully support it.

Do you have a plan to add proper support for it in near future? I even wanted to start working on it right away but since SK2 doesn't even have a specification for EMsg of 151, I'm not sure what should be done, as my knowledge of SK2 internals is rather poor. It doesn't look like it's just a body definition. If we could get some basic support for sending and receiving packets, then later on we could write proper (new) handlers for that, such as SteamFriends.

It might be even already possible to parse and send those packets from within SK2, but I'm not sure how to achieve that. Please let me know if you do.

BTW, this chat is also fully supported in web browser - I also took a quick look and web browser achieves that with websocket connection to CM servers. Nothing interesting for us specifically probably, but worth mentioning.

[yaakov-h]

Check out the steamclient-beta branch and #565.

[JustArchi]

Trying to start with ChatRoom.SendChatMessage#1 but I'm probably too stupid for this 😅

I guess I'll just wait for finished implementation instead, unless you have extra surplus of time and willings to check below and find out what is wrong 😓

internal sealed class ArchiBetaHandler : ClientMsgHandler {
	public override void HandleMsg(IPacketMsg packetMsg) { }

	internal async Task<bool> SendChatMessage(ulong chatGroupID, ulong chatID, string message) {
		CChatRoom_SendChatMessage_Request request = new CChatRoom_SendChatMessage_Request {
			chat_group_id = chatGroupID,
			chat_id = chatID,
			message = message
		};

		try {
			await SendMessage("ChatRoom.SendChatMessage#1", request);
		} catch {
			return false;
		}

		return true;
	}

	private AsyncJob<SteamUnifiedMessages.ServiceMethodResponse> SendMessage<TRequest>(string name, TRequest message, bool isNotification = false) where TRequest : IExtensible {
		if (message == null) {
			throw new ArgumentNullException(nameof(message));
		}

		ClientMsgProtobuf<CMsgClientServiceMethod> msg = new ClientMsgProtobuf<CMsgClientServiceMethod>(EMsg.ServiceMethodCallFromClient) { SourceJobID = Client.GetNextJobID() };

		using (MemoryStream ms = new MemoryStream()) {
			Serializer.Serialize(ms, message);
			msg.Body.serialized_method = ms.ToArray();
		}

		msg.Body.method_name = name;
		msg.Body.is_notification = isNotification;

		Client.Send(msg);

		return new AsyncJob<SteamUnifiedMessages.ServiceMethodResponse>(Client, msg.SourceJobID);
	}
}

[Serializable]
[ProtoContract(Name = nameof(CChatRoom_SendChatMessage_Request))]
public class CChatRoom_SendChatMessage_Request : IExtensible {
	[ProtoMember(1, IsRequired = false, Name = "chat_group_id", DataFormat = DataFormat.FixedSize)]
	public ulong chat_group_id { get; set; }

	[ProtoMember(2, IsRequired = false, Name = "chat_id", DataFormat = DataFormat.FixedSize)]
	public ulong chat_id { get; set; }

	[ProtoMember(3, IsRequired = false, Name = "message", DataFormat = DataFormat.Default)]
	public string message { get; set; }

	private IExtension extensionObject;
	IExtension IExtensible.GetExtensionObject(bool createIfMissing) => Extensible.GetExtensionObject(ref extensionObject, createIfMissing);
}

[voided]

You'd probably want to take a look at the SteamUnifiedMessages handler. That plus your re-created protos may get you the rest of the way.

I would posit that you may not need to use the ServiceMethodCallFromClient EMsg, but I might be wrong. I don't see any reason this group chat stuff is any different than existing unified service messages.

https://github.com/SteamRE/SteamKit/blob/master/Samples/8.UnifiedMessages/Program.cs for usage.

[yaakov-h]

IIRC, the body of the message is the recreated proto request.

You would need to send a ClientMsgProtobuf<CChatRoom_SendChatMessage_Request> with the ServiceMethodCallFromClient EMsg value and set the job name in the header to ChatRoom.SendChatMessage#1.

[JustArchi]

Thank you both! I managed to get first working example, sharing it for reference:

internal sealed class ArchiBetaHandler : ClientMsgHandler {
	public override void HandleMsg(IPacketMsg packetMsg) { }

	internal async Task<SteamUnifiedMessages.ServiceMethodResponse> SendChatMessage(ulong chatGroupID, ulong chatID, string message) {
		ClientMsgProtobuf<CChatRoom_SendChatMessage_Request> request = new ClientMsgProtobuf<CChatRoom_SendChatMessage_Request>(EMsg.ServiceMethodCallFromClient) {
			Body = {
				chat_group_id = chatGroupID,
				chat_id = chatID,
				message = message
			},

			SourceJobID = Client.GetNextJobID()
		};

		request.Header.Proto.target_job_name = "ChatRoom.SendChatMessage#1";

		Client.Send(request);

		try {
			return await new AsyncJob<SteamUnifiedMessages.ServiceMethodResponse>(Client, request.SourceJobID);
		} catch {
			return null;
		}
	}
}

public class CChatRoom_SendChatMessage_Request : IExtensible {
	[ProtoMember(1, IsRequired = false, Name = "chat_group_id", DataFormat = DataFormat.Default)]
	public ulong chat_group_id { get; set; }

	[ProtoMember(2, IsRequired = false, Name = "chat_id", DataFormat = DataFormat.Default)]
	public ulong chat_id { get; set; }

	[ProtoMember(3, IsRequired = false, Name = "message", DataFormat = DataFormat.Default)]
	public string message { get; set; }

	private IExtension extensionObject;
	IExtension IExtensible.GetExtensionObject(bool createIfMissing) => Extensible.GetExtensionObject(ref extensionObject, createIfMissing);
}

Now I'll try to add all missing stuff, since right now even response doesn't return, but hey, it works 🎉

[JustArchi]

@voided you were right too, this also works:

var uniMessages = SteamClient.GetHandler<SteamUnifiedMessages>();

var request = new CChatRoom_SendChatMessage_Request {
	chat_group_id = 0, // must be valid
	chat_id = 0, // must be valid
	message = "Test2"
};

uniMessages.SendMessage("ChatRoom.SendChatMessage#1", request)

Then it's just a matter of reverse-engineering protobufs and hooking it to unified messages, perfect 🎉.

[JustArchi]

Note to self and other people to not waste productivity over stupid things:

ClientMsgProtobuf<CMsgClientUIMode> request = new ClientMsgProtobuf<CMsgClientUIMode>(EMsg.ClientCurrentUIMode) { Body = { chat_mode = 2 } };
Client.Send(request);

Send this to enable beta chat mode. Otherwise you won't receive majority of callbacks and won't be able to move forward.

... Don't ask how many hours I wasted debugging only to find out about this 😅

[JustArchi]

I've successfully written very basic protobufs for sending and receiving new Steam group messages, this is a good start - steamclient-beta...JustArchi:archi-wip

I'm not sending PR with this as we'll probably want to automate generation of those (I guess?), but feel free to make use of them for time being. I'll probably go with private messaging next.

I'm wondering whether we want to make a bit more user-friendly methods to access those, and how exactly they should look like. I mean, I can totally see SteamChatRoom.IncomingChatMessageCallback and SteamChatRoom.SendChatMessage() already possible, although I'm not sure yet how those handlers should work, as under the hood everything is nicely handled by SteamUnifiedMessages, it'd basically be a wrapper over IChatRoom that would do something similar to what I'm already doing at lower level:

private async void OnServiceMethod(SteamUnifiedMessages.ServiceMethodNotification callback) {
	switch (callback.MethodName) {
		case "ChatRoomClient.NotifyIncomingChatMessage#1":
			CChatRoom_IncomingChatMessage_Notification body = (CChatRoom_IncomingChatMessage_Notification) callback.Body;

			if (body.message == "ping") {
				CChatRoom_SendChatMessage_Request request = new CChatRoom_SendChatMessage_Request {
					chat_group_id = body.chat_group_id,
					chat_id = body.chat_id,
					message = "pong"
				};

				await ChatRoomService.SendMessage(x => x.SendChatMessage(request));
			}

			break;
	}
}

On the other hand it might make sense to make SteamUnifiedMessages.ServiceMethodNotification generic firstly, so we could subscribe more easily to given notifications - currently you can see this awful body cast. Entire switch logic could be easily moved somewhere deeper inside SK2, as SteamUnifiedMessages is already smart enough to map ChatRoomClient.NotifyIncomingChatMessage#1 into NotifyIncomingChatMessage from IChatRoomClient interface. At this point I'm not even sure if what I want to see is possible to do in C#, but it sounds like it could be, as basically we just want to catch all ServiceMethod EMsg with body of CChatRoom_IncomingChatMessage_Notification.

In any case, it starts looking really good, thanks again for initial help, I'll shut up now and let you work in peace 😀. Those are just random ideas, it's your decision whether they make sense or not 🎉.

[yaakov-h]

I'll shut up now and let you work in peace 😀

Oh please don't, you've been very useful. 😉

Currently, between @DoctorMcKay and @xPaw, we've got a rough .proto file out of the Javascript frontend: https://github.com/SteamDatabase/Protobufs/blob/ff8a4dbb6a1ad54e8248bf87617f4686244a6d85/steam/WebUI/friends.proto

I think the next step would be to clean that up so that we don't have to manually maintain all the new unified services:

• Reverse-engineer the proto service definition.
• Dump each service into it's own .proto file.
• Reverse-engineer the enums.
• Generate C# code from the new .proto files.
• Wrap it with either the SteamFriends handler, or a new one entirely.

[JustArchi]

@yaakov-h Thanks a lot! I used your friends.proto, manually extracted from it interesting for me Chat bits, patched them for missing info, generated *.cs files and manually added interesting me interfaces. In case you'd be interested, here is my dirty branch that I'm currently using for testing beta stuff - nothing appropriate for PR, but useful for alpha tests before we get appropriate bits in SK2 itself.

I can say that this works really good, it's still very dirty but I managed to add support for everything in both of my ArchiBoT and ASF projects. Nothing really huge, but for now everything works great and I'll keep adding (and testing) other stuff. I'm positively shocked how consistent and reliable all of that is. I mean look, I can finally hook every request to its own response, strong-type all of that and finally have async output without having to deal with crap like #491 and wondering why suddenly working things broke.

Thanks once again for everything, I can continue breaking things on my own now 😀.

[yaakov-h]

Compileable proto in 6593190.

[JustArchi]

@yaakov-h I've noticed that some of new interfaces have NotImplemented types, in particular:

rpc AckChatMessage (.NotImplemented) returns (.NoResponse);

I've verified that this one takes CChatRoom_AckChatMessage_Notification and in fact returns no response. Could you take a look why they were generated like that? Thanks a lot, this is the only interface that I've tried until now that has this mismatch (there are others but I didn't RE them manually).

[yaakov-h]

@xPaw any ideas? ^^^

[xPaw]

Well, that's how @Ne3tCode decided to do it, if there's a response proto. Looks like he managed to fix AckChatMessage and NotifyUserVoiceStatus but not others.

For example, CChatRoom_GetRoles_Response doesn't appear to have a request or notification proto?

[Ne3tCode]

I checked js code [not much] carefully and can confirm that ChatRoom.AckChatMessage and ChatRoomClient.NotifyAckChatMessageEcho reuses the same proto CChatRoom_AckChatMessage_Notification, also VoiceChat.NotifyUserVoiceStatus and VoiceChatClient.NotifyUserVoiceStatus reuses CVoiceChat_UserVoiceStatus_Notification proto. So I fixed it.
All other NotImplemented protos are not defined and unused in friends.js code.

P.S. CChatRoomMember.state field (enum) is also unused. If someone want to help RE enums I just leave this link here.

// Nephrite

[JustArchi]

Since the issue died a bit, let me refresh it with what I managed to do in my projects to hopefully help @yaakov-h and the rest of the team to eventually tackle down this one.

I didn't have much needs so I basically reverse-engineered only the parts responsible for receiving the messages and sending them (including joining chat rooms). This is enough for basic chat implementation, but there is a room for improvement in regards to stuff like e.g. parsing the messages and alike.

AckChatMessage and AckMessage mark messages as read, appropriately for the group chat and private chat.

There are new APIs for friends management, I've added AddFriend and RemoveFriend which could work as replacement of current SteamFriends functions.

You can get ID of the chat room from the clan's ID using GetClanChatRoomInfo, this could be useful for implementing a basic JoinChatRoomGroup, there is also GetMyChatRoomGroups for accessing those already joined.

Finally there is SendMessage and SendChatMessage for private chat and group chat. They should work as replacements for existing methods, for example first one allows sending typing statuses and alike.

For handling incoming chat messages, I hooked into SteamUnifiedMessages.ServiceMethodNotification where I listen for ChatRoomClient.NotifyIncomingChatMessage#1 and FriendMessagesClient.IncomingMessage#1. Both received protos include message_no_bbcode property, but for some reason it's not always available (also with normal chat messages), so I have this solution for escaping normal message in case no_bbcode version isn't available. From my tests it looks like unescaping [ and \ is enough. When mixing bbcode with non-bbcode (so contains_bbcode = true), you need to apply escaping which is the reverse of the above, to the part you wish to not treat as special. That bbcode is still needed for putting stuff in /quote or other /pre, and bbcode will still be needed for stuff like mentions. In fact, it'd make sense to code a helper for including those special things in the message.

All of the info above gives a sneak peek into how new chat works, but there are still SK2 project decisions that need to be reviewed before deciding to implement all of that.

1. New chat uses unified messages exclusively, both for sending and receiving. We should consider making it easier for people to consume and send those, you can see my current receiving code with that awful switch and data casting, it probably could be written much better in easier form of subscribing to particular message and defining its body (if we can't determine that from the endpoint alone).
2. Once above is done, it'd make sense to add wrappers to make new chat easier to consume. A lot of functions could be very simple wrappers over SteamUnifiedMessages.UnifiedService<IChatRoom>, SteamUnifiedMessages.UnifiedService<IFriendMessages> and potentially other services, not that much different from what I've implemented in my ArchiHandler.
3. I'm not so sure if wrappers above actually make sense in regards to unified messages. I mean, we could make them, creating something like SteamFriendsV2 with all the callbacks and functions, but I'm not sure if the focus of this issue shouldn't be put on making unified messages easier to send and consume (so basically point 1 I've stated above), and then full focus on documenting examples of how to work with them. In my opinion unified messages are much easier to consume than raw protobufs in requests we're using all the time, and while SteamFriends made a lot of sense in the past, I totally see how new SteamChat could just have a bunch of events to subscribe to and exposed unified services, a bit improved in regards to point 1.

Those are as usual just my thoughts, I'm not sure if they're even helpful at all and that I'm not actually confusing everybody around, but I think that instead of hunting new APIs, messages and protos, it'd be more wise to have a good foundation of the basic concepts and documentation of how those new things work together so the users could just implement what they need instead of learning that SteamFriendsV2.SendChatMessage() sends a group chat message, which is really just a fancy name for UnifiedChatRoomService.SendMessage(x => x.SendChatMessage()).

As usual feel free to browse my ASF project for working implementation of everything mentioned above. I tried my best to make it as good as it made sense in regards to my use cases, which is exactly why I realized that there is not really that much we need extra from SK2 to make it super friendly and easy to consume, merely cutting down on the excessive parsing noise and making some interfaces easier to use.

[chakany]

I was wondering if this could allow the ability to join voice channels, and do things such as play music. (Like a Discord Music bot)

[xPaw]

@JustArchi It appears that you're basically the only user of the new chat system, would you be able to bring some of the new handlers and methods from your implementation into SK?

[JustArchi]

If I'm the only user then SK2 doesn't need that code just for me 😁.

I'm short on time, but I'll see if I can come up with some PR in regards to this, if the rest of the team would prefer that instead of coding themselves.

[yaakov-h]

I, for one, would definitely appreciate:

• having a basic chat implementation in SteamKit
• not having to write it myself
  😉

[cooljeanius]

matterbridge no longer has Steam chat support due to this; see 42wim/matterbridge@9592cff and Philipp15b/go-steam#94. (I came across this because SuperTux is going to be coming to Steam, and people on the SuperTux Discord might want to bridge their Discord server with Steam chat via matterbridge)

[yaakov-h]

The go-steam team are in the same position as us, nobody has the spare time to figure out the pieces, how they fit together, and a neat API to wrap it all.

Archi has done quite a bit of work above, assuming that the implementation hasn't changed and those comments are still true then it shouldn't be too hard to build wrappers in either library.

[JustArchi]

I have a basic implementation in ASF that allows to read chat messages and write them, but I didn't have time and motivation to extract all those parts and put in SK2 as of today.

Steam chat is a giant beast and requires more work than I put into it however, as proper implementation would also need to handle Steam-specific bbcode, send stickers/emotes/images, parse them and do whole lot more than what I do with my basic read/write pure plaintext.

You're more than welcome to use my work if you plan on adding those bits to SK2 or any other Steam-related lib, but I'm just saying it's nowhere close to being finished and this is one of the reasons why I wasn't that eager to just put it in SK2 - because I can't commit myself to it as of now.

[xPaw]

FWIW the old chat implementation in steamkit still works.

[JustArchi]

FWIW the old chat implementation in steamkit still works.

Only for private chat, group chat requires completely new implementation that ASF has.

[Efreak]

@cooljeanius you might be interested in icewind1991/mx-puppet-steam.

[cooljeanius]

@cooljeanius you might be interested in icewind1991/mx-puppet-steam.

making it a link for cases where it might not have auto-linkified: https://github.com/icewind1991/mx-puppet-steam

[heinrich5991]

I think this is being used by mx-puppet-steam: https://github.com/DoctorMcKay/node-steam-user/wiki/SteamChatRoomClient.

[cooljeanius]

I think this is being used by mx-puppet-steam: Wiki: SteamChatRoomClient (DoctorMcKay/node-steam-user).

Hm, maybe @DoctorMcKay can help?