Slack doesn't finish connecting, endlessly fetching channel names

[mpmilano]

It looks like there might be something wrong with the parsing or fetching of channel names in this plugin --- according to the log file, it looks like the "login" step fails to complete for one of my 7 slacks due to the process of fetching channel names being re-started over and over again. The first two channel name fetch requests complete as expected and introduce newly-discovered channels; the next several hundred requests just reveal the same two sets of channel names, and no new names are discovered.

This started happening some time yesterday or this morning, during which the slack in question created only one new public channel, called "erdos-scheduler"

I should be noted that receiving messages works as expected during the endless channel-name fetching, but as the "login" step has not completed pidgin does not allow the sending of messages.

Any idea what this could be?

[hmcclungiii]

I think I'm having the same problem, though I believe it has something to do with pagination and rate limiting. The first request works fine, however the next request doesn't update the pagination token and it just retrieves the same response over and over again until finally rate limited.

[piyushgarg]

+1

[dylex]

Which phase of login does this happen in ("Loading Users", "Loading conversations", ...) or could you send the repeated request and response from the logs? It's hard to see how the conversations.list call could loop unless slack keeps sending back the same cursor, but seeing the debug logs would probably point to something.

[piyushgarg]

slack: api call: https://.slack.com/api/users.list?token=xoxs-22xx-66xxxxxx&presence=false&limit=100&cursor=dXNlcxxxxxxx%3D

There could be hundred to thousands of users in different channels (15-20) I belong to, does it mean the above call will never be completed?

Hi @dylex , how can I suppress personal information from the debug logs?

[dylex]

Ah, yeah, if there are a lot of users on the slack, that can take a long time, since we can only fetch so many at a time and slack rate limits. See #81. There's an experimental workaround on the history branch by enabling the lazy_load option. This will prevent it from loading all the users at login, and only load them as they're seen. It'd be great to get feedback on how well this works.

[piyushgarg]

Hi @dylex , Super!! It worked within 5 seconds, I guess it should be part of master branch.

[hmcclungiii]

Hi @dylex ,

It looks like the lazy_load option is allowing me to get my slack account loaded, though this is just a band aid as the original issue still exists if I don't use the lazy_load option.

I think where I said "pagination token" and you said "cursor" we may be referring to the same thing. As with @piyushgarg , I'm looking at an account with thousands of users and at least 100 channels.

I'll try to get my logs together tomorrow to show you what I've discovered. This is a company account so I have to go through them and make sure I don't accidentally post any company sensitive data.

[EionRobb]

The SLACK_PAGINATE_LIMIT is set to 100 results per page however a lot of the endpoints can go up to 1000. Is there any reason to not request as many results at a time as possible to reduce the round-trip time?

[dylex]

I'm pretty sure I set that to the limit whenever I wrote it, but if they've increased it we should certainly revisit. users.list recommends 200, but trying something larger sounds reasonable. I'll try 500.

@hmcclungiii you say it's providing the exact same cursor every time? That definitely sounds like a bug somewhere and it'd be great to see at least one request and response pair, even with only the metadata.

(The only reason I haven't merged history in yet is I'm stil seeing an occasional crash, I think when getting activity or a message from a new/unknown user, but haven't been able to track it down beyond some sort of double-free memory corruption.)

[mpmilano]

Well, bumping the limit did seem to fix the problem for me, at least. Might easily still be masking the bug tho.

[petermolnar]

I started having this as well. I tried everything, but this kept crashing pidgin, spectrum2, and even bitlbee via purple. lazy_load solved it.

[hmcclungiii]

@hmcclungiii you say it's providing the exact same cursor every time? That definitely sounds like a bug somewhere and it'd be great to see at least one request and response pair, even with only the metadata.

@dylex Hi! Sorry, I lost track of this thread and had to quit using the software for some time. I apologize for the nearly years long delay in responding. It looks like this issue still persists. I'll see what I can do about reproducing it again and getting you some valuable debug info very soon.

[ghost]

This appears to be a bug on their end. While working on #137, I discovered that conversations.list returns different data based on whether you pass parameters as JSON or as form data, even though the parameters themselves are identical. Probably they have two different implementations of this endpoint, and the JSON variant has a bug.

While working on #137, I've become blocked by this bug, so I'm planning to switch everything to form data, since this is what the official client uses. Hopefully this will fix both issues in one go.