Error: Invalid session, reconnecting loop

[Corosauce]

As of 2 days ago, discord just loops endlessly with this below, as best I can remember I changed nothing, discord might have been having a bad connection day when it started too I can't quite remember. Since the issue I've tried disabling backlog to reduce load, built the plugin from source again and restarted the bitlbee service and done a full reconnect, not sure what else to try, anyone else having this issue, am I missing something?

For the record when I had backlog on, it would retreive it or spam "Error: Failed to get backlog (0)." and then still get invalid session.

Using bitlbee v3.4.2-0ubuntu1 with a discord account with 2FA on.

Log:

Jun 01 18:15:30 account discord on
Jun 01 18:17:47 discord - Logging in: Logged in
Jun 01 18:17:52 discord - Remote host is closing websocket connection
Jun 01 18:17:52 discord - Performing soft-reconnect
Jun 01 18:18:03 discord - Error: Invalid session, reconnecting
Jun 01 18:18:03 discord - Signing off..
Jun 01 18:18:06 discord - Reconnecting in 5 seconds..
Jun 01 18:21:23 discord - Logging in: Logged in
Jun 01 18:21:23 discord - Remote host is closing websocket connection
Jun 01 18:21:23 discord - Performing soft-reconnect
Jun 01 18:21:41 discord - Error: Invalid session, reconnecting
Jun 01 18:21:41 discord - Signing off..
Jun 01 18:21:45 discord - Reconnecting in 5 seconds..

[dgw]

Try setting the token manually, as described in #118 (comment) (instructions for getting the token in your browser are further down the thread).

[Corosauce]

Thanks for the tip, sadly doesn't seem to help. Logged into discord on firefox and got the token via the storage inspector way, then:

(snipped out the auth token from pasted log for security reasons, but I entered what was set as token from firefox)

account discord set token_cache SNIP
token_cache = `SNIP'
save
Configuration saved
account discord on
discord - Logging in: Logged in
discord - Remote host is closing websocket connection
discord - Performing soft-reconnect
discord - Error: Invalid session, reconnecting
discord - Signing off..
discord - Reconnecting in 5 seconds..
discord - Logging in: Logged in
discord - Remote host is closing websocket connection
discord - Performing soft-reconnect
discord - Error: Invalid session, reconnecting
discord - Signing off..
discord - Reconnecting in 5 seconds..

[sm00th]

Error: Invalid session, reconnecting is not an issue with logging in, it just means that soft reconnect failed and you have to do a full login again. It does sound kinda scary but I didn't want to rename it since it is what discord calls it.

The issue doesn't seem to be with logging in. Can you see if there is anything interesting in the debug logs before discord closes websocket? See https://github.com/sm00th/bitlbee-discord/blob/master/README#L152

You should strip the login info, token and the huge 'READY' with server info is probably not needed.

[Corosauce]

Alrighty, I've done that and outputted it to a log, stripped out the token and giant READY lines, the remains of the info looks tame enough to be ok.

I had it do the connection cycle a couple times just to make sure it got everything, hard to stop it from connecting when its working so hard processing all the data.

Here is the 55mb file, give it a download and I'll delete it for safety once you've replied: http://share.coros.us/files/bitlbeediscord-debuglog.txt

Things worth noting, I'm one of those users with a lot of discord channels, the combined user count is easily many 1000s im sure. It takes a good 3-5 mins or more for it to stop sitting at 100% for bitlbee.

I can't quite spot where it is dropping the connection, my programmer eyes might not be trained well enough for network data ;)

[sm00th]

Here is the 55mb file, give it a download and I'll delete it for safety once you've replied

Got it, you can delete it. I'll have to look at it tomorrow though.

[sm00th]

This is caused by that huge server list. bitlbee-discord wouldn't send heartbeats until it got all the info on servers and as you mentioned it takes 3-5 minutes to get that while discord expects a heartbeat every ~40 seconds. Can't really check but 09bd77e should help make bitlbee-discord start sending heartbeats right after it establishes websocket connection. If it doesn't work please attach another log like the last, you can also remove the guild_sync blocks while leaving marks where they were, something like s/^.*GUILD_SYNC.*$/GSYNC_REDACTED/ should be fine.

[Corosauce]

Thanks for the help so far, same issue with new commits, removing GUILD_SYNCS sure sized the log down heh, log: https://gist.github.com/Corosauce/77fa2e13536e65bab934e1cf41e0574d

[sm00th]

This is rather strange, the behavior didn't change at all. Maybe I misjudged how evenloop prioritizes tasks and additional changes are needed, but are you sure you got the new commit running, i.e. restarted bitlbee with your irc client disconnected and all that? You check by seeing if the new always_afk option is available.

[Corosauce]

Yeah I myself was paranoid I didn't have the latest build so I deleted the la and so files in the place where the latest built plugins go and it showed it was missing in bitlbee, so it was definitely using latest, and yeah I have the always_afk option so they must be built with the latest source I grabbed. I had also done the full proper service restart while disconnected in the irc client etc.

In the meantime I used the logs to figure out that it was the official minecraft discord and only that one that was causing the insane load times, for now I left their discord to work around the problem since I didn't have any special privileges to worry about.

I would be happy to rejoin their discord to help you test the plugin for making it work for these huge servers though.

The bitlbee 100% cpu usage time went from 5 minutes to 3 friggin seconds during connect once I left it, they must have 10's of thousands of users, insane.

[sm00th]

The bitlbee 100% cpu usage time went from 5 minutes to 3 friggin seconds during connect once I left it, they must have 10's of thousands of users, insane.

Ouch, I was thinking about modifying bitlbee-discord so that it would be able to send heartbeats in-between guild-syncs, but if it is just one guild that is causing this delay it won't help much. Can't think of many ways to work around this. Maybe we'll be able to split up the user list and add it in multiple portions, but that will be very hackish.

[Alcaro]

Wouldn't it be better to not take five minutes to process the guild sync?

[CryptoSiD]

I'm unable to sign in since ~3 days or so

[19:36:30] (@bitlbee): discord - Heartbeat timed out, reconnecting...
[19:36:30] (@bitlbee): discord - Performing soft-reconnect
[19:36:40] (@bitlbee): discord - Error: Invalid session, reconnecting
[19:36:40] (@bitlbee): discord - Signing off..

Is there any temporary solution for this?

[sm00th]

Bitlbee's 'develop' branch contains a commit that is supposed to help with this: bitlbee/bitlbee#113

[Corosauce]

So finally after over a year I finally got around to rebuilding my setup with latest optimizations on the bitlbee discord, according to my logs it went from taking 2 minutes give or take 10 seconds to process the initial connection and time out my discord, to 8 seconds with new update, I'm going to consider this issue solved.

Also holy crap the minecraft discord is at 500K members and hit the discord limit until they fix it which is pending, I was going to test the new setup with it but I can't join it right now, owell.