[Performance] Loading chats takes 10+ minutes for guides and can often fail altogether

[kevinksullivan]

Slack https://expensify.slack.com/archives/C036S0BVAHH/p1687540879337529

If you haven’t already, check out our contributing guidelines for onboarding and email contributors@expensify.com to request to join our Slack channel!

---

What performance issue do we need to solve?

Guides (who have a lot of policies and reports) are often unable to move between chats without NewDot showing a blank chat screen and loading for minutes (or failing to load altogether.

• Example 1
• Example 2

What is the impact of this on end-users?

Guides cannot load chats and talk with their assigned leads.

List any benchmarks that show the severity of the issue

Please also provide exact steps taken to collect metrics above if any so we can independently verify the results.

Proposed solution (if any)

No idea

List any benchmarks after implementing the changes to show impacts of the proposed solution (if any)

Note: These should be the same as the benchmarks collected before any changes.

Platforms:

Which of our officially supported platforms is this issue occurring on?

• Android / native
• Android / Chrome
• iOS / native
• iOS / Safari
• MacOS / Chrome / Safari
• MacOS / Desktop

Version Number: v1.3.30-5
Reproducible in staging?: Y
Reproducible in production?: Y
Email or phone of affected tester (no customers): christopher.bowers@team.expensify.com , natasha.varner@team.expensify.com

Notes/Photos/Videos: Any additional supporting documentation

Videos / screenshots in the slack link examples above

View all open jobs on Upwork

[melvin-bot]

Triggered auto assignment to @arosiclair (Engineering), see https://stackoverflow.com/c/expensify/questions/4319 for more details.

[iwiznia]

I need to investigate why the reauthentication mechanism did not fire and got stuck in an infinite loop retrying over and over.

[iwiznia]

Logs https://www.expensify.com/_devportal/tools/logSearch/#query=request_id:(%227dbafc181f8544f7-ATL%22)+AND+timestamp:[2023-06-23T06:23:17.195Z+TO+2023-06-23T08:23:17.195Z]&index=logs_expensify-022942

Throwing exception with message: '407 Unauthorized - Invalid token' from auth/lib/AuthToken.cpp:1038

interesting that's Unauthorized invalid token and not token expired... 🤔

[iwiznia]

Here's the app logs https://www.expensify.com/_devportal/tools/logSearch/#query=request_id:(%227dbf356c7ab21f9d-ATL%22)+AND+timestamp:[2023-06-23T18:41:28.436Z+TO+2023-06-23T20:41:28.436Z]&index=logs_expensify-022949 I don't get why I don't see any calls to reauthenticate, I tried locally and I do see them

[iwiznia]

Wait actually the requests stuck are like this https://www.expensify.com/_devportal/tools/logSearch/#query=request_id:(%227dbf35385fc91f9d-ATL%22)+AND+timestamp:[2023-06-23T18:41:20.104Z+TO+2023-06-23T20:41:20.104Z]&index=logs_expensify-022949

Throwing exception with message: '407 Unauthorized - Token Revoked' from auth/lib/AuthToken.cpp:814

Revoked... still it's 407

[iwiznia]

Code is here which means the authToken was issued before we updated private_minimumAuthTokenIssueTime

[iwiznia]

Could not find that authToken being set for that account in the logs https://www.expensify.com/_devportal/tools/logSearch/#query=blob%3A%22Invalidating%20authTokens%20by%20setting%20PRIVATE_MINIMUM_AUTH_TOKEN_ISSUE_TIME%20for%20accountID%3A%2011736036%22%20AND%20timestamp%3A%5B2023-06-20T00%3A00%20TO%202023-06-24T23%3A59%5D&index=_all

[iwiznia]

I can't see them having log out as they said https://www.expensify.com/_devportal/tools/logSearch/#query=blob%3A%22LogOut%22%20AND%20email%3A%22natasha.varner%40team.expensify.com%22%20AND%20timestamp%3A%5B2023-06-20T00%3A00%20TO%202023-06-24T23%3A59%5D&index=_all

[iwiznia]

Going to close this to have a clean slate for the issue so we don't distract new people with my previous investigations...

[iwiznia]

New issue here #21766

[aimane-chnaif]

Can I get compensation here for reviewing linked PR?

[aimane-chnaif]

Can I get compensation here for reviewing linked PR?

@iwiznia kindly bump

[kevinksullivan]

Hi @aimane-chnaif , would this be $250? Also do you receive payment in newdot or in upwork?

[aimane-chnaif]

@kevinksullivan I am not sure. We agreed all internal review compensations are flatten 1k afaik. Easy PR will cover for big PR.
Discussion: https://expensify.slack.com/archives/C02NK2DQWUX/p1669905679029849.
Btw I am fine either.

[kevinksullivan]

@aimane-chnaif thanks, where have you been receiving payment for recent work? NewDot or Upwork?

[aimane-chnaif]

@aimane-chnaif thanks, where have you been receiving payment for recent work? NewDot or Upwork?

upwork

[kevinksullivan]

offer sent @aimane-chnaif

https://www.upwork.com/jobs/~010ec101896471dba7

lmk when you accept!

[aimane-chnaif]

Accepted, thanks!

[kevinksullivan]

All set