publicRooms API can be incredibly slow over federation, causing 504s

[ara4n]

split out from element-hq/element-web#3327

[ara4n]

@erikjohnston fixed the perf for the non-federated case.

[erikjohnston]

I made it faster for all cases, so I would expect it to be as slow over federation as if a client asked for a non-limited public rooms list (which I think the mobile apps do?)

[ara4n]

empirically I think this is still very slow? e.g: disroot->matrix.org from Nils in https://matrix.to/#/!QtykxKocfZaZOUrTwp:matrix.org/$15323461925250swAwa:disroot.org

Yeah -- I think it's just so slow that I get no result
I'm not getting any room list from Matrix.org after changing to it in the drop down on the room list page

[hawkowl]

This is basically just another manifestation of #4043 and will be fixed by the stats branch. Closing this as a duplicate