Test Fleet for Autosharding

[SionoiS]

O great Infra masters I seek guidance on my quest to build a new fleet.

@jm-clius and me talked about autosharding and we are going to need a fleet to test this new feature as it will become the new default.

[SionoiS]

Note to self: For testing I'm thinking we want LIGHTPUSH, FILTER and STORE, define some content topics.

[SionoiS]

Note: For DNS disc we will do the same than for the static shard fleet.

launch the fleet with DNS disc disabled, construct and publish the tree and then update config.

[alrevuelta]

Would suggest being more specific, eg:

• we need wakunode2 running with flags --flag1=value1 flag2=value2
• x amount of disk y amount of ram z cpu specs
• open ports
• amount of nodes (just 1?)
• • volumes that need to be mounted.

[SionoiS]

I updated the PR.

At least 2 nodes for now is ok.

[yakimant]

@SionoiS, let's continue here.
Fleet is up and running.

Can you please help to figure out on conn-limit:
https://github.com/status-im/infra-nim-waku/tree/master/ansible/roles/conn-limit/defaults
Do you know anything about it? I didn't enable it for now.

What is next?

• Shared PostgreSQL like shards.test
• DNS discovery
• Canary checks
• CI: https://ci.infra.status.im/job/nim-waku/job/deploy-waku-test/
• https://fleets.status.im
• conn-limit add or not?

[yakimant]

store protocol is not enabled, so postgres connection is not setup

[yakimant]

We already have an enrtree for test.waku.nodes.status.im domain (wakuv2.test fleet).
Currently it's being replaced with test.wakuv2.nodes.status.im
See https://github.com/status-im/infra-misc/issues/171

But it is still in use with old client apps, docker images and some nodes in theory.
Although it's a test fleet, so I doubt it's critical.

There are 2 solutions with this overlap:

1. Remove enrtree record on test.waku.nodes.status.im for wakuv2.test
2. Have a different (temporary) name for waku.test fleet until we remove the wakuv2.test one. This can happen in 2-4 months.

@jakubgs, @jm-clius, what do you think?
I vote for 1.

[jakubgs]

I would also prefer option number one, but I'll let Hanno add his two cents. If the wakuv2.test fleet is going to be decommissioned soon I don't think it's worth the hassle to try to preserve the old broken DNS entry.

But I guess it would be sensible to ask around a bit more before killing wakuv2.test, just to be sure.

[SionoiS]

@jm-clius is out until the 25th

It's still unclear to me if wakuv2.test was ever used. I vote for 1.

[yakimant]

Let's wait for @jm-clius, unless DNS discovery is blockin waku.test use.

[SionoiS]

I've asked and it is used by some external projects. So let's wait and see.

[jakubgs]

I've asked and it is used by some external projects.

Then they shouldn't be using it. The fleet is for our internal use, research and development. Not for external use. Anyone using it externally for their own purposes has themselves to blame when it disappears.

[jm-clius]

@SionoiS do you know which external projects are using the test fleet? I agree that they shouldn't expect stability if they do.
@yakimant is the new test.wakuv2.nodes.status.im already active? If so, I'd go for option 1 of just replacing the entries at test.waku. We can just announce that those who want to keep making use of the old fleet needs to use the new test.wakuv2 domain.

[SionoiS]

@SionoiS do you know which external projects are using the test fleet?

Railgun but I'm waiting on a response to be sure. @jm-clius

[jm-clius]

Railgun but I'm waiting on a response to be sure. @jm-clius

Thanks! I think we have a good enough relationship with them to ask them to start using the new DNS discovery URL when it's available.

[SionoiS]

Railgun but I'm waiting on a response to be sure. @jm-clius

They are not using our infra anymore. Confirmed.

@yakimant

[yakimant]

Enabled dns discovery:
fc0f80d