Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

E2E Testing fixes and cleanup #1765

Merged
merged 10 commits into from
Nov 8, 2023
Merged

E2E Testing fixes and cleanup #1765

merged 10 commits into from
Nov 8, 2023

Conversation

wilwade
Copy link
Collaborator

@wilwade wilwade commented Nov 7, 2023

Goal

The goal of this PR is the final misc pieces of getting the e2e tests less flaky. Does this fix all of it? No. Most? Yes.

Closes #1731

Discussion

  • Correct batch error asserting
  • Ids used as "bad" should use close to the max
  • Use Immortal Eras due to issues with speed and AncientBirthBlock
  • Cleanup before funding
  • Move assertAddNewKey
  • Add new assertHasMessage

@@ -60,16 +62,6 @@ describe("Capacity Transactions", function () {
assert.notEqual(schemaId, undefined, "setup should populate schemaId");
});

async function assertAddNewKey(capacityKeys: KeyringPair, addKeyPayload: AddKeyData, newControlKeypair: KeyringPair) {
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Moved to helpers

@@ -240,8 +232,7 @@ describe("Capacity Transactions", function () {
page_size: 999
}
);
const response: MessageResponse = get.content[get.content.length - 1];
assert.equal(response.payload, "0xdeadbeef", "payload should be 0xdeadbeef");
assertHasMessage(get, x => x.payload.isSome && x.payload.toString() === "0xdeadbeef");
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

New Helper

@@ -254,8 +245,6 @@ describe("Capacity Transactions", function () {
before(async function () {
capacityKeys = createKeys("CapacityKeys");
capacityProvider = await createMsaAndProvider(fundingSource, capacityKeys, "CapacityProvider", FUNDS_AMOUNT);
})
beforeEach(async function () {
await assert.doesNotReject(stakeToProvider(fundingSource, capacityKeys, capacityProvider, amountStaked));
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No need to stake before each

Comment on lines +46 to -51
assert.fail("batchAll should have caused an error");
} catch (err) {
error = err;
assert.notEqual(error, undefined, " batchAll should return an error");
assert.notEqual(err, undefined, " batchAll should return an error");
}
assert.notEqual(error, undefined, " batchAll should return an error");
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is the best way to do an error assertion.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just curious, any objection to using assert.rejects?
https://nodejs.org/api/assert.html#assertrejectsasyncfn-error-message

Also, I wonder if at some point (not now) we should switch these to Jest (as all of our other Node projects are using it for testing)?

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Or, if we want to keep using Mocha here, I'd be in favor of eventually swapping out the standard Node assert package for something a bit more capable. But definitely not a priority item.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah right. Assert rejects likely would be better. I was just updating it to still match the existing pattern, but that would be better. Next time.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Re: Jest

We might be able to, but I wasn't willing to make the switch yet.

@@ -155,7 +157,7 @@ describe("MSA Key management", function () {
assert.notEqual(event, undefined, 'should have added public key');

// Cleanup
await assert.doesNotReject(ExtrinsicHelper.deletePublicKey(keys, thirdKey.publicKey).signAndSend());
await assert.doesNotReject(ExtrinsicHelper.deletePublicKey(keys, thirdKey.publicKey).signAndSend('current'));
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Important fix. deletePublicKey is a free transaction, so doesn't use up a nonce.

Comment on lines -28 to -29
await ExtrinsicHelper.api.disconnect();
await ExtrinsicHelper.apiPromise.disconnect();
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No need to wait for this

@wilwade wilwade marked this pull request as ready for review November 7, 2023 17:07
@wilwade wilwade enabled auto-merge (squash) November 7, 2023 20:32
Copy link
Collaborator

@mattheworris mattheworris left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

  • Read through code
  • Executed make e2e-tests multiple times against a running node (make start)

@wilwade wilwade merged commit 8c8384f into main Nov 8, 2023
27 checks passed
@wilwade wilwade deleted the testing/final-fixes branch November 8, 2023 14:02
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

e2e-tests inconsistent
3 participants