-
Notifications
You must be signed in to change notification settings - Fork 428
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Rearranged service_domainn_db_SUITE to run more tests in parallel #4176
Conversation
This comment was marked as outdated.
This comment was marked as outdated.
Codecov ReportAll modified and coverable lines are covered by tests ✅
Additional details and impacted files@@ Coverage Diff @@
## master #4176 +/- ##
==========================================
+ Coverage 84.27% 84.31% +0.04%
==========================================
Files 551 551
Lines 33467 33467
==========================================
+ Hits 28204 28219 +15
+ Misses 5263 5248 -15 ☔ View full report in Codecov by Sentry. |
44a493b
to
8d865bc
Compare
This comment was marked as outdated.
This comment was marked as outdated.
8d865bc
to
62a8081
Compare
This comment was marked as outdated.
This comment was marked as outdated.
62a8081
to
90ad055
Compare
This comment was marked as outdated.
This comment was marked as outdated.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
That's some nice rework, groups/0
is now so much more readable, and tests are nicely separated in better classified groups. Also, nice we can parallelise so many things!
Only two small comments in the random domain generation below :)
elasticsearch_and_cassandra_26 / elasticsearch_and_cassandra_mnesia / 71a2523 small_tests_25 / small_tests / 71a2523 small_tests_26 / small_tests / 71a2523 small_tests_26_arm64 / small_tests / 71a2523 dynamic_domains_pgsql_mnesia_25 / pgsql_mnesia / 71a2523 ldap_mnesia_25 / ldap_mnesia / 71a2523 ldap_mnesia_26 / ldap_mnesia / 71a2523 dynamic_domains_mysql_redis_26 / mysql_redis / 71a2523 dynamic_domains_pgsql_mnesia_26 / pgsql_mnesia / 71a2523 internal_mnesia_26 / internal_mnesia / 71a2523 dynamic_domains_mssql_mnesia_26 / odbc_mssql_mnesia / 71a2523 graphql_muc_light_SUITE:admin_cli:admin_muc_light:end_per_group{error,
{{unregistering_failed,
{amount,2},
{unregistered_items,
[{{<<"_admin_invite_user_1064">>,
[{escalus_event_mgr,<0.24144.0>},
{tc_name,admin_invite_user},
{escalus_cleaner,<0.24143.0>},
{watchdog,<0.24142.0>},
{muc_light_host,<<"muclight.domain.example.com">>},
{secondary_muc_light_host,<<"muclight.domain.example.org">>},
{protocol,cli},
{schema_endpoint,admin},
{{ejabberd_cwd,mongooseim@localhost},
"/home/circleci/project/_build/mim1/rel/mongooseim"},
{preset,"odbc_mssql_mnesia"},
{mim_data_dir,
"/home/circleci/project/big_tests/tests/graphql_muc_light_SUITE_data"},
{tc_logfile,
"https://circleci-mim-results.s3.eu-central-1.amazonaws.com/PR/4176/203834/odbc_mssql_mnesia.26.1.2-amd64/big/ct_run.test%40aba79982b5e2.2023-12-18_10.56.55/big_tests.tests.graphql_muc_light_SUITE.logs/run.2023-12-18_11.01.04/graphql_muc_light_suite.admin_invite_user.50178.html"},
{tc_group_properties,[{name,admin_muc_light}]},
{tc_group_path,[[{name,admin_cli}]]},
{data_dir,
"/home/circleci/project/big_tests/_build/default/lib/mongoose_tests/ebin/graphql_muc_light_SUITE_data/"},
{priv_dir,
"https://circleci-mim-results.s3.eu-central-1.amazonaws.com/PR/4176/203834/odbc_mssql_mnesia.26.1.2-amd64/big/ct_run.test%40aba79982b5e2.2023-12-18_10.56.55/big_tests.tests.graphql_muc_light_SUITE.logs/run.2023-12-18_11.01.04/log_private/"},
{{saved_modules,mongooseim@localhost,<<"test type">>},
#{mod_vcard =>
#{matches => 30,
host => {prefix,<<"vjud.">>},
search => true,iqdisc => parallel,backend => rdbms},
... mysql_redis_26 / mysql_redis / 71a2523 pgsql_cets_26 / pgsql_cets / 71a2523 pgsql_mnesia_25 / pgsql_mnesia / 71a2523 pgsql_mnesia_26 / pgsql_mnesia / 71a2523 mssql_mnesia_26 / odbc_mssql_mnesia / 71a2523 |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
👌🏽
This PR refactors service_domainn_db_SUITE. New grouping of test cases is introduced. Some of the tests are using randomised domain names so they don't interfere each other.
For MSSQL the test are being run sequentially because of the random fails (due to poor dev container with MSSQL that we use on the CI). To finally check if we don't introduce new flaky tests I ran CI multiple times and none of the time this suite failed - https://app.circleci.com/pipelines/github/esl/MongooseIM/11803.
There are flaky tests in other suites though. I'd be nice if we addressed it in the future.