Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add mssql integration tests and relocate existing unit tests #39831

Merged
merged 2 commits into from
May 27, 2024

Conversation

shahar1
Copy link
Contributor

@shahar1 shahar1 commented May 25, 2024

closes: #38577

This PR:

  • Relocates existing unit tests for mssql from the integration directory back to the unit tests (see Restore providers' unit tests marked as integration tests #39768)
  • Creates a Docker Compose integration yaml file for mssql.
  • Adds an integration test for mssql hook.
  • Adds integration tests for:
    • bigquery_to_mssql.py
    • mssql_to_gcs.py
    • mssql_to_hive.py

cc: @potiuk, @Taragolis, @eladkal


^ Add meaningful description above
Read the Pull Request Guidelines for more information.
In case of fundamental code changes, an Airflow Improvement Proposal (AIP) is needed.
In case of a new dependency, check compliance with the ASF 3rd Party License Policy.
In case of backwards incompatible changes please leave a note in a newsfragment file, named {pr_number}.significant.rst or {issue_number}.significant.rst, in newsfragments.

@shahar1 shahar1 force-pushed the mssql-integration-tests branch 2 times, most recently from f83ab8f to 4e900b1 Compare May 25, 2024 20:20
@shahar1 shahar1 force-pushed the mssql-integration-tests branch from 4e900b1 to 29e3701 Compare May 25, 2024 20:26
@shahar1 shahar1 changed the title Add mssql integration tests Add mssql integration tests and relocate existing unit tests May 25, 2024
@potiuk
Copy link
Member

potiuk commented May 26, 2024

Nice. - few nits and solving failing tests (I guess some rebase needed?)

@shahar1 shahar1 force-pushed the mssql-integration-tests branch from 29e3701 to c0ca8c5 Compare May 26, 2024 17:00
@potiuk
Copy link
Member

potiuk commented May 26, 2024

One more thing: you also need to add Mssql as individual integration to test in .github/workflows/integration-tests.yml. This way the integration tests will be run on public runners (only in self-hosted runners all integrations are run together).

@shahar1 shahar1 force-pushed the mssql-integration-tests branch from c0ca8c5 to 42d4c49 Compare May 26, 2024 19:38
@shahar1 shahar1 requested a review from kaxil as a code owner May 26, 2024 19:38
@potiuk
Copy link
Member

potiuk commented May 27, 2024

Nice!

@potiuk potiuk merged commit f067727 into apache:main May 27, 2024
69 checks passed
@potiuk
Copy link
Member

potiuk commented May 27, 2024

@eladkal @shahar1 I also added #39866 as follow-up -> I realized that the last step was not obvious (having to add integration test step to integration-tests.yml was quite a bit duplication - also recently the "self-hosted" "all-testable" integrations when all of the integrations are started together have been somewhat flaky (as opposed to individual integrations) - so the #39866 will simplify it: removes the duplication and always runs the integration tests individually on public runners for both mssql and postgres.

@utkarsharma2 utkarsharma2 added the changelog:skip Changes that should be skipped from the changelog (CI, tests, etc..) label Jun 3, 2024
fdemiane pushed a commit to fdemiane/airflow that referenced this pull request Jun 6, 2024
…e#39831)

* Add MsSQL integration tests and relocate unit tests

* Update .github/workflows/integration-tests.yml

---------

Co-authored-by: Jarek Potiuk <jarek@potiuk.com>
@shahar1 shahar1 deleted the mssql-integration-tests branch June 12, 2024 13:13
romsharon98 pushed a commit to romsharon98/airflow that referenced this pull request Jul 26, 2024
…e#39831)

* Add MsSQL integration tests and relocate unit tests

* Update .github/workflows/integration-tests.yml

---------

Co-authored-by: Jarek Potiuk <jarek@potiuk.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area:dev-tools area:providers changelog:skip Changes that should be skipped from the changelog (CI, tests, etc..) provider:microsoft-mssql
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Add mssql test integration
4 participants