Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add typing stubs to pre-commit mypy run #3266

Merged
merged 4 commits into from
Oct 11, 2022
Merged

Add typing stubs to pre-commit mypy run #3266

merged 4 commits into from
Oct 11, 2022

Conversation

akkomar
Copy link
Collaborator

@akkomar akkomar commented Oct 11, 2022

This is a follow-up to https://mozilla.slack.com/archives/C01E8GDG80N/p1665434432654689

We've seen mypy pre-commit hook failing and complaining about missing library stubs for some of the dependencies. Per pre-commit mirror docs it runs in an isolated environment so it seems to be making sense.

The issue might be related to #3248.

Changes proposed here do fix the mypy hook run for me on Python 3.10.7. However since I don't understand why we're seeing these errors I am not entirely sure if the local environment (mine and @kik-kik ) isn't at fault here.

Checklist for reviewer:

  • Commits should reference a bug or github issue, if relevant (if a bug is referenced, the pull request should include the bug number in the title)
  • If the PR comes from a fork, trigger integration CI tests by running the Push to upstream workflow and provide the <username>:<branch> of the fork as parameter. The parameter will also show up
    in the logs of the manual-trigger-required-for-fork CI task together with more detailed instructions.
  • If adding a new field to a query, ensure that the schema and dependent downstream schemas have been updated

For modifications to schemas in restricted namespaces (see CODEOWNERS):

@akkomar akkomar requested review from kik-kik and relud October 11, 2022 10:23
Copy link
Collaborator Author

@akkomar akkomar left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Lint (accidentally added via web editor)

Comment on lines +35 to +40
additional_dependencies: [types-pytz==2022.4.0.0,
types-ujson==5.5.0,
types-python-dateutil==2.8.19,
types-requests==2.28.11.2,
types-attrs==19.1.0,
types-PyYAML==6.0.12]
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These are actually duplicates of values from requirements.in which is not great.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I did notice types-attrs does not seem to be in the requirements file. I will try and see if adding it there is enough.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

looks like we need to specify the additional_dependencies...

@dataops-ci-bot
Copy link

Integration report for "Lint"

sql.diff

Click to expand!
Only in bigquery-etl-main/sql/moz-fx-data-shared-prod/activity_stream/on_save_recs: schema.yaml
Only in bigquery-etl-main/sql/moz-fx-data-shared-prod/burnham/discovery: schema.yaml
Only in bigquery-etl-main/sql/moz-fx-data-shared-prod/burnham/events: schema.yaml
Only in bigquery-etl-main/sql/moz-fx-data-shared-prod/eng_workflow/build: schema.yaml
Only in bigquery-etl-main/sql/moz-fx-data-shared-prod/eng_workflow/hgpush: schema.yaml
Only in bigquery-etl-main/sql/moz-fx-data-shared-prod/firefox_desktop/baseline: schema.yaml
Only in bigquery-etl-main/sql/moz-fx-data-shared-prod/firefox_desktop/deletion_request: schema.yaml
Only in bigquery-etl-main/sql/moz-fx-data-shared-prod/firefox_desktop/newtab: schema.yaml
Only in bigquery-etl-main/sql/moz-fx-data-shared-prod/firefox_desktop_background_update/baseline: schema.yaml
Only in bigquery-etl-main/sql/moz-fx-data-shared-prod/firefox_translations/deletion_request: schema.yaml
Only in bigquery-etl-main/sql/moz-fx-data-shared-prod/firefox_translations/events: schema.yaml
Only in bigquery-etl-main/sql/moz-fx-data-shared-prod/glean_dictionary/deletion_request: schema.yaml
Only in bigquery-etl-main/sql/moz-fx-data-shared-prod/mdn_yari/events: schema.yaml
Only in bigquery-etl-main/sql/moz-fx-data-shared-prod/messaging_system/moments: schema.yaml
Only in bigquery-etl-main/sql/moz-fx-data-shared-prod/messaging_system/spotlight: schema.yaml
Only in bigquery-etl-main/sql/moz-fx-data-shared-prod/mobile/activation: schema.yaml
Only in bigquery-etl-main/sql/moz-fx-data-shared-prod/mozdata/event: schema.yaml
Only in bigquery-etl-main/sql/moz-fx-data-shared-prod/mozilla_lockbox/addresses_sync: schema.yaml
Only in bigquery-etl-main/sql/moz-fx-data-shared-prod/mozilla_lockbox/baseline: schema.yaml
Only in bigquery-etl-main/sql/moz-fx-data-shared-prod/mozilla_lockbox/bookmarks_sync: schema.yaml
Only in bigquery-etl-main/sql/moz-fx-data-shared-prod/mozilla_lockbox/creditcards_sync: schema.yaml
Only in bigquery-etl-main/sql/moz-fx-data-shared-prod/mozilla_lockbox/deletion_request: schema.yaml
Only in bigquery-etl-main/sql/moz-fx-data-shared-prod/mozilla_lockbox/history_sync: schema.yaml
Only in bigquery-etl-main/sql/moz-fx-data-shared-prod/mozilla_lockbox/logins_sync: schema.yaml
Only in bigquery-etl-main/sql/moz-fx-data-shared-prod/mozilla_lockbox/tabs_sync: schema.yaml
Only in bigquery-etl-main/sql/moz-fx-data-shared-prod/mozilla_mach/baseline: schema.yaml
Only in bigquery-etl-main/sql/moz-fx-data-shared-prod/mozilla_mach/deletion_request: schema.yaml
Only in bigquery-etl-main/sql/moz-fx-data-shared-prod/mozilla_mach/events: schema.yaml
Only in bigquery-etl-main/sql/moz-fx-data-shared-prod/mozilla_mach/metrics: schema.yaml
Only in bigquery-etl-main/sql/moz-fx-data-shared-prod/mozilla_mach/usage: schema.yaml
Only in bigquery-etl-main/sql/moz-fx-data-shared-prod/mozillavpn/deletion_request: schema.yaml
Only in bigquery-etl-main/sql/moz-fx-data-shared-prod/mozillavpn/events: schema.yaml
Only in bigquery-etl-main/sql/moz-fx-data-shared-prod/mozillavpn/main: schema.yaml
Only in bigquery-etl-main/sql/moz-fx-data-shared-prod/mozphab/baseline: schema.yaml
Only in bigquery-etl-main/sql/moz-fx-data-shared-prod/mozphab/deletion_request: schema.yaml
Only in bigquery-etl-main/sql/moz-fx-data-shared-prod/mozphab/events: schema.yaml
Only in bigquery-etl-main/sql/moz-fx-data-shared-prod/mozphab/usage: schema.yaml
Only in bigquery-etl-main/sql/moz-fx-data-shared-prod/mozza/event: schema.yaml
Only in bigquery-etl-main/sql/moz-fx-data-shared-prod/org_mozilla_bergamot/custom: schema.yaml
Only in bigquery-etl-main/sql/moz-fx-data-shared-prod/org_mozilla_bergamot/deletion_request: schema.yaml
Only in bigquery-etl-main/sql/moz-fx-data-shared-prod/org_mozilla_bergamot/events: schema.yaml
Only in bigquery-etl-main/sql/moz-fx-data-shared-prod/org_mozilla_connect_firefox/baseline: schema.yaml
Only in bigquery-etl-main/sql/moz-fx-data-shared-prod/org_mozilla_connect_firefox/metrics: schema.yaml
Only in bigquery-etl-main/sql/moz-fx-data-shared-prod/org_mozilla_fenix/addresses_sync: schema.yaml
Only in bigquery-etl-main/sql/moz-fx-data-shared-prod/org_mozilla_fenix/baseline: schema.yaml
Only in bigquery-etl-main/sql/moz-fx-data-shared-prod/org_mozilla_fenix/bookmarks_sync: schema.yaml
Only in bigquery-etl-main/sql/moz-fx-data-shared-prod/org_mozilla_fenix/creditcards_sync: schema.yaml
Only in bigquery-etl-main/sql/moz-fx-data-shared-prod/org_mozilla_fenix/events: schema.yaml
Only in bigquery-etl-main/sql/moz-fx-data-shared-prod/org_mozilla_fenix/fog_validation: schema.yaml
Only in bigquery-etl-main/sql/moz-fx-data-shared-prod/org_mozilla_fenix/installation: schema.yaml
Only in bigquery-etl-main/sql/moz-fx-data-shared-prod/org_mozilla_fenix/migration: schema.yaml
Only in bigquery-etl-main/sql/moz-fx-data-shared-prod/org_mozilla_fenix/sync: schema.yaml
Only in bigquery-etl-main/sql/moz-fx-data-shared-prod/org_mozilla_fenix_nightly/activation: schema.yaml
Only in bigquery-etl-main/sql/moz-fx-data-shared-prod/org_mozilla_fenix_nightly/addresses_sync: schema.yaml
Only in bigquery-etl-main/sql/moz-fx-data-shared-prod/org_mozilla_fenix_nightly/baseline: schema.yaml
Only in bigquery-etl-main/sql/moz-fx-data-shared-prod/org_mozilla_fenix_nightly/deletion_request: schema.yaml
Only in bigquery-etl-main/sql/moz-fx-data-shared-prod/org_mozilla_fenix_nightly/events: schema.yaml
Only in bigquery-etl-main/sql/moz-fx-data-shared-prod/org_mozilla_fenix_nightly/history_sync: schema.yaml
Only in bigquery-etl-main/sql/moz-fx-data-shared-prod/org_mozilla_fenix_nightly/logins_sync: schema.yaml
Only in bigquery-etl-main/sql/moz-fx-data-shared-prod/org_mozilla_fenix_nightly/tabs_sync: schema.yaml
Only in bigquery-etl-main/sql/moz-fx-data-shared-prod/org_mozilla_fennec_aurora/activation: schema.yaml
Only in bigquery-etl-main/sql/moz-fx-data-shared-prod/org_mozilla_fennec_aurora/bookmarks_sync: schema.yaml
Only in bigquery-etl-main/sql/moz-fx-data-shared-prod/org_mozilla_fennec_aurora/deletion_request: schema.yaml
Only in bigquery-etl-main/sql/moz-fx-data-shared-prod/org_mozilla_fennec_aurora/events: schema.yaml
Only in bigquery-etl-main/sql/moz-fx-data-shared-prod/org_mozilla_fennec_aurora/migration: schema.yaml
Only in bigquery-etl-main/sql/moz-fx-data-shared-prod/org_mozilla_fennec_aurora/sync: schema.yaml
Only in bigquery-etl-main/sql/moz-fx-data-shared-prod/org_mozilla_firefox/activation: schema.yaml
Only in bigquery-etl-main/sql/moz-fx-data-shared-prod/org_mozilla_firefox/addresses_sync: schema.yaml
Only in bigquery-etl-main/sql/moz-fx-data-shared-prod/org_mozilla_firefox/baseline: schema.yaml
Only in bigquery-etl-main/sql/moz-fx-data-shared-prod/org_mozilla_firefox/bookmarks_sync: schema.yaml
Only in bigquery-etl-main/sql/moz-fx-data-shared-prod/org_mozilla_firefox/creditcards_sync: schema.yaml
Only in bigquery-etl-main/sql/moz-fx-data-shared-prod/org_mozilla_firefox/first_session: schema.yaml
Only in bigquery-etl-main/sql/moz-fx-data-shared-prod/org_mozilla_firefox/installation: schema.yaml
Only in bigquery-etl-main/sql/moz-fx-data-shared-prod/org_mozilla_firefox/logins_sync: schema.yaml
Only in bigquery-etl-main/sql/moz-fx-data-shared-prod/org_mozilla_firefox_beta/baseline: schema.yaml
Only in bigquery-etl-main/sql/moz-fx-data-shared-prod/org_mozilla_firefox_beta/creditcards_sync: schema.yaml
Only in bigquery-etl-main/sql/moz-fx-data-shared-prod/org_mozilla_firefox_beta/first_session: schema.yaml
Only in bigquery-etl-main/sql/moz-fx-data-shared-prod/org_mozilla_firefox_beta/fog_validation: schema.yaml
Only in bigquery-etl-main/sql/moz-fx-data-shared-prod/org_mozilla_firefox_beta/logins_sync: schema.yaml
Only in bigquery-etl-main/sql/moz-fx-data-shared-prod/org_mozilla_firefox_beta/metrics: schema.yaml
Only in bigquery-etl-main/sql/moz-fx-data-shared-prod/org_mozilla_firefox_beta/migration: schema.yaml
Only in bigquery-etl-main/sql/moz-fx-data-shared-prod/org_mozilla_firefox_beta/startup_timeline: schema.yaml
Only in bigquery-etl-main/sql/moz-fx-data-shared-prod/org_mozilla_firefox_vpn/baseline: schema.yaml
Only in bigquery-etl-main/sql/moz-fx-data-shared-prod/org_mozilla_firefox_vpn/deletion_request: schema.yaml
Only in bigquery-etl-main/sql/moz-fx-data-shared-prod/org_mozilla_firefox_vpn/metrics: schema.yaml
Only in bigquery-etl-main/sql/moz-fx-data-shared-prod/org_mozilla_firefoxreality/metrics: schema.yaml
Only in bigquery-etl-main/sql/moz-fx-data-shared-prod/org_mozilla_focus/baseline: schema.yaml
Only in bigquery-etl-main/sql/moz-fx-data-shared-prod/org_mozilla_focus/deletion_request: schema.yaml
Only in bigquery-etl-main/sql/moz-fx-data-shared-prod/org_mozilla_focus/fog_validation: schema.yaml
Only in bigquery-etl-main/sql/moz-fx-data-shared-prod/org_mozilla_focus/metrics: schema.yaml
Only in bigquery-etl-main/sql/moz-fx-data-shared-prod/org_mozilla_focus_beta/baseline: schema.yaml
Only in bigquery-etl-main/sql/moz-fx-data-shared-prod/org_mozilla_focus_beta/fog_validation: schema.yaml
Only in bigquery-etl-main/sql/moz-fx-data-shared-prod/org_mozilla_focus_nightly/activation: schema.yaml
Only in bigquery-etl-main/sql/moz-fx-data-shared-prod/org_mozilla_focus_nightly/deletion_request: schema.yaml
Only in bigquery-etl-main/sql/moz-fx-data-shared-prod/org_mozilla_focus_nightly/metrics: schema.yaml
Only in /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/org_mozilla_ios_fennec/baseline_clients_daily: schema.yaml
Only in bigquery-etl-main/sql/moz-fx-data-shared-prod/org_mozilla_ios_fennec/metrics: schema.yaml
Only in bigquery-etl-main/sql/moz-fx-data-shared-prod/org_mozilla_ios_fennec/temp_bookmarks_sync: schema.yaml
Only in bigquery-etl-main/sql/moz-fx-data-shared-prod/org_mozilla_ios_firefox/temp_clients_sync: schema.yaml
Only in bigquery-etl-main/sql/moz-fx-data-shared-prod/org_mozilla_ios_firefox/temp_history_sync: schema.yaml
Only in bigquery-etl-main/sql/moz-fx-data-shared-prod/org_mozilla_ios_firefox/temp_logins_sync: schema.yaml
Only in bigquery-etl-main/sql/moz-fx-data-shared-prod/org_mozilla_ios_firefoxbeta/events: schema.yaml
Only in bigquery-etl-main/sql/moz-fx-data-shared-prod/org_mozilla_ios_firefoxbeta/first_session: schema.yaml
Only in bigquery-etl-main/sql/moz-fx-data-shared-prod/org_mozilla_ios_firefoxbeta/temp_logins_sync: schema.yaml
Only in bigquery-etl-main/sql/moz-fx-data-shared-prod/org_mozilla_ios_firefoxbeta/temp_tabs_sync: schema.yaml
Only in bigquery-etl-main/sql/moz-fx-data-shared-prod/org_mozilla_ios_firefoxbeta/topsites_impression: schema.yaml
Only in bigquery-etl-main/sql/moz-fx-data-shared-prod/org_mozilla_ios_focus/deletion_request: schema.yaml
Only in bigquery-etl-main/sql/moz-fx-data-shared-prod/org_mozilla_ios_klar/deletion_request: schema.yaml
Only in bigquery-etl-main/sql/moz-fx-data-shared-prod/org_mozilla_klar/activation: schema.yaml
Only in bigquery-etl-main/sql/moz-fx-data-shared-prod/org_mozilla_klar/baseline: schema.yaml
Only in bigquery-etl-main/sql/moz-fx-data-shared-prod/org_mozilla_klar/deletion_request: schema.yaml
Only in bigquery-etl-main/sql/moz-fx-data-shared-prod/org_mozilla_klar/fog_validation: schema.yaml
Only in bigquery-etl-main/sql/moz-fx-data-shared-prod/org_mozilla_mozregression/events: schema.yaml
Only in bigquery-etl-main/sql/moz-fx-data-shared-prod/org_mozilla_mozregression/metrics: schema.yaml
Only in bigquery-etl-main/sql/moz-fx-data-shared-prod/org_mozilla_mozregression/usage: schema.yaml
Only in bigquery-etl-main/sql/moz-fx-data-shared-prod/org_mozilla_reference_browser/baseline: schema.yaml
Only in bigquery-etl-main/sql/moz-fx-data-shared-prod/org_mozilla_reference_browser/deletion_request: schema.yaml
Only in bigquery-etl-main/sql/moz-fx-data-shared-prod/org_mozilla_reference_browser/events: schema.yaml
Only in bigquery-etl-main/sql/moz-fx-data-shared-prod/org_mozilla_tv_firefox/baseline: schema.yaml
Only in bigquery-etl-main/sql/moz-fx-data-shared-prod/org_mozilla_tv_firefox/metrics: schema.yaml
Only in bigquery-etl-main/sql/moz-fx-data-shared-prod/org_mozilla_vrbrowser/baseline: schema.yaml
Only in bigquery-etl-main/sql/moz-fx-data-shared-prod/org_mozilla_vrbrowser/creditcards_sync: schema.yaml
Only in bigquery-etl-main/sql/moz-fx-data-shared-prod/org_mozilla_vrbrowser/deletion_request: schema.yaml
Only in bigquery-etl-main/sql/moz-fx-data-shared-prod/pine/baseline: schema.yaml
Only in bigquery-etl-main/sql/moz-fx-data-shared-prod/pine/deletion_request: schema.yaml
Only in bigquery-etl-main/sql/moz-fx-data-shared-prod/telemetry/addon_install_blocked: schema.yaml
Only in bigquery-etl-main/sql/moz-fx-data-shared-prod/telemetry/crash: schema.yaml
Only in bigquery-etl-main/sql/moz-fx-data-shared-prod/telemetry/disable_sha1rollout: schema.yaml
Only in bigquery-etl-main/sql/moz-fx-data-shared-prod/telemetry/downgrade: schema.yaml
Only in bigquery-etl-main/sql/moz-fx-data-shared-prod/telemetry/flash_shield_study: schema.yaml
Only in bigquery-etl-main/sql/moz-fx-data-shared-prod/telemetry/frecency_update: schema.yaml
Only in bigquery-etl-main/sql/moz-fx-data-shared-prod/telemetry/heartbeat: schema.yaml
Only in bigquery-etl-main/sql/moz-fx-data-shared-prod/telemetry/installation: schema.yaml
Only in bigquery-etl-main/sql/moz-fx-data-shared-prod/telemetry/modules: schema.yaml
Only in bigquery-etl-main/sql/moz-fx-data-shared-prod/telemetry/prio: schema.yaml
Only in bigquery-etl-main/sql/moz-fx-data-shared-prod/telemetry/shield_study: schema.yaml
Only in bigquery-etl-main/sql/moz-fx-data-shared-prod/telemetry/shield_study_error: schema.yaml
Only in bigquery-etl-main/sql/moz-fx-data-shared-prod/telemetry/third_party_modules: schema.yaml
Only in bigquery-etl-main/sql/moz-fx-data-shared-prod/telemetry/tls13_middlebox_testing: schema.yaml
Only in bigquery-etl-main/sql/moz-fx-data-shared-prod/telemetry/untrusted_modules: schema.yaml
Only in bigquery-etl-main/sql/moz-fx-data-shared-prod/viu_politica/regret_details: schema.yaml
Only in bigquery-etl-main/sql/moz-fx-data-shared-prod/viu_politica/video_index: schema.yaml

@dataops-ci-bot
Copy link

Integration report for "Lint"

sql.diff

No content detected.

@kik-kik
Copy link
Contributor

kik-kik commented Oct 11, 2022

I can confirm using this config solves the errors on my machine

@akkomar akkomar marked this pull request as ready for review October 11, 2022 15:51
@akkomar akkomar enabled auto-merge (squash) October 11, 2022 16:01
@akkomar akkomar merged commit 2330cc1 into main Oct 11, 2022
@akkomar akkomar deleted the akkomar-patch-1 branch October 11, 2022 16:06
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants