Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

consistent job / pytest report / artifact name correspondence #30392

Merged
merged 4 commits into from
Apr 24, 2024

Conversation

ydshieh
Copy link
Collaborator

@ydshieh ydshieh commented Apr 22, 2024

What does this PR do?

Nothing big in this PR: just change some job names (or more precisely, job id) and artifact names.

The rule is simple: the test report (directory) name and the artifact name should correspond to the job id with a post-fix test_reports.

For example:

jobs:
  run_models_gpu:  # (job id)
  ...
  steps:
    - name:  run_test
      run: pytest ... --make-reports=run_models_gpu_test_reports  # (pytest report)

      - name: "Test suite reports artifacts: run_models_gpu_test_reports"  # (artifact)
       ... 
       with:
          name: run_models_gpu_test_reports  # (artifact)
          path: /transformers/reports/run_models_gpu_test_reports  # (pytest report)

If there is matrix involved like machine_type and/or (model) folders, it should be

${{ matrix.machine_type }}_run_models_gpu_${{ matrix.folders }}_test_reports

@ydshieh
Copy link
Collaborator Author

ydshieh commented Apr 22, 2024

cc @younesbelkada , @SunMarc and @fxmarty

(not to ping you for review, but to let you know about this change)

@HuggingFaceDocBuilderDev

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

@ydshieh
Copy link
Collaborator Author

ydshieh commented Apr 22, 2024

I have verified the changes work for daily (schedule) ci.

For other CI like push, nightly, past and amd, they all need rework since we pass the 256 models (daily-ci is OK after #28773, but this hasn't applied to other CI).

I will rework other CI progressively.

@ydshieh ydshieh requested a review from amyeroberts April 22, 2024 15:13
Copy link
Collaborator

@amyeroberts amyeroberts left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for making all the job names consistent!

@ydshieh ydshieh force-pushed the better_job_artifact_name branch from 0e9ac57 to ef0f2fc Compare April 24, 2024 13:39
@ydshieh ydshieh force-pushed the better_job_artifact_name branch from 9107220 to 2e7c1c1 Compare April 24, 2024 20:00
@ydshieh ydshieh merged commit fbb41cd into main Apr 24, 2024
8 checks passed
@ydshieh ydshieh deleted the better_job_artifact_name branch April 24, 2024 20:32
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants