Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add github action to report memory diff user diffsyms tool #6962

Merged
merged 1 commit into from
Jun 9, 2021

Conversation

kghost
Copy link
Contributor

@kghost kghost commented May 19, 2021

Problem

The PR add a github actions job to automatically post the memory difference of a new PR. It help us to diagnose memory usage.

Change overview

After building the new image, it download the prebuild base image from master branch, and use diffsyms tool to generate a diff, then call post_comment script to add a comment.

Testing

This is the example result of this PR kghost#3

if: ${{ github.event_name == 'pull_request' }}
with:
workflow: examples-esp32.yaml
name: esp32-example-build-${{ github.event.pull_request.base.ref }}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What will be the effect if the base binaries are not available (we are frequently running out of quota so have quite aggressive cleanup scripts)?

Trying to ensure that our CI does not turn red on this.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As our master branch is moving forward, new base binaries will be build for every commit. One can rebase his PR on the newest master to get a new image.

This part is not essential, so even it fails, it doesn't cause any trouble.

And I have several other ideas to reduce the usage of artifacts. Let me investigate them later

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I guess my comment would be translated to:

Do you need a continue-on-error: true here?

@andy31415
Copy link
Contributor

Please fill up the 'Summary' for this PR. Especially important: how did you test? The Github script should be manually testable at least in some PR where we could see the expected output.

@kghost
Copy link
Contributor Author

kghost commented May 19, 2021

The action runs on the pull_request event, so I can't test it in my own repo.

When the actions are done, it will post a comment in this PR, let us wait and check whether it appears.

@kghost
Copy link
Contributor Author

kghost commented May 20, 2021

Here is a example result of this PR

kghost#3

The workflow in this PR is not working. Unless the PR is merged, otherwise github uses a read-only credential to run the proposal workflow.

Copy link
Contributor

@woody-apple woody-apple left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Per the updated template, can you update the PR here?

#### Problem
What is being fixed?  Examples:
* Fix crash on startup
* Fixes #12345 12345 Frobnozzle is leaky (exactly like that, so GitHub will auto-close the issue).

#### Change overview
What's in this PR

#### Testing
How was this tested? (at least one bullet point required)
    • If unit tests were added, how do they cover this issue?
    • If unit tests existed, how were they fixed/modified to prevent this in future?
    • If integration tests were added, how do they verify this change?
    • If manually tested, what platforms controller and device platforms were manually tested, and how?
    • If no testing is required, why not?

@kghost
Copy link
Contributor Author

kghost commented May 20, 2021

Per the updated template, can you update the PR here?

@woody-apple Updated, thanks

@kpschoedel
Copy link
Contributor

Here is a example result of this PR

kghost#3

Yikes, diffsyms is clearly not handling duplicate-named symbols adequately. I'll take a look unless someone else wants to (I suspect adding size as a secondary sort key will make them match up better).

@kghost
Copy link
Contributor Author

kghost commented May 20, 2021

Yikes, diffsyms is clearly not handling duplicate-named symbols adequately. I'll take a look unless someone else wants to (I suspect adding size as a secondary sort key will make them match up better).

This can be a separate issue.

# continue
if not comment.body.startswith(args.title):
continue
logging.info('Removing obsolete comment with heading "%s"', (args.title))
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This seems like a pretty sharp knife: if someone accidentally passes in empty string, this will delete all the comments, no?

I'd prefer it if all comments posted by this script had a known prefix from the script itself (which gets prepended to args.title).

Also, this is presumably copied in large part from bloat_check.py. How feasible would it be to make that script use this one?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let me re-enable the user id check, to see if we can guard this one

@andy31415

This comment has been minimized.

@andy31415 andy31415 requested a review from woody-apple May 21, 2021 15:07
- name: Report symbol diff
if: ${{ github.event_name == 'pull_request' }}
run: |
pip3 install cxxfilt pyelftools humanfriendly pandas && \
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

should or could we install these packages in the docker image so that we do not run pip install on every commit?

We are trying to make CI faster. If possible, please create a separate PR that updates dockerfile and version.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I updated scripts/requirements.in for this yesterday (#6975) so they should get in to future rolls.

@andy31415
Copy link
Contributor

@woody-apple the summary was updated to match the template

@stale
Copy link

stale bot commented Jun 1, 2021

This pull request has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs.

@stale stale bot added the stale Stale issue or PR label Jun 1, 2021
@stale stale bot removed the stale Stale issue or PR label Jun 3, 2021
@woody-apple
Copy link
Contributor

/rebase

Copy link
Contributor

@andy31415 andy31415 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good, but would like to merge only once we ensure this does not turn PRs red if we cannot compute the difference.

I imagine in time we may want to tune verbosity as well.

@woody-apple woody-apple merged commit 1f1b715 into project-chip:master Jun 9, 2021
@andy31415
Copy link
Contributor

This was merged before we ensured PRs were not moved to red when we compute the difference (PRs fail on 'baseline download failed).

Will revert @kghost

andy31415 added a commit to andy31415/connectedhomeip that referenced this pull request Jun 11, 2021
andy31415 added a commit that referenced this pull request Jun 11, 2021
nikita-s-wrk pushed a commit to nikita-s-wrk/connectedhomeip that referenced this pull request Sep 23, 2021
nikita-s-wrk pushed a commit to nikita-s-wrk/connectedhomeip that referenced this pull request Sep 23, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants