Add support for performance archiving on Compy #2943
Conversation
Added logic for compy to provenance.py, specified the location of the performance archive in the compy entry in config_machines.xml (enabled for all projects) and added the job progress monitoring script syslog.compy. Also removed performance archiving support for edison, now that it has been decommissioned. [BFB] - Bit-For-Bit
|
@rljacob , please reassign reviewers and integrator as appropriate. Thanks. |
There was a problem hiding this comment.
Code looks right. Someone else with token can test on Compy.
Since /compyfs is the the top level SAVE_TIMING_DIR, I presume 'performance_archive' will be created under that.
Just a clarification: Is that path limited to e3sm project only? If not, it's probably better to have e3sm specific suffix to differentiate from other BER projects if we have shared access.
|
Hmm... I see that /compyfs (without e3sm suffix) is being used everywhere in the machine file. So, perhaps that's not a concern. Default permissions enable group write to performance_archive. So, it's fine as long as non-E3SM users can't meddle with it. |
What is your concern? Vandalism of some sort? Shouldn't be anything sensitive or critical being written to the performance archive. |
|
I was just thinking of accidental deletes and thought /compfs/performance_archive may not convey that it's related to e3sm. I'm used to our project specific directories at the LCFs. Anyway, I see /compyfs used everywhere in the machine file. So, this is not a problem. |
|
@sarats , thanks for asking about this. This is a BER system, and the accounts (once we start using them consistently) will allow us to differentiate between jobs in the performance archive. For example, at NERSC all supported projects (acme, m2830, m2833) archive in the same location. This isn't supported at the OLCF because of the directory permissions policies for proj-shared. |
|
Would be nice to get this intergrated before people jump on and start using the system. @singhbalwinder and @bibiraju , could you please do quick reviews (or ask to be unassigned)? Thanks. |
|
Thanks @bibiraju , @singhbalwinder , and @sarats . This is ready to go @jgfouca . |
|
Reassigning this to @bibiraju since its Compy related and JimF is pretty busy. |
Add support for performance archiving on Compy #2943 Added logic for compy to provenance.py, specified the location of the performance archive in the compy entry in config_machines.xml (enabled for all projects) and added the job progress monitoring script syslog.compy. Also removed performance archiving support for edison, now that it has been decommissioned. [BFB] - Bit-For-Bit
Add support for performance archiving on Compy #2943 Added logic for compy to provenance.py, specified the location of the performance archive in the compy entry in config_machines.xml (enabled for all projects) and added the job progress monitoring script syslog.compy. Also removed performance archiving support for edison, now that it has been decommissioned. [BFB] - Bit-For-Bit
Add support for performance archiving on Compy #2943 Added logic for compy to provenance.py, specified the location of the performance archive in the compy entry in config_machines.xml (enabled for all projects) and added the job progress monitoring script syslog.compy. Also removed performance archiving support for edison, now that it has been decommissioned. [BFB] - Bit-For-Bit
Automatically Merged using E3SM Pull Request AutoTester PR Title: Frontier: disable hipInit before MPI_Init PR Author: dqwu PR LABELS: cmake, AT: AUTOMERGE, AT: Integrate Without Testing, Frontier
Automatically Merged using E3SM Pull Request AutoTester PR Title: Frontier: disable hipInit before MPI_Init PR Author: dqwu PR LABELS: cmake, AT: AUTOMERGE, AT: Integrate Without Testing, Frontier
Added logic for compy to provenance.py, specified
the location of the performance archive in the
compy entry in config_machines.xml (enabled for
all projects) and added the job progress monitoring
script syslog.compy. Also removed performance
archiving support for edison, now that it has been
decommissioned.
[BFB] - Bit-For-Bit