Adds the basic logging capabilities for Omega

[grnydawn]

This PR add the basic logging capabilities for Omega. These includes:

• saving log in a log file in a build directory or to a user specified path.
• Logging macros whose name starts with "LOG_"
• Multiple logging levels: TRACE, DEBUG, INFO, WARN, WARN, ERROR, and CRITICAL
• Custom log formatter for YAKL data types(experimental yet)

Checklist

• Documentation:
  • Design document has been generated and added to the docs
  • User's Guide has been updated
  • Developer's Guide has been updated
  • Documentation has been built locally and changes look as expected
• Testing
  • A comment in the PR documents testing used to verify the changes including any tests that are added/modified/impacted.
  • CTest unit tests for new features have been added per the approved design.
  • Polaris tests for new features have been added per the approved design (and included in a test suite)
  • Unit tests have passed. Please provide a relevant CDash build entry for verification.
  • Polaris test suite has passed
  • Performance related PRs: Please include a relevant PACE experiment link documenting performance before and after.
• [ ] Stealth Features
  • If any stealth features are included in the PR, please confirm that they have been documented.

This PR implements only the basic capabilities of logging. Following PR should includes E3SM logging integration, Multi-process support, multi-thread support, buffered/unbuffered logging and more stable YAKL data type support.

[grnydawn]

I forgot to apply clang-format. I will update the PR soon after updating code format using clang-format.

[philipwjones]

@grnydawn the actual functionality looks good. Most comments are just trying to establish the consistent style for OMEGA. Thanks

[grnydawn]

@xylar , I got an CI/CD check error as following. Could you look at it? I have no idea how to fix this error:

Run trilom/file-changes-action@v1.2.3
Warning: Received event from pull_request, but also received a before(f79027e) or after(61ade58) value.
I am assuming you want to use a Push event but forgot something, so I'm giving you a message.
Error: There was an issue sorting changed files from Github.
Exception: {
"error": "500/TypeError",
"from": "sortChangedFiles",
"message": "There was an issue sorting files changed.",
"payload": "{}"
}
/home/runner/work/_actions/trilom/file-changes-action/v1.2.3/dist/index.js:1
with:
output:
githubToken: ***
fileOutput: json
env:
CANCEL_OTHERS: true
PATHS_IGNORE: ["/README.md", "/doc/**"]
INPUT_RUN_POST: true
CONDA: /usr/share/miniconda3
CONDA_PKGS_DIR: /home/runner/conda_pkgs_dir
Warning: Received event from pull_request, but also received a before(f79027e7b6d68afed74be47c51a472c80f9bfe22) or after(61ade583d84668b0a12200c04d29d05) value.
I am assuming you want to use a Push event but forgot something, so I'm giving you a message.
Error: There was an issue sorting changed files from Github.
Exception: {
"error": "500/TypeError",
"from": "sortChangedFiles",
"message": "There was an issue sorting files changed.",
"payload": "{}"
}
/home/runner/work/_actions/trilom/file-changes-action/v1.2.3/dist/index.js:1

[grnydawn]

@philipwjones , while I am having an CI check issue, I think I have done updating logging per suggestions here. Could you check this if it is ready to merge?

I have not implemented all of the test cases noted in logging design document yet because I think I will add more tests(especially multi-processing and multi-threading tests) after I finish merging my updates in Cmake-based build.

[xylar]

@grnydawn, that's nothing you did. It's a CI problem. If I have permission, I'll push a fix to your branch.

[xylar]

@grnydawn, I don't have permission to push to your branch so I'll make a separate PR to fix this.

[xylar]

@grnydawn, you can rebase onto #30 if you want.

[grnydawn]

@xylar Thanks for the quick fix. I will try to rebase it.

[grnydawn]

@philipwjones , the CI issue is now resolved. I think I have implemented all the change requests in this PR.

[xylar]

@grnydawn, thanks for the changes and in general for the hard work on this. I'm good with it going in now.

[philipwjones]

@grnydawn thanks for the changes. Almost there, but two more little things: First, I'm getting a FAIL in your unit test for YAKL type 1 so CTest is failing. I'm not sure it'll be that common to be logging full YAKL arrays so it's not a huge deal, but we should have a passing unit test before merging. Second, I think the test directory should mirror the src directory, so we should move the logging unit test driver into test/infra (and the path modified accordingly in the Cmake test definition).

[grnydawn]

@philipwjones , Thanks for the comments. In my ctest, all tests were passed. Could you share "Testing/Temporary/LastTest.log" under the output directory with me? I will move the logging test driver into test/infra. Thanks.

[philipwjones]

Here's the output - not much info other than the first YAKL test failed. This was a test I ran on Summit with nvidia.

Start testing: Aug 21 18:43 EDT

1/1 Testing: LOGGING_TEST
1/1 Test: LOGGING_TEST
Command: "/gpfs/alpine/scratch/pwjones/cli115/omega/test/testLogging.exe"
Directory: /gpfs/alpine/scratch/pwjones/cli115/omega/test
"LOGGING_TEST" start time: Aug 21 18:43 EDT
Output:

Default log level 1: PASS
Default log level 2: PASS
Yakl data type 1: FAIL
Yakl data type 2: PASS
Pool Memory High Water Mark: 0
Pool Memory High Water Efficiency: 0

Test time = 0.00 sec

Test Fail Reason:
Error regular expression found in output. Regex=[FAIL]
"LOGGING_TEST" end time: Aug 21 18:43 EDT
"LOGGING_TEST" time elapsed: 00:00:00

[grnydawn]

@philipwjones Thanks for the info. There was a bug in my log formatter for yakl. I fixed it now and the logging test driver is moved to test/infra now. Thanks!

[philipwjones]

Thanks for the fixes @grnydawn I'll approve now

[grnydawn]

@philipwjones Thanks. Could you merge this PR to develop. It looks that I am not authorized to merge.

[philipwjones]

@grnydawn Can you please rebase to develop? There are some old commits here and when I tried to do the automatic rebase-and-merge, it reported some conflicts. If you can do a rebase, then force push to this branch, that would be great. And if you want to squash commits along the way for an even cleaner PR, that would be even better... Thanks.

[grnydawn]

@philipwjones I tried to squash the updates in develop branch to this PR . But as I am not familiar with the git commands, there could be some mistake. I have just pushed the changes and let me know if any changes is needed.

[philipwjones]

@grnydawn Sorry if I was unclear or confused things. You needed to do a rebase to develop first to eliminate all the unrelated commits that you had on your branch from other merges so that you only had the commits for the Logging itself. Then after that rebase, do a squash to reduce the number of commits that were only related to this PR. Unfortunately, now you have a bit of a mess because all the unrelated commits are also squashed. Probably the best thing to do now is to create a fresh branch from the latest develop and then cherry-pick the commits that were for this Logging functionality.

[xylar]

@grnydawn, I have posted detailed instructions for you on Slack of how I would clean this up with a rebase. Please let me know if that works for you or if I can help.

[grnydawn]

@philipwjones , sorry for the mess. I will see Xylar's comments on Slack and see what is the best way to fix this.

@xylar, Thanks a lot for the details explanations. I will try to follow the instructions. One question: my remote branch of ykim/omega/logging is already filled with unnecessary commits and history. I wonder if you expect me to correct them using your instructions, or are the instructions for the future use? For now, it seems to be something doable for me to create a new branch only for this PR as Phil suggested.

[xylar]

One question: my remote branch of ykim/omega/logging is already filled with unnecessary commits and history. I wonder if you expect me to correct them using your instructions, or are the instructions for the future use? For now, it seems to be something doable for me to create a new branch only for this PR as Phil suggested.

My instructions are for fixing the existing branch, ykim/omega/logging. There is no need to start with a new branch. You can fix what you have. You can run git branch temp before you start if you like. Then you can always do git reset --hard temp later to get back to where you started.

[xylar]

I think it is worth taking the time to do a rebase as I suggested on Slack because this is an important capability for you to have and something you will reuse constantly while you are working on Omega.

[philipwjones]

@grnydawn - no worries. Everything in git is fixable and @xylar is more of an expert than I am so hopefully his instructions get you back on track. I will second Xylar's comment that the rebase workflow is really important and it keeps the commit history clean, so no matter how you end up fixing this, getting familiar with rebase is critical.

As always, when you get frustrated with git, visit the xkcd comic's take on git: https://xkcd.com/1597/ - especially the alt text you get when you hover over the image...

[grnydawn]

Thanks @xylar and @philipwjones . I will try to fix it and see how far I can go ^^

[grnydawn]

@xylar and @philipwjones , I think I have done the fixing per Xylar's instructions. Hope this works but, otherwise, pls feel free to let me know what I can do. Thanks a lot for the helps.

[xylar]

The rebase and squash looks good to me.

[philipwjones]

Yep, looks good. I'll take another quick look, then go ahead and merge this. Thanks for fixing.