Cmake system for MPAS

[jgfouca]

This PR switches E3SM's default build-system to be cmake-based. It does not alter the current standard MPAS make system.

[jgfouca]

This will need to be merged before:
E3SM-Project/E3SM#3043

[xylar]

@jgfouca, maybe you've already discussed this with the MPAS team and I'm just not in the loop.

My understanding is that we don't make pull requests directly into e3sm/develop and that these changes would need PRs into develop, ocean/develop, seaice/develop and landice/develop for different pieces of these changes. Then, e3sm/develop would be updated by merging each of these into e3sm/develop.

@mark-petersen, @jonbob and others, please correct me if I'm wrong.

Also, this is something we may need to discuss at our MPAS developers' meeting on Monday.

[jonbob]

I think @xylar is correct -- one way or another we'll have to get the changes into each core, so this will become several PR's within MPAS. But I don't think we should make @jgfouca create all those separate PR's -- we can take that on. Thanks @xylar for pointing this out

[jgfouca]

@xylar , there's only one mpas submodule now, right?

[submodule "components/mpas-source"]
	path = components/mpas-source
	url = git@github.com:MPAS-Dev/MPAS-Model.git

How would I have different branches for the various cores from E3SM's point of view?

[jgfouca]

Nevermind, I see what you mean.

[jonbob]

@jgfouca - there are machinations about how we pull that single submodule together that require this to be three MPAS PR's under-the-covers, and then a single one into E3SM

[xylar]

@jonbob, unfortunately, I think it will be 4 PRs, since we need one into develop as well (probably before the ones in the individual cores). @jgfouca, yep, we'll take care of this on our side so you don't have to worry about the details. I just wanted to warn you that it'll be a little bit of a process.

[jgfouca]

@xylar , @jonbob , thanks for the help. Let me make sure the CMake still works when I do an upstream merge from develop. I had this working a few months ago, but I assume there have been changes to MPAS in the meantime.

[jgfouca]

@xylar , @jonbob ,

I did an upstream merge from the latest submodule HEAD commit. There were only a couple very minor changes needed to CMake to reflect changes made to Makefiles since July.

This PR should be ready to merge.

[jgfouca]

Whoops, looks like there's an issue with per-component flags with MPAS. I'll let you know when it's fixed.

[mgduda]

@jgfouca Would you be available this coming Monday, 4 November at 11:00 MT? We'll be having a developers's telecon, and one of the items to discuss is an interest in a CMake build for MPAS-Atmosphere as well. It would be good to see where our efforts might complement or conflict with each other.

[jgfouca]

@mgduda , yes, I'm available. Can you email me an invite?

[jgfouca]

@xylar , @jonbob , this is ready for merge.

[mark-petersen]

@jgfouca thanks for your work on this. I changed the base branch for this PR from e3sm/develop to develop. I copied the files over and separated commits by core, as that might be useful later.

@xylar, in this case I think it's OK to have a single PR for all brand-new files into each core's directory. After this is merged we will merge develop back into each core, which is simpler than 4 separate PRs.

[jgfouca]

@xylar ,

Oh, man! I think it's quite dangerous to go the easy way here. This would interfere in all kinds of ways with any future plan to use cmake to build standalone MPAS components, which I think we want to have as a goal or at least a possibility. Another thing to discuss on Monday.

Yeah, I definitely don't want to conflict with existing CMake efforts. My understanding is that no such effort/plan for MPAS exists; maybe that's changed since summer.

If MPAS does want a CMake build system, then it should be pretty easy to leverage the work I've done to do the builds of the cores and framework, so you guys would only need to set up the root CMakeLists.txt file.

[mark-petersen]

Notes from telecon today:

1. We requested that information for each core, framework, and operators directory is pushed down into each of those subdirectories, probably as include files.
2. @jgfouca could then complete this PR in the short term (~ 1 week) so this PR can get merged.
3. For the next many months, we will have these CMake files without a standalone CMake process. During that time, we will work on creating the stand-alone CMake top level file. @jgfouca volunteered to help us set up a template for that, and we would help fill in the details of each machine and compiler.

[jgfouca]

@mark-petersen , I've added a couple commits addressing the topics in the telecon. In my opinion, this is ready to be merged.

One thing I should point out is that this branch stopped working after you rebased it. The rebase point differs significantly from what the submodule was set to and it no longer works with E3SM. I had to continue development from the original branch and I cherry-picked the additional commits onto this branch. I'm totally unfamiliar with MPAS' development workflow, so maybe this expected.

[mark-petersen]

@jgfouca thanks for your work on this. Yes, the MPAS workflow is confusing. This PR will be merged into MPAS-Dev/develop, and that into MPAS-Dev/e3sm/develop, which ends up in mpas-source in e3sm.

Your cherry-picking is fine. If you want to further test this PR in e3sm you can cd into components/mpas-source, add your fork, merge in this PR in locally.

[mark-petersen]

Based on visual inspection, @jgfouca fulfilled our request of keeping file information in each subdirectory within src. When we get approval from a few others on the list, I'll go ahead and merge.

[jgfouca]

Thanks, @mark-petersen ! I hope the other reviewers can look at it soon because the all-hands is coming up fast!

[jgfouca]

@mgduda @matthewhoffman @akturner @jonbob please review as soon as you can.

[matthewhoffman]

I'm also approving by visual inspection. The landice core-specific stuff has been moved to the landice dir, so that's good.

[jonbob]

Approve by visual inspection

[mark-petersen]

@mgduda could you approve, or tell us of any outstanding concerns? We are working towards a deadline on this one.

[mgduda]

The changes in this PR have no impact on builds of MPAS-Atmosphere cores using the stand-alone build system. However, as discussed on the 4 November telecon, we will need to update the CMakeLists.txt files to accommodate stand-alone builds of MPAS-Atmosphere cores in future.

[jgfouca]

@mark-petersen , many thanks for pushing this along. Is there another PR I can follow for the updating of the MPAS submodule in E3SM?

[mark-petersen]

I can make a single E3SM PR right now with the mpas-source update, or we can update mpas-source in E3SM-Project/E3SM#3043. What is your preference?

[mark-petersen]

If the later option takes longer, the first might be faster and more direct.

[jgfouca]

@mark-petersen , I think the best option is to update via a fresh E3SM PR. I'm still not entirely certain how quickly 3043 will be able to make it into master, and based on git history, MPAS submodule is updated frequently, so updating it only in 3043 could be a source of conflicts.