Move PIO type control to individual streams for MPAS-O and MPAS-SEAICE #1456

jonbob · 2017-04-25T21:52:35Z

This PR moves control of PIO_TYPENAME to individual files by including settings for "io_type" for each file in the mpas-o and mpas-seaice streams. For high-res grids, where some 3D fields require special handling, specific files are given the following setting:
io_type="pnetcdf,cdf5"
Otherwise, all streams are assigned the default PIO_TYPENAME for the machine being run on. The drivers are also modified to remove the setting of a default io_type, which would overwrite the specific stream settings.

Tested for correct streams output via:
-compset GMPAS-IAF -mach edison -compiler intel -res T62_oEC60to30v3
-compset GMPAS-IAF -mach edison -compiler intel -res T62_oRRS18to6v3

Need throughput testing at high-res!

[BFB]

mark-petersen

This looks like just what we need. I inspected visually, did not test it myself. But the lines changed are the same as what I used to test the io_type="pnetcdf,cdf5" on edison by directly changing the streams file. Thanks!

vanroekel

@jonbob Thanks so much for making this automated! I had just a few needed changes. I might just suggest making all timeSeriesStats cdf5 in case someone puts normal velocity in there. It is in timeSeriesStatsMonthly by default, so the io_type should be changed for that one at least.

vanroekel · 2017-04-26T13:53:30Z

components/mpas-o/cime_config/buildnml

@@ -379,6 +397,7 @@ if ( -e "$CASEROOT/SourceMods/src.mpaso/$STREAM_NAME" ) {
 	print $stream_file '' . "\n";
 	print $stream_file '<stream name="globalStatsOutput"' . "\n";
 	print $stream_file '        type="output"' . "\n";
+	print $stream_file '        io_type="' . "$OCN_PIO_TYPENAME" . '"' . "\n";


I think we could use io_type="netcdf" here too

testing on titan suggests netcdf here is much faster (1.4 max seconds per process with pnetcdf vs. .06 max seconds with netcdf)

@vanroekel - our idea has been to overwrite the io_type for streams we know need cdf5, but otherwise fall back to the machine defaults. So I'll make the changes to the streams you've specified, and check that titan's default is netcdf. If it's not, we'll have to ask the titan POC to make that change... Thanks for the careful review!

@vanroekel - but you're also correct that we discussed using "netcdf" for streams we know are small and will be negatively impacted by possible using pnetcdf. @mark-petersen , what do you think about this one?

wolf was really slow with pnetcdf, so we set all defaults to netcdf. Are there any examples of the opposite - machines that are very slow for netcdf for small files? I've never heard that, so it seems like it would be safe to set the default to netcdf for all small files.

@mark-petersen - Agreed. I think this is a safer approach and more likely to be stable over the range of machines. I'll consider this PR ready to go as it currently is then...

vanroekel · 2017-04-26T13:53:47Z

components/mpas-o/cime_config/buildnml

@@ -398,6 +417,7 @@ if ( -e "$CASEROOT/SourceMods/src.mpaso/$STREAM_NAME" ) {
 	print $stream_file '' . "\n";
 	print $stream_file '<stream name="surfaceAreaWeightedAveragesOutput"' . "\n";
 	print $stream_file '        type="output"' . "\n";
+	print $stream_file '        io_type="' . "$OCN_PIO_TYPENAME" . '"' . "\n";


same as above.

vanroekel · 2017-04-26T13:54:23Z

components/mpas-o/cime_config/buildnml

@@ -655,6 +688,7 @@ if ( -e "$CASEROOT/SourceMods/src.mpaso/$STREAM_NAME" ) {
        print $stream_file '' . "\n";
        print $stream_file '<stream name="timeSeriesStatsMonthlyOutput"' . "\n";
        print $stream_file '        type="output"' . "\n";
+	print $stream_file '        io_type="' . "$OCN_PIO_TYPENAME" . '"' . "\n";


This definitely needs cdf5, due to normal velocity being in here.

I agree that in this PR any stream with normalVelocity should be cdf5 for high rez. Thanks @vanroekel for thinking of that.

On Thursday I'll propose making a separate monthly time-averaged stream for edge variables, but that is yet to be agreed upon and tested.

and this one

vanroekel · 2017-04-26T13:54:50Z

components/mpas-o/cime_config/buildnml

@@ -743,6 +780,7 @@ if ( -e "$CASEROOT/SourceMods/src.mpaso/$STREAM_NAME" ) {
        print $stream_file '' . "\n";
        print $stream_file '<stream name="timeSeriesStatsMonthlyRestart"' . "\n";
        print $stream_file '        type="input;output"' . "\n";
+	print $stream_file '        io_type="' . "$OCN_PIO_TYPENAME" . '"' . "\n";


This definitely also needs cdf5. Normal velocity is in here, which violates constraints.

and one more...

vanroekel · 2017-04-26T13:55:51Z

components/mpas-o/cime_config/buildnml

@@ -558,6 +587,7 @@ if ( -e "$CASEROOT/SourceMods/src.mpaso/$STREAM_NAME" ) {
 	print $stream_file '' . "\n";
 	print $stream_file '<stream name="eliassenPalmOutput"' . "\n";
 	print $stream_file '        type="output"' . "\n";
+	print $stream_file '        io_type="' . "$OCN_PIO_TYPENAME" . '"' . "\n";


I think this will need cdf5 too, it is a very heavy stream.

OK, changed this one

vanroekel · 2017-04-26T15:43:41Z

Thanks @jonbob, looks good to me!

mark-petersen · 2017-04-26T21:39:40Z

This final version looks good to me too.

vanroekel · 2017-05-05T14:37:02Z

components/mpas-o/cime_config/buildnml

@@ -279,6 +295,7 @@ if ( -e "$CASEROOT/SourceMods/src.mpaso/$STREAM_NAME" ) {
 	print $stream_file '' . "\n";
 	print $stream_file '<stream name="output"' . "\n";
 	print $stream_file '		type="output"' . "\n";
+	print $stream_file '		io_type="' . "$OCN_PIO_TYPENAME" . '"' . "\n";


@jonbob testing in the 18to6 spinup I note that this stream needs pnetcdf,cdf5 as well. normalVelocity is in this stream, which violate size constraints.

…stion

This PR moves control of PIO_TYPENAME to individual files by including settings for "io_type" for each file in the mpas-o and mpas-seaice streams. For high-res grids, where some 3D fields require special handling, specific files are given the following setting: io_type="pnetcdf,cdf5" Otherwise, all streams are assigned the default PIO_TYPENAME for the machine being run on. The drivers are also modified to remove the setting of a default io_type, which would overwrite the specific stream settings. Tested for correct streams output via: -compset GMPAS-IAF -mach edison -compiler intel -res T62_oEC60to30v3 -compset GMPAS-IAF -mach edison -compiler intel -res T62_oRRS18to6v3 [NML] [BFB]

jonbob · 2017-05-15T19:16:35Z

Merged to next

This PR moves control of PIO_TYPENAME to individual files by including settings for "io_type" for each file in the mpas-o and mpas-seaice streams. For high-res grids, where some 3D fields require special handling, specific files are given the following setting: io_type="pnetcdf,cdf5" Otherwise, all streams are assigned the default PIO_TYPENAME for the machine being run on. The drivers are also modified to remove the setting of a default io_type, which would overwrite the specific stream settings. Tested for correct streams output via: -compset GMPAS-IAF -mach edison -compiler intel -res T62_oEC60to30v3 -compset GMPAS-IAF -mach edison -compiler intel -res T62_oRRS18to6v3 [NML] [BFB]

jonbob · 2017-05-16T21:39:06Z

Merged to master

Move PIO type control to individual streams for MPAS-O and MPAS-SEAICE

95d4b18

jonbob added BFB PR leaves answers BFB mpas-ocean PIO mpas-seaice labels Apr 25, 2017

jonbob self-assigned this Apr 25, 2017

jonbob requested review from mark-petersen and vanroekel April 25, 2017 21:52

mark-petersen approved these changes Apr 26, 2017

View reviewed changes

vanroekel requested changes Apr 26, 2017

View reviewed changes

Adding changes suggested by Luke Van Roekel's review

7d7443e

vanroekel approved these changes Apr 26, 2017

View reviewed changes

vanroekel reviewed May 5, 2017

View reviewed changes

Add special handling at high-res to one more stream, per Luke's sugge…

51ea5bd

…stion

jonbob merged commit 51ea5bd into master May 16, 2017

jonbob deleted the jonbob/mpas/move-pio-control branch May 16, 2017 21:39

vanroekel mentioned this pull request May 31, 2017

Slow init time in OCN for hires problems (g18to6.T62_oRRS18to6v3.GMPAS-IAF) #1547

Closed

vanroekel mentioned this pull request Apr 4, 2018

MPAS-O PIO warnings during restart write #1138

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Move PIO type control to individual streams for MPAS-O and MPAS-SEAICE #1456

Move PIO type control to individual streams for MPAS-O and MPAS-SEAICE #1456

jonbob commented Apr 25, 2017 •

edited

Loading

mark-petersen left a comment

vanroekel left a comment

vanroekel Apr 26, 2017

vanroekel Apr 26, 2017

jonbob Apr 26, 2017

jonbob Apr 26, 2017

mark-petersen Apr 26, 2017

jonbob Apr 26, 2017

vanroekel Apr 26, 2017

vanroekel Apr 26, 2017

mark-petersen Apr 26, 2017

jonbob Apr 26, 2017

vanroekel Apr 26, 2017

jonbob Apr 26, 2017

vanroekel Apr 26, 2017

jonbob Apr 26, 2017

vanroekel commented Apr 26, 2017

mark-petersen commented Apr 26, 2017

vanroekel May 5, 2017

jonbob commented May 15, 2017

jonbob commented May 16, 2017

Move PIO type control to individual streams for MPAS-O and MPAS-SEAICE #1456

Move PIO type control to individual streams for MPAS-O and MPAS-SEAICE #1456

Conversation

jonbob commented Apr 25, 2017 • edited Loading

mark-petersen left a comment

Choose a reason for hiding this comment

vanroekel left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

vanroekel commented Apr 26, 2017

mark-petersen commented Apr 26, 2017

Choose a reason for hiding this comment

jonbob commented May 15, 2017

jonbob commented May 16, 2017

jonbob commented Apr 25, 2017 •

edited

Loading