MusicBox does not run the TS1 case correctly

[carl-drews]

There seems to be a problem with the ts1 test case:

modeling2:/home/drews/MusicBox/music-box/src/acom_music_box> cat goMusicBox
#python3 main.py -vv --config examples/configs/analytical/my_config.json --output examples/configs/analytical/mySolution.csv
#music_box -vv --config examples/configs/analytical/my_config.json --output examples/configs/analytical/mySolution.csv
#music_box -vv --config examples/configs/flow_tube/my_config.json --output examples/configs/flow_tube/mySolution.csv
#music_box -vv --config examples/configs/carbon_bond_5/my_config.json --output examples/configs/carbon_bond_5/mySolution.csv
music_box -vv --config examples/configs/ts1/my_config.json --output examples/configs/ts1/mySolution.csv

modeling2:/home/drews/MusicBox/music-box/src/acom_music_box> ./goMusicBox
2024-12-12 15:09:17 - DEBUG - main.main - /home/drews/MusicBox/music-box/src/acom_music_box/main.py
2024-12-12 15:09:17 - INFO - main.main - Start time: 2024-12-12 15:09:17.619041
2024-12-12 15:09:17 - DEBUG - main.main - Working directory = /home/drews/MusicBox/music-box/src/acom_music_box
2024-12-12 15:09:17 - DEBUG - main.main - Configuration file = examples/configs/ts1/my_config.json
2024-12-12 15:09:17 - ERROR - conditions.read_initial_rates_from_file - Unexpected format in key: temperature [K]
Traceback (most recent call last):
File "/home/drews/.conda/envs/musicbox/bin/music_box", line 8, in
sys.exit(main())
^^^^^^
File "/home/drews/MusicBox/music-box/src/acom_music_box/main.py", line 127, in main
myBox.loadJson(musicBoxConfigFile)
File "/home/drews/MusicBox/music-box/src/acom_music_box/music_box.py", line 224, in loadJson
self.initial_conditions = Conditions.from_config_JSON(path_to_json, data)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/drews/MusicBox/music-box/src/acom_music_box/conditions.py", line 146, in from_config_JSON
reaction_rates = Conditions.read_initial_rates_from_file(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/drews/MusicBox/music-box/src/acom_music_box/conditions.py", line 195, in read_initial_rates_from_file
raise ValueError(error)
ValueError: Unexpected format in key: temperature [K]
modeling2:/home/drews/MusicBox/music-box/src/acom_music_box>

Here is the CSV file
modeling2:/home/drews/MusicBox/music-box/src/acom_music_box/examples/configs/ts1> vim initial_conditions.csv

The header row looks like this:
temperature [K],pressure [Pa],ALKNIT [mol m-3],BCARY [mol m-3],BENZENE [mol m-3],BIGALD [mol m-3],BIGALD1 [mol m-3],...
but the python code is expecting something more like this:
time.s,ENV.pressure.Pa,ENV.temperature.K,ENV.air_density.kg,USER.usr_DMS_OH,USER.usr_CO_OH,SURF.usr_NC4CH2OH_aer,SURF.usr_NTERPOOH_aer...

[carl-drews]

TS1 used to run correctly, so perhaps the correct file got inadvertently swapped out at some point.

[carl-drews]

Running the MusicBox tests modifies these two most recent files:

modeling2:/home/drews/MusicBox/music-box-220-config-file-csv/src/acom_music_box/examples/configs/ts1> ls -ltr
total 284
drwxr-xr-x 2 drews support     83 Dec  9 12:09 camp_data/
-rw-r--r-- 1 drews support   3037 Dec 12 09:41 initial_conditions_badFormat.csv
-rw-r--r-- 1 drews support  10053 Dec 12 16:03 my_config.json
-rw-r--r-- 1 drews support 258614 Dec 12 16:09 mySolution.csv
-rw-r--r-- 1 drews support   3037 Dec 12 17:40 initial_conditions.csv
-rw-r--r-- 1 drews support   6187 Dec 12 17:40 initial_config.json
modeling2:/home/drews/MusicBox/music-box-220-config-file-csv/src/acom_music_box/examples/configs/ts1>

That is wrong. Those 2 files are both under git version control.

[K20shores]

@carl-drews #280 brought in code coverage for us. After that came in I made a series of commits which increased our test coverage. I also included a test for the waccm conversion tool . If running the tests changes a file, it may be that the tool updates files in the current directory. Perhaps that's what's going on?

[carl-drews]

I'm also getting an error on this test:

modeling2:/home/drews/MusicBox/music-box/tests/integration> python3 test_mixing_ratios.py 
Traceback (most recent call last):
  File "/home/drews/MusicBox/music-box/tests/integration/test_mixing_ratios.py", line 114, in <module>
    test = TestAnalytical()
           ^^^^^^^^^^^^^^
NameError: name 'TestAnalytical' is not defined
modeling2:/home/drews/MusicBox/music-box/tests/integration>

[K20shores]

I'm also getting an error on this test:

modeling2:/home/drews/MusicBox/music-box/tests/integration> python3 test_mixing_ratios.py 
Traceback (most recent call last):
  File "/home/drews/MusicBox/music-box/tests/integration/test_mixing_ratios.py", line 114, in <module>
    test = TestAnalytical()
           ^^^^^^^^^^^^^^
NameError: name 'TestAnalytical' is not defined
modeling2:/home/drews/MusicBox/music-box/tests/integration>

Ah, to run the tests I just run pytest which knows to call classes directly. That class has been renamed and rewritten. If you want to run it individually, just run pytest test_mixing_ratios.py in the integrations directly.

[carl-drews]

Yes, the WACCM conversion test is the one that changes files:
tests/integration/test_waccm_conversion.py::test_waccm_to_music_box_conversion

I will dig into that one . . .

[carl-drews]

The change in CSV header format looks like this:
The checked-in file has the correct header format for MusicBox:
time.s,ENV.pressure.Pa,ENV.temperature.K,ENV.air_density.kg,USER.usr_DMS_OH,USER.usr_CO_OH,SURF.usr_NC4CH2OH_aer,SURF.usr_NTERPOOH_aer,SURF.usr_TERPNIT_aer,SURF.usr_NC4CHO_aer,SURF.usr_HONITR_aer,SURF.usr_N2O5_aer,SURF.usr_ONITR_aer,SURF.usr_GLYOXAL_aer,SURF.usr_ISOPNITB_aer,SURF.usr_ISOPNITA_aer,SURF.usr_NO3_aer,SURF.usr_HO2_aer,SURF.usr_NO2_aer,USER.het17,USER.het16,USER.het15,USER.het14,USER.het13,USER.het12,USER.het11,USER.het10,USER.het9,USER.het8,…

The file generated by the WACCM test script has the incorrect format:
temperature [K],pressure [Pa],ALKNIT [mol m-3],BCARY [mol m-3],BENZENE [mol m-3],BIGALD [mol m-3],BIGALD1 [mol m-3],BIGALD2 [mol m-3],BIGALD3 [mol m-3],BIGALD4 [mol m-3],BIGALK [mol m-3],BIGENE [mol m-3],C2H2 [mol m-3],C2H4 [mol m-3],C2H5OH [mol m-3],C2H6 [mol m-3],C3H6 [mol m-3],C3H8 [mol m-3],CH2O [mol m-3],CH3CHO [mol m-3],CH3CN [mol m-3],...

This difference does not look difficult to correct. But I will correct the overwriting first.

[carl-drews]

I have created a config directory for the WACCM example, and placed the test output there:

modeling2:/home/drews/MusicBox/music-box-220-config-file-csv/src/acom_music_box/examples/configs/waccm> ls -ltr
total 12
-rw-r--r-- 1 drews support 3037 Dec 19 14:48 initial_conditions.csv
-rw-r--r-- 1 drews support 6187 Dec 19 14:48 initial_config.json

[carl-drews]

Corrected WACCM output so that headers have the reaction_type prefix:
ENV.temperature [K],ENV.pressure [Pa],CONC.ALKNIT [mol m-3]

[carl-drews]

This issue is fixed as part of this branch: https://github.com/NCAR/music-box/tree/220-config-file-csv