Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Miscellaneous enhancements for 2023-W44 #137

Merged
merged 64 commits into from
Nov 23, 2023
Merged

Miscellaneous enhancements for 2023-W44 #137

merged 64 commits into from
Nov 23, 2023

Conversation

khaeru
Copy link
Member

@khaeru khaeru commented Oct 31, 2023

  • Improve ExoDataSource:
    • "name" attribute used for keys.
    • "extra_dims" attribute if returned data will have more dimensions than (n, y).
    • transform() method that can be overloaded by subclasses.
  • Improve iamc_like_data_for_query():
    • Allow selecting a specific archive_member.
    • Configure handling of region codes that are not ISO 3166 country names.
  • Add .project.advance, an exogenous data source for the ADVANCE project database, fuzzed data for testing, tests. Supersedes .tools.advance.
  • Improve and combine mix-models ssp make-test-data CLI command and .tools.advance._fuzz_data() to mix-models testing fuzz-private-data, usable for SSP, ADVANCE, and other data sources.
  • Add .tools.iamc.describe() to generate code lists given data in the IAMC structure.
  • Add .workflow.make_click_command() to automatically generate a CLI for any Workflow.
  • Add fetch CLI command; supersedes message_data.tools.download and the narrower message_ix_models.snapshot CLI.
  • Remove message_ix_models/data/water/ files from packaging; this shrinks the package from >20 MB to <5 MB.
    • Provide an archive of these files + code + CLI to allow downloading them.
  • Rename report.computationsreport.operator (cf. Change term ‘computations’? khaeru/genno#34, Add an Operator class khaeru/genno#98, Rename 'report', 'util' + miscellaneous clean-ups ixmp#500).
  • Improve test coverage in .report.

How to review

Read the diff and note that the CI checks all pass.

PR checklist

  • Continuous integration checks all ✅
  • Add or expand tests; coverage checks both ✅
  • Add, expand, or update documentation.
  • Update doc/whatsnew.

@khaeru khaeru added the enh New features or functionality label Oct 31, 2023
@khaeru khaeru self-assigned this Oct 31, 2023
--nodes is now stored on Context automatically.
Copy link

codecov bot commented Oct 31, 2023

Codecov Report

Merging #137 (5b52a7b) into main (294ac47) will increase coverage by 0.9%.
The diff coverage is 92.6%.

Additional details and impacted files
@@           Coverage Diff           @@
##            main    #137     +/-   ##
=======================================
+ Coverage   73.3%   74.3%   +0.9%     
=======================================
  Files         82      89      +7     
  Lines       5663    5942    +279     
=======================================
+ Hits        4155    4416    +261     
- Misses      1508    1526     +18     
Files Coverage Δ
message_ix_models/cli.py 100.0% <ø> (ø)
message_ix_models/model/bare.py 100.0% <100.0%> (ø)
message_ix_models/model/build.py 100.0% <100.0%> (ø)
message_ix_models/model/cli.py 100.0% <100.0%> (ø)
message_ix_models/model/emissions.py 98.0% <100.0%> (ø)
message_ix_models/model/macro.py 100.0% <100.0%> (ø)
message_ix_models/model/snapshot.py 100.0% <100.0%> (ø)
message_ix_models/model/structure.py 100.0% <100.0%> (ø)
message_ix_models/model/water/reporting.py 4.8% <100.0%> (ø)
message_ix_models/project/advance/data.py 100.0% <100.0%> (ø)
... and 31 more

This avoids triggering import side-effects until the user actually runs
the command.

Also expand docstring.
Use function-scoped test_context to avoid polluting session_context.
- Use existing test tools instead of monkeypatch.
- Share a pytest mark with test_workflow().
- Remove unused "_base_url" key.
- Scope to individual test function.
- Adjust usage in test_workflow(): the "base" key now has a platform name, so
  can be truncated.
khaeru added a commit that referenced this pull request Nov 4, 2023
khaeru added a commit that referenced this pull request Nov 4, 2023
@khaeru khaeru added water MESSAGEix-Nexus (water) variant structure Infomation and metadata about model structure report genno-based reporting and post-solve processing labels Nov 14, 2023
@khaeru khaeru marked this pull request as ready for review November 14, 2023 13:53
@khaeru khaeru force-pushed the enh/2023-W44 branch 5 times, most recently from 6f9d1b3 to 4c59e0d Compare November 22, 2023 23:58
khaeru added a commit that referenced this pull request Nov 23, 2023
Use forms that are robust to 'reporting' → 'report' rename.
- Import RENAME_DIMS through a function that provides backwards
compatibility.
- Use MESSAGE.items and MACRO.items; remove redefinitions.
- Simplify simulate_qty() to accept a list of dimensions directly.
Replace manual version checks and marks with this utility.
This remains disabled, as it cannot handle wrapper_descriptor.
Enable nitpicky, napoleon_preprocess_types
@khaeru khaeru merged commit fb5a380 into main Nov 23, 2023
19 checks passed
khaeru added a commit that referenced this pull request Apr 2, 2024
@glatterf42 glatterf42 deleted the enh/2023-W44 branch August 14, 2024 06:45
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enh New features or functionality report genno-based reporting and post-solve processing structure Infomation and metadata about model structure water MESSAGEix-Nexus (water) variant
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants