Add a new benchmark

[olejandro]

No description provided.

[olejandro]

@siddharth-krishna is the CI failing related to changes in #293 ?

[siddharth-krishna]

No, it looks like the tool errored on the new benchmark (the error is hidden somewhere in the middle of the log because we run benchmarks in parallel):

Traceback (most recent call last):
  File "/home/runner/work/xl2times/xl2times/xl2times/.venv/bin/xl2times", line 8, in <module>
    sys.exit(main())
             ^^^^^^
  File "/home/runner/work/xl2times/xl2times/xl2times/xl2times/__main__.py", line 606, in main
    run(args)
  File "/home/runner/work/xl2times/xl2times/xl2times/xl2times/__main__.py", line 520, in run
    tables = convert_xl_to_times(
             ^^^^^^^^^^^^^^^^^^^^
  File "/home/runner/work/xl2times/xl2times/xl2times/xl2times/__main__.py", line 171, in convert_xl_to_times
    output = transform(config, input, model)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/runner/work/xl2times/xl2times/xl2times/xl2times/transforms.py", line 1592, in fill_in_missing_pcgs
    df["primarycg"] = df.apply(expand_pcg_from_suffix, axis=1)
                      ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/runner/work/xl2times/xl2times/xl2times/.venv/lib/python3.11/site-packages/pandas/core/frame.py", line 10374, in apply
    return op.apply().__finalize__(self, method="apply")
           ^^^^^^^^^^
  File "/home/runner/work/xl2times/xl2times/xl2times/.venv/lib/python3.11/site-packages/pandas/core/apply.py", line 916, in apply
    return self.apply_standard()
           ^^^^^^^^^^^^^^^^^^^^^
  File "/home/runner/work/xl2times/xl2times/xl2times/.venv/lib/python3.11/site-packages/pandas/core/apply.py", line 1063, in apply_standard
    results, res_index = self.apply_series_generator()
                         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/runner/work/xl2times/xl2times/xl2times/.venv/lib/python3.11/site-packages/pandas/core/apply.py", line 1081, in apply_series_generator
    results[i] = self.func(v, *self.args, **self.kwargs)
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/runner/work/xl2times/xl2times/xl2times/xl2times/transforms.py", line 1581, in expand_pcg_from_suffix
    return df["process"] + "_" + df["primarycg"]
           ~~~~~~~~~~~~~~^~~~~
TypeError: unsupported operand type(s) for +: 'NoneType' and 'str'
 (__main__:MainThread:pid-5927 "/home/runner/work/xl2times/xl2times/xl2times/utils/run_benchmarks.py:251")
2025-01-20 18:02:14.323 | ERROR    : tool failed on DemoS_special-t1 (__main__:MainThread:pid-5927 "/home/runner/work/xl2times/xl2times/xl2times/utils/run_benchmarks.py:252")

[olejandro]

Thanks for finding it!

[github-actions]

Regression test results on commit 15244fd

       Benchmark    Time (s)                 GDX Diff     Accuracy       Correct    Additional
----------------  ----------  -----------------------  -----------  ------------  ------------
   DemoS_001-all   3.2   3.9            OK         OK  100.0 100.0    118    118      3      3
   DemoS_002-all   3.7   4.5            OK         OK  100.0 100.0    344    344      3      3
   DemoS_003-all   4.1   4.2            OK         OK  100.0 100.0    633    633      6      6
       DemoS_004   4.3   5.5            OK         OK  100.0 100.0    662    662     12     12
      DemoS_004a   4.4   5.4            OK         OK  100.0 100.0    665    665     12     12
      DemoS_004b   4.3   5.1            OK         OK  100.0 100.0    665    665     12     12
   DemoS_004-all   4.6   5.3            OK         OK  100.0 100.0    667    667     12     12
   DemoS_005-all   4.9   5.4            13         13   99.7  99.7   1160   1160     12     12
   DemoS_006-all   6.1   6.3            13         13   99.7  99.7   1258   1258     12     12
   DemoS_007-all   7.1   7.4            13         13   99.8  99.8   2155   2155     12     12
DemoS_007-all-1r   5.5   5.9            11         11   99.8  99.8   1179   1179     12     12
   DemoS_008-all  10.2  10.0            13         13   99.9  99.9   5333   5333     18     18
   DemoS_009-all  11.0  11.1            26         26   99.9  99.9   5807   5807     29     29
   DemoS_010-all  11.0  11.0           822        822   88.4  88.4   6139   6139     29     29
   DemoS_011-all  11.2  11.2           854        854   88.0  88.0   6150   6150     29     29
   DemoS_012-all  11.6  11.9           901        901   88.2  88.2   6306   6306     53     53
DemoS_special-t1   5.6   5.5  Error runn… Error runn…   92.5  92.5   2110   2110     48     48
    TIMES-IE-all  33.9  33.9          3211       3211   97.5  97.5  42384  42384   2826   2826
    TIMES-IE-NoM  30.4  30.0           363        363   99.3  99.3  40528  40528    632    632
    TIMES-IE-MCB  30.9  31.0          1186       1186   97.6  97.6  41991  41991    632    632
    TIMES-NZ-KEA  59.4  60.3          4004       4004   98.2  98.2  76362  76362    472    472
    TIMES-NZ-TUI  47.6  47.9          4038       4038   98.2  98.2  75988  75988    488    488
 (__main__:MainThread:pid-5894 "/home/runner/work/xl2times/xl2times/xl2times/utils/run_benchmarks.py:373")
2025-01-23 16:48:29.114 | INFO     : Total runtime: 322.59s (main: 314.98s) (__main__:MainThread:pid-5894 "/home/runner/work/xl2times/xl2times/xl2times/utils/run_benchmarks.py:397")
2025-01-23 16:48:29.114 | INFO     : Change in runtime (negative == faster): +7.61s (+2.4%) (__main__:MainThread:pid-5894 "/home/runner/work/xl2times/xl2times/xl2times/utils/run_benchmarks.py:398")
2025-01-23 16:48:29.114 | INFO     : Change in correct rows (higher == better): +0 (+0.0%) (__main__:MainThread:pid-5894 "/home/runner/work/xl2times/xl2times/xl2times/utils/run_benchmarks.py:405")
2025-01-23 16:48:29.114 | INFO     : Change in additional rows: +0 (+0.0%) (__main__:MainThread:pid-5894 "/home/runner/work/xl2times/xl2times/xl2times/utils/run_benchmarks.py:412")
2025-01-23 16:48:29.114 | SUCCESS  : No regressions. You're awesome! (__main__:MainThread:pid-5894 "/home/runner/work/xl2times/xl2times/xl2times/utils/run_benchmarks.py:426")

This comment will be updated when new commits are added to the PR.

[Antti-L]

I see you are getting GAMS errors from the new case. But there shouldn't be any. 😃

[olejandro]

True! Seems to be element redefined and domain violation errors. The dd files are taken from here: https://github.com/olejandro/demos-dd/tree/add-a-test. I have not tried running them locally yet...