Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Error handling with stats and files with different dimensions. #19

Open
faeglas opened this issue Nov 9, 2023 · 0 comments
Open

Error handling with stats and files with different dimensions. #19

faeglas opened this issue Nov 9, 2023 · 0 comments

Comments

@faeglas
Copy link
Collaborator

faeglas commented Nov 9, 2023

Currently when using the stats command and a file_id with multiple files, the error thrown is this:

Traceback (most recent call last):
  File "./probtest.py", line 57, in <module>
    cli()
  File "/scratch/d1000/apps/buildbot/bb-venv/lib/python3.6/site-packages/click/core.py", line 1128, in __call__
    return self.main(*args, **kwargs)
  File "/scratch/d1000/apps/buildbot/bb-venv/lib/python3.6/site-packages/click/core.py", line 1053, in main
    rv = self.invoke(ctx)
  File "/scratch/d1000/apps/buildbot/bb-venv/lib/python3.6/site-packages/click/core.py", line 1659, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/scratch/d1000/apps/buildbot/bb-venv/lib/python3.6/site-packages/click/core.py", line 1395, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/scratch/d1000/apps/buildbot/bb-venv/lib/python3.6/site-packages/click/core.py", line 754, in invoke
    return __callback(*args, **kwargs)
  File "/scratch/mch/fgessler/ICON/GLORI_test_small/CPU/externals/probtest/engine/stats.py", line 91, in stats
    file_specification,
  File "/scratch/mch/fgessler/ICON/GLORI_test_small/CPU/externals/probtest/engine/stats.py", line 15, in create_stats_dataframe
    df = df_from_file_ids(file_id, input_dir, file_specification)
  File "/scratch/mch/fgessler/ICON/GLORI_test_small/CPU/externals/probtest/util/dataframe_ops.py", line 150, in df_from_file_ids
    fid_dfs = unify_time_index(fid_dfs)
  File "/scratch/mch/fgessler/ICON/GLORI_test_small/CPU/externals/probtest/util/dataframe_ops.py", line 174, in unify_time_index
    df = df.reindex(columns=unique_times, level=time_multiindex_index)
  File "/scratch/d1000/apps/buildbot/bb-venv/lib64/python3.6/site-packages/pandas/util/_decorators.py", line 309, in wrapper
    return func(*args, **kwargs)
  File "/scratch/d1000/apps/buildbot/bb-venv/lib64/python3.6/site-packages/pandas/core/frame.py", line 4036, in reindex
    return super().reindex(**kwargs)
  File "/scratch/d1000/apps/buildbot/bb-venv/lib64/python3.6/site-packages/pandas/core/generic.py", line 4464, in reindex
    axes, level, limit, tolerance, method, fill_value, copy
  File "/scratch/d1000/apps/buildbot/bb-venv/lib64/python3.6/site-packages/pandas/core/frame.py", line 3877, in _reindex_axes
    columns, method, copy, level, fill_value, limit, tolerance
  File "/scratch/d1000/apps/buildbot/bb-venv/lib64/python3.6/site-packages/pandas/core/frame.py", line 3925, in _reindex_columns
    allow_dups=False,
  File "/scratch/d1000/apps/buildbot/bb-venv/lib64/python3.6/site-packages/pandas/core/generic.py", line 4532, in _reindex_with_indexers
    copy=copy,
  File "/scratch/d1000/apps/buildbot/bb-venv/lib64/python3.6/site-packages/pandas/core/internals/managers.py", line 1285, in reindex_indexer
    self.axes[axis]._can_reindex(indexer)
  File "/scratch/d1000/apps/buildbot/bb-venv/lib64/python3.6/site-packages/pandas/core/indexes/base.py", line 3292, in _can_reindex
    raise ValueError("cannot reindex from a duplicate axis")
ValueError: cannot reindex from a duplicate axis

We could make a simple check to make sure all files a user is trying to use stats on have the same dimensions and give a more meaningful and lightweight error message. E.g: "Error: File a and File b have different dimensions. They must have the same dimensions."

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant