Non-intuitive need to "refresh dataframe" after operating on .data

[Rlamboll]

As mentioned in #396, when operating directly on the .data object, derivative objects are not updated. This is resolved by "refreshing" the dataframe via DataFrame(df.data), like so:

import pyam
import pandas as pd

Make arbitrary dataframe

_mc = "model_c"
_sa = "scen_a"
_sb = "scen_b"
_eco2 = "Emissions|CO2"
_gtc = "Gt C/yr"
_ech4 = "Emissions|CH4"
_mtch4 = "Mt CH4/yr"
_msrvu = ["model", "scenario", "region", "variable", "unit"]
simple_df = pd.DataFrame(
[
[_mc, _sa, "World", _eco2, _gtc, 0, 1000, 5000],
[_mc, _sb, "World", _eco2, _gtc, 1, 1000, 5000],
[_mc, _sa, "World", _ech4, _mtch4, 0, 300, 500],
[_mc, _sb, "World", _ech4, _mtch4, 1, 300, 500],
],
columns=_msrvu + [2010, 2030, 2050],
)
simple_df = pyam.IamDataFrame(simple_df)

Operate on data

simple_df.data["scenario", 0] = "not a"

Show that scenarios hasn't updated until the df is refreshed:

print(simple_df.scenarios())
remade_df = pyam.IamDataFrame(simple_df.data)
print(remade_df.scenarios())

Printed lines 1 and 2 are not equal.

It's not clear that this is a bug to me, and it might be annoying/slowing to ensure that all calls to scenarios etc. check for consistency before outputting the values.

[danielhuppmann]

Sorry @Rlamboll, this is a user error rather than a bug - you aren't supposed to work on data in this way - you are making it inconsistent with the meta dataframe (from which scenarios() is derived).

I guess we should refactor this to _data (same with meta) to make that more obvious...

[Rlamboll]

It's not an error, as stated it works perfectly and is mostly an explanation of why I wanted the "refresh" function described in #396. Please don't refactor data, that will break everything in Silicone.