You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
The goal here is to add simulation-based calibration model checking to the forecasttools toolbox. This then provides a set of checks for the self-consistency/correctness of a Bayesian model + a posterior sampling method. This is aimed as acting as a unit test for models.
Proposed solution
My proposal is to smooth any issue with using SBC to work with tidybayes (this is likely already available) objects that are structured over multiple sbc trials. In reality, most models we will try outputing an Arviz object so the idea will be to leverage Arviz to tidybayes conversions we already have in place.
At least as a first pass, the user will be responsible for generating the sbc trials.
As a diagram the expected eventual workflow is as follows:
graph TD
Generate_data[Generate datasets from model] -->|inference| posterior_draws[Posterior draws for each dataset]
posterior_draws -->|Arviz| make_data_set[Make InferenceData object over datasets]
make_data_set -->|Serialise| serialise[Serialize to netCDF]
make_data_set -->|direct convert| convert[Convert to tidybayes]
serialise -->|deserialise and convert| convert
convert -->|SBC| sbc[Run SBC checks and return report]
Loading
The text was updated successfully, but these errors were encountered:
Goal
The goal here is to add simulation-based calibration model checking to the
forecasttools
toolbox. This then provides a set of checks for the self-consistency/correctness of a Bayesian model + a posterior sampling method. This is aimed as acting as a unit test for models.Proposed solution
My proposal is to smooth any issue with using
SBC
to work withtidybayes
(this is likely already available) objects that are structured over multiple sbc trials. In reality, most models we will try outputing anArviz
object so the idea will be to leverageArviz
totidybayes
conversions we already have in place.At least as a first pass, the user will be responsible for generating the sbc trials.
As a diagram the expected eventual workflow is as follows:
The text was updated successfully, but these errors were encountered: