Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Multi-ci-levels don't match rows of "grid" like inputs #688

Closed
qdread opened this issue Dec 17, 2024 · 2 comments
Closed

Multi-ci-levels don't match rows of "grid" like inputs #688

qdread opened this issue Dec 17, 2024 · 2 comments
Labels
Bug 🐛 Something isn't working

Comments

@qdread
Copy link

qdread commented Dec 17, 2024

I noticed that after updating from v0.14.0 to v0.15.0, the behavior when printing an object of class describe_posterior created by calling describe_posterior() on an object of class emmGrid is incorrect, if I specify more than one CI width. There was clearly a breaking change having to do with the reshaping that is done to print the output. An incorrect median is printed for some of the rows, and some of the credible intervals are not printed. Warnings from reshapeWide are printed. I have kept R version and all other package versions constant, and only changed the bayestestR version, to produce the following output.

Thanks for the excellent package. Please let me know if I can help fix the bug!

Code to run

library(emmeans)
library(bayestestR)
library(brms)

mod <- brm(sales1 ~ factor(store), data = oranges, seed = 1)
emm <- emmeans(mod, ~ store)

describe_posterior(emm, centrality = 'median', ci = c(0.90, 0.95))

Output using bayestestR 0.14.0

This is the expected behavior.

Summary of Posterior Distribution

Parameter | Median |         90% CI |        95% CI |     pd |          ROPE | % in ROPE
----------------------------------------------------------------------------------------
1         |  12.75 | [ 8.35, 17.26] | [7.41, 18.08] |   100% | [-0.10, 0.10] |        0%
2         |   8.30 | [ 3.78, 12.83] | [2.84, 13.89] | 99.83% | [-0.10, 0.10] |        0%
3         |   6.59 | [ 2.06, 11.15] | [1.13, 11.93] | 98.95% | [-0.10, 0.10] |        0%
4         |   8.30 | [ 3.85, 12.57] | [2.86, 13.45] | 99.78% | [-0.10, 0.10] |        0%
5         |   9.76 | [ 5.30, 14.39] | [4.52, 15.34] |   100% | [-0.10, 0.10] |        0%
6         |  15.25 | [10.90, 19.57] | [9.96, 20.41] |   100% | [-0.10, 0.10] |        0%

Output using bayestestR 0.15.0

This is not the correct output. As you can see we get duplicated medians, incorrect CIs, and some blank CIs. The output is followed by warnings from reshapeWide.

Summary of Posterior Distribution

store | Median |        90% CI |        95% CI |     pd |          ROPE | % in ROPE
-----------------------------------------------------------------------------------
1     |  12.75 | [8.35, 17.26] |               |   100% | [-0.10, 0.10] |        0%
2     |  12.75 |               | [7.41, 18.08] |   100% | [-0.10, 0.10] |        0%
3     |   8.30 | [3.78, 12.83] |               | 99.83% | [-0.10, 0.10] |        0%
4     |   8.30 |               | [2.84, 13.89] | 99.83% | [-0.10, 0.10] |        0%
5     |   6.59 | [2.06, 11.15] |               | 98.95% | [-0.10, 0.10] |        0%
6     |   6.59 |               | [1.13, 11.93] | 98.95% | [-0.10, 0.10] |        0%
Warning messages:
1: In reshapeWide(data, idvar = idvar, timevar = timevar, varying = varying,  :
  some constant variables (Median,pd) are really varying
2: In reshapeWide(data, idvar = idvar, timevar = timevar, varying = varying,  :
  multiple rows match for CI=0.9: first taken
3: In reshapeWide(data, idvar = idvar, timevar = timevar, varying = varying,  :
  multiple rows match for CI=0.95: first taken
@strengejacke
Copy link
Member

@mattansb ?

@strengejacke strengejacke added the Bug 🐛 Something isn't working label Dec 17, 2024
@mattansb
Copy link
Member

Good catch!

Should be fixed now

library(bayestestR)
library(brms)


data("oranges")
oranges$store <- factor(oranges$store)
mod <- brm(sales1 ~ store, data = oranges, seed = 1)




# emmeans
emm <- emmeans::emmeans(mod, ~ store)

describe_posterior(emm, centrality = 'median', ci = c(0.90, 0.95))
#> Summary of Posterior Distribution
#> 
#> store | Median |         90% CI |        95% CI |     pd |          ROPE | % in ROPE
#> ------------------------------------------------------------------------------------
#> 1     |  12.75 | [ 8.35, 17.26] | [7.41, 18.08] |   100% | [-0.10, 0.10] |        0%
#> 2     |   8.30 | [ 3.78, 12.83] | [2.84, 13.89] | 99.83% | [-0.10, 0.10] |        0%
#> 3     |   6.59 | [ 2.06, 11.15] | [1.13, 11.93] | 98.95% | [-0.10, 0.10] |        0%
#> 4     |   8.30 | [ 3.85, 12.57] | [2.86, 13.45] | 99.78% | [-0.10, 0.10] |        0%
#> 5     |   9.76 | [ 5.30, 14.39] | [4.52, 15.34] |   100% | [-0.10, 0.10] |        0%
#> 6     |  15.25 | [10.90, 19.57] | [9.96, 20.41] |   100% | [-0.10, 0.10] |        0%



pr <- marginaleffects::avg_predictions(mod, variables = "store")

describe_posterior(pr, centrality = 'median', ci = c(0.90, 0.95))
#> Summary of Posterior Distribution
#> 
#> store | Median |         90% CI |        95% CI |     pd |          ROPE | % in ROPE
#> ------------------------------------------------------------------------------------
#> 1     |  12.75 | [ 8.35, 17.26] | [7.41, 18.08] |   100% | [-0.10, 0.10] |        0%
#> 2     |   8.30 | [ 3.78, 12.83] | [2.84, 13.89] | 99.83% | [-0.10, 0.10] |        0%
#> 3     |   6.59 | [ 2.06, 11.15] | [1.13, 11.93] | 98.95% | [-0.10, 0.10] |        0%
#> 4     |   8.30 | [ 3.85, 12.57] | [2.86, 13.45] | 99.78% | [-0.10, 0.10] |        0%
#> 5     |   9.76 | [ 5.30, 14.39] | [4.52, 15.34] |   100% | [-0.10, 0.10] |        0%
#> 6     |  15.25 | [10.90, 19.57] | [9.96, 20.41] |   100% | [-0.10, 0.10] |        0%




# data frame with rvar
grid <- data.frame(store = 1:6)
grid$pred <- posterior_epred(mod, newdata = grid) |> posterior::rvar()

describe_posterior(grid, rvar_col = "pred", centrality = 'median', ci = c(0.90, 0.95))
#> Summary of Posterior Distribution
#> 
#> store | Median |         90% CI |        95% CI |     pd |          ROPE | % in ROPE
#> ------------------------------------------------------------------------------------
#> 1.00  |  12.75 | [ 8.35, 17.26] | [7.41, 18.08] |   100% | [-0.10, 0.10] |        0%
#> 2.00  |   8.30 | [ 3.78, 12.83] | [2.84, 13.89] | 99.83% | [-0.10, 0.10] |        0%
#> 3.00  |   6.59 | [ 2.06, 11.15] | [1.13, 11.93] | 98.95% | [-0.10, 0.10] |        0%
#> 4.00  |   8.30 | [ 3.85, 12.57] | [2.86, 13.45] | 99.78% | [-0.10, 0.10] |        0%
#> 5.00  |   9.76 | [ 5.30, 14.39] | [4.52, 15.34] |   100% | [-0.10, 0.10] |        0%
#> 6.00  |  15.25 | [10.90, 19.57] | [9.96, 20.41] |   100% | [-0.10, 0.10] |        0%

@mattansb mattansb changed the title print method for describe_posterior of emmeans with more than one CI produces incorrect output in v0.15.0 Multi-ci-levels don't match rows of "grid" like inputs Dec 17, 2024
mattansb added a commit that referenced this issue Dec 20, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Bug 🐛 Something isn't working
Projects
None yet
Development

No branches or pull requests

3 participants