loo results not reproducible between calls (loo$pointwise$mcse_elpd_loo field)

[mcol]

Two calls to loo() on the same object will not produce identical results, as mcse_elpd() calls rnorm() without setting a seed. Perhaps this is by design, but if so, it's not documented. This affects only the mcse_elpd_loo column of loo$pointwise.

[jgabry]

Yeah this is true. @avehtari I remember we added the code to compute mcse_elpd_loo together one time you were in NYC, but I forget if we put in this particular behavior by design.

[avehtari]

Quasi Monte Carlo or quadrature would be a better choice. Sorry this has been hanging.

[jgabry]

let’s add this to the list of things to sort out when you visit in a few weeks.