Repository navigation
Deal with big sample sizes and the slow down they produce in the page #3
Description
When particularly fast functions are benchmarked for long, the sample size gets too large and plotly.js basically crashes or takes VERY LONG to render.
Current work arounds:
- reduce
:timeto indirectly reduce sample size - remove the individual graphs from the resulting HTML by hand (they are most of the cause of this happening) and maybe also the box plot (is also fed raw run times)
Proposed solutions (might go into separate issues)
- print a warning when sample sizes exceed a certain threshold (like say 100_000)
- provide the option to split the individual graphs into separate files and then link to them
- render the box plot giving manually calculated quadrilles on the elixir side see this thread
Activity
PragTob commentedon May 7, 2017
Splitting of files for individual benchmarks is implemented in 0.2.0
Schultzer commentedon Sep 27, 2025
Wouldn't using a bar chart solve this, if there is a lot of data points, then it should use the normalized values like mean and average, makes no sense to me why you would want those fancy graphs unless explicitly set. I'm currently experiencing massive rendering issues even when only running one iteration.
PragTob commentedon Oct 5, 2025
👋
The
max_sample_sizethat I have yet to release should mostly combat all of this and we should be much better.I like having those fancy graphs as they have way more information about the distribution of values. Just the bar charts with stdev can sometimes lie to you/tell by far not the whole story. I wanna promote their usage.
Schultzer commentedon Oct 5, 2025
Is this in main?
PragTob commentedon Oct 6, 2025
@Schultzer yup! Just my bad for not releasing it in too long, I'm hoping I'll get to it this week or next.