[lvaroqui]

Export the traces and logs that divan benchmarks emit with tracing and log, for the telemetry mode of the CodSpeed runner (CodSpeedHQ/codspeed#589) to collect.

Using it takes the telemetry feature of codspeed-divan-compat and a telemetry build:

cargo codspeed build -m telemetry            # or with other modes, e.g. -m simulation,telemetry
codspeed run -m telemetry -- cargo codspeed run

When CODSPEED_OTEL_ENDPOINT is set, divan::main() installs a tracing subscriber exporting over OTLP/HTTP:

• spans as traces;
• tracing events and log records as logs, carrying the trace and span IDs of the span they are emitted in.

CODSPEED_TELEMETRY_FILTER takes EnvFilter directives, info by default. Benchmarks that install their own subscriber add divan::telemetry::layer() to it.

cargo-codspeed

• A new telemetry measurement mode builds the benchmarks to run once, like simulation. It can be built along other modes.

• Measured builds keep tracing and log as they are: without a subscriber, which only the telemetry build installs, a disabled callsite costs a runtime level check, as in production.

• Every build passes --cfg=codspeed_build, and the telemetry build --cfg=codspeed_telemetry too, so that a project can compile tracing and log out of its measured builds from its own manifest:

  [target.'cfg(all(codspeed_build, not(codspeed_telemetry)))'.dev-dependencies]
  tracing = { version = "0.1", features = ["max_level_off"] }
  log = { version = "0.4", features = ["max_level_off"] }

• The OpenTelemetry dependencies sit in a cfg(codspeed_telemetry) target table. With the feature on, measured builds only add tracing and tracing-subscriber to the dependencies of codspeed (5 crates, against about 100 in the telemetry build), and a project without the feature gets nothing new in its lockfile.

Timing

Spans and log records keep the wall clock times of the SDK. They also carry their times on the monotonic clock of the benchmark markers, which wall clock adjustments don't move: spans as codspeed.monotonic_start_ns and codspeed.monotonic_end_ns, log records as codspeed.monotonic_time_ns. These are 0 without instrument hooks.

Worth a careful look

• How the span times are taken, in crates/codspeed/src/telemetry/monotonic.rs. MonotonicTimestampLayer comes before the OpenTelemetry layer and takes the same moments it does: the start when the tracing span is created, the end when it is last exited, or when it is closed if it was never entered. It sets both attributes when the span closes, before the OpenTelemetry layer ends and exports it. A span exited on several threads keeps its latest exit.
• The filters hint TRACE as their max level, so that LogTracer keeps the level of the EnvFilter: without it, filter_fn lifted it to TRACE.
• init warms the subscriber up with a span and an event that are not exported: the first span of the process took 18 to 31 µs longer than the next ones.
• The exporter's own HTTP stack (hyper, tonic, h2, reqwest, as in the upstream basic-otlp example) is always filtered out, or exporting would emit more spans in an endless loop. Telemetry of these crates emitted by the benchmarks is lost with it.
• reqwest pulls icu_* 2.x crates, which require rustc 1.88: they are resolved with the MSRV-aware fallback resolver. sharded-slab fails to build with the lazy_static 1.0 that minimal-versions picks, hence a lazy_static = "1.1" floor.

crates/divan_compat/examples/benches/telemetry.rs emits telemetry in each of these ways: nested spans with events, an event outside of any span, records of the log crate, and nothing at all. The telemetry tests only build with --cfg codspeed_telemetry; CI runs them, and checks the MSRV and minimal versions, with it. The cargo-codspeed tracing tests check that tracing and log are kept in every mode, and that the manifest table above compiles them out of the simulation and walltime builds but not the telemetry one.

The feature will be documented on the docs site rather than in the README.

Closes COD-3862

[codspeed]

Merging this PR will regress 16 benchmarks

4 warnings, 1 note

⚠️ Unable to measure the execution of 59 benchmarks

These benchmarks measured no execution time, so their results are not comparable and count as unchanged. This usually happens when the compiler removes the code under test.

Prevent compiler optimizations

See affected benchmarks

⚠️ 17 benchmarks are dominated by system calls

System calls cannot be simulated consistently and are left out of the measure. The reported value understates the real cost. Switch to the Walltime instrument to measure them.

Learn more about system calls

See affected benchmarks

⚠️ 8 benchmarks with a significant change ran on different hardware

Their base and head results were measured on machines with a different CPU, core count or memory, so the changes might not come from the code. Run base and head on the same runner type.

Learn more about Macro Runners

See affected benchmarks

⚠️ 63 benchmarks with a significant change ran in a different software environment

Their base and head results were measured with a different OS or toolchain, such as compiler, runtime or library versions, so the changes might come from this difference rather than from the code. Open an affected benchmark to see what changed.

See affected benchmarks

ℹ️ 57 benchmarks have no flamegraph

• The flamegraph wasn't generated: most of the time was spent in runtime internals, which would leave nothing useful to analyze. (9 benchmarks)
• The flamegraph wasn't generated: no execution data was recorded while the benchmark ran. Make sure the compiler didn't optimize the benchmark out. (48 benchmarks)

See affected benchmarks

⚡ 61 improved benchmarks
❌ 16 regressed benchmarks
✅ 529 untouched benchmarks
🆕 12 new benchmarks

Warning

Please fix the performance issues or acknowledge them on CodSpeed.

Performance Changes

	Mode	Benchmark	BASE	HEAD	Efficiency
❌	CPU Simulation	spawn_on_parent_runtime	55.2 µs	91 µs	-39.38%
❌	Walltime	recursive_memoized[BTreeMap<u64, u64>, 10]	605 ns	995 ns	-39.2%
❌	CPU Simulation	recursive_memoized[HashMap<u64, u64>, 0]	1.6 µs	2.4 µs	-32.43%
❌	CPU Simulation	rem	118.9 ns	174.4 ns	-31.85%
❌	Walltime	hamiltonian_cycle[7]	1 µs	1.3 µs	-18.26%
❌	CPU Simulation	iter_manual_simple	894.9 ns	1,054.5 ns	-15.13%
❌	Walltime	from_elem_decimal[1024]	179 ns	207 ns	-13.53%
❌	CPU Simulation	filter_items_with_counter	7.7 µs	8.5 µs	-9.7%
❌	CPU Simulation	generate_gray_code[2]	5.5 µs	6 µs	-8.61%
❌	CPU Simulation	generate_gray_code[3]	6.9 µs	7.5 µs	-7.52%
❌	Walltime	hamiltonian_cycle[5]	878 ns	941 ns	-6.7%
❌	Walltime	iterative[10]	17 ns	18 ns	-5.56%
❌	CPU Simulation	generate_combinations[5]	12.3 µs	12.9 µs	-4.79%
❌	Walltime	recursive_memoized[BTreeMap<u64, u64>, 30]	3.4 µs	3.5 µs	-4.42%
❌	CPU Simulation	generate_combinations[6]	16.2 µs	16.9 µs	-3.84%
❌	CPU Simulation	fibo_recursive	1.6 µs	1.7 µs	-3.28%
⚡	CPU Simulation	b	4.9 µs	2.3 µs	×2.1
⚡	CPU Simulation	from_elem[1024]	5.4 µs	2.7 µs	+97.51%
⚡	Walltime	string_processing_multi_counter	706 ns	364 ns	+93.96%
⚡	CPU Simulation	build_vec	5.4 µs	2.8 µs	+93.82%
...	...	...	...	...	...

ℹ️ Only the first 20 benchmarks are displayed. Go to the app to view all benchmarks.

Tip

Investigate this regression by commenting @codspeedbot fix this regression on this PR, or directly use the CodSpeed MCP with your agent.

---

_{Comparing cod-3862-featrust-otel-cargo-feature-in-codspeed-divan-compat (4f9f700) with main (e3145d0)}

[greptile-apps]

Confidence Score: 5/5

[Medium impact] The PR appears safe to merge; no new actionable issue was established.

Summary

Adds a separate telemetry build mode and optional OTLP/HTTP export for Divan benchmarks.

• Divan benchmarks send spans and logs to an OTLP receiver.
• Exported spans and logs include benchmark-clock timestamps.
• cargo-codspeed builds benchmarks in a separate telemetry mode.

The shared-span timestamp race is fixed by keeping the greatest exit time under the lock. lvaroqui dismissed the print-call rule as inapplicable because this repository uses stderr, exporter failures must not return through tracing, and the other calls serve Cargo or test fixtures. Greptile retracted that finding.

Diagram

%%{init: {'theme': 'neutral'}}%%
flowchart LR
  A["cargo codspeed build -m telemetry"] --> B["Benchmark executable"]
  B --> C["divan::main()"]
  C --> D["Install subscriber and keep guard"]
  D --> E["Run benchmarks"]
  E --> F["Spans → traces"]
  E --> G["Events and log records → logs"]
  F --> H["OTLP/HTTP receiver"]
  G --> H
  E --> I["Return from main"]
  I --> J["Guard flushes exporters"]

Loading

_{Reviews (3) · Last reviewed commit: "feat: keep tracing in measured builds, r..." · Reviewed by Greptile}