Benchmarks measure glz::generic, which is not Glaze's performance path

[stephenberry]

First: nice work on the library, and thank you for how carefully the comparison harness is written. The agrees() gate in corpus_vs.cpp:311-322, the per-contender byte denominators, and especially gl_extract_reach with its comment about the 3841.68 GB/s bug are more rigor than most head-to-head JSON benchmarks bother with. The path_cache in glaze_shim.cpp:195-242 is a real attempt to use glz::lazy_json well rather than as a strawman, and the lazy chart honestly shows Glaze winning by two orders of magnitude at low N.

This issue is about the DOM rows specifically (parse, serialize, and the headline charts in the README), where I think the comparison is measuring something other than what it appears to measure.

I'm the author of Glaze, so treat this as an interested party's report rather than a neutral one — but the substance should stand on its own.

The issue

Every Glaze row in the DOM benchmarks goes through glz::generic. From comparison/glaze_shim.cpp:49-53:

struct state {
  glz::generic doc;
  std::string out;
  bool live = false;
};

and gl_parse (glaze_shim.cpp:260-268) is glz::read_json(s->doc, sv).

glz::generic is not how Glaze is used when performance matters, and it is not a peer of a tape. It is literally a tree of standard C++ containers:

// glaze/json/generic_fwd.hpp, elided
template <num_mode Mode = num_mode::f64, template <class> class MapType = ordered_small_map>
struct generic_json {
  using array_t  = std::vector<generic_json<Mode, MapType>>;
  using object_t = MapType<generic_json<Mode, MapType>>;
  using null_t   = std::nullptr_t;
  using val_t    = std::variant<null_t, double, std::string, bool, array_t, object_t>;
  val_t data;
};

using generic = generic_json<num_mode::f64>;

Parsing a document into it means, per node: a variant discriminant, an owning std::string for every string value and every object key, a std::vector per array with reallocation as it grows, and an ordered_small_map per object which allocates its entry array and — once the object exceeds 8 keys — lazily builds a sorted hash index for O(log n) lookup. Every node owns its own memory and every node is independently mutable and destructible.

glz::generic exists for the case where you genuinely do not know the schema at compile time and you want an ergonomic, owning, mutable tree. Its correct peer group is nlohmann::json, boost::json::value, and rapidjson::Document — which is exactly where it lands in your charts. It is not the API anyone reaches for on a hot path, and Glaze's design work has gone almost entirely elsewhere.

Why the DOM comparison is decided before any code runs

The comparison isn't cjson's parser against Glaze's parser. It's a flat 16-byte-slot arena against an owning node tree. Looking at what cjson's tape does not do that glz::generic must:

• Strings are never materialized. parse.hpp:447-460: the no-escape fast path null-terminates in the buffer copy and stores {offset, length}; unescape only runs when escapes are actually present. glz::generic allocates and fills a std::string for every string value and every key.
• Objects get no key index. doc.hpp:634-644 is a linear scan over members at access time. ordered_small_map allocates its entry array eagerly and builds a lookup index for objects past 8 keys. Glaze pays that inside the timed region; cjson never pays it anywhere in this benchmark.
• Nothing is individually owned. Teardown is one free against a full recursive destructor walk.
• No mutability guarantees. The tape's mutation story (mutate.hpp) is explicitly documented as invalidating handles on structural edits; glz::generic gives you an arbitrarily mutable tree.

Your own instruction counters show this cleanly. On the 26 MB headline: 797M instructions for glaze vs 135M for cjson. Glaze is genuinely retiring ~6x the instructions, because building an owning node tree genuinely costs that. The measurement is real. What it measures is representation, not parsing.

The giveaway is the clustering: rapidjson, glaze, boost.json and nlohmann all land within ~3x of each other, and cjson, simdjson-dom and yyjson land in a separate band. That's a tape-vs-tree split. Grouping all eight into one bar chart implies a single ranking that the data doesn't support.

I'd note for balance that cjson does execute more instructions than simdjson (135M vs 117M) and yyjson (135M vs 125M) on that same chart and still shows fewer cycles — that's a real IPC/locality win over the other tape parsers, and it's the part of the headline chart I'd actually stand behind as a like-for-like result.

How Glaze is meant to be used for performance

Glaze's premise is that you skip the intermediate representation entirely. Compile-time reflection over your struct generates the parser, so bytes go straight from the buffer into your fields — no DOM, no tape, no index, no per-node allocation:

struct twitter_user {
  int64_t id{};
  std::string screen_name{};
};

struct status {
  int64_t id{};
  std::string text{};
  twitter_user user{};
};

struct twitter {
  std::vector<status> statuses{};
};

// no metadata needed — pure reflection
inline constexpr glz::opts partial{ .error_on_unknown_keys = false };

twitter t;
auto ec = glz::read<partial>(t, buffer);

error_on_unknown_keys = false is what makes this a partial-extraction benchmark rather than a full-schema one, which is the realistic shape for something like twitter.json. Two other options are directly relevant to a benchmark harness:

• .minified = true — asserts the input has no insignificant whitespace and takes a faster reader. Worth a separate row if the corpus has minified variants.
• .partial_read = true — reads into the deepest structural object named by the target type and then stops without parsing the rest of the input.

Writing works the same way, and takes a caller-supplied buffer so it can be reused across iterations exactly like your cjson-reuse row:

std::string out;             // retained across reps
auto ec = glz::write_json(t, out);

For the "I only want a few fields out of a big document" case, get_view_json and lazy_json are the right APIs, and you're already benchmarking both.

Suggested test setup

Concretely, what I think would make this comparison say something true:

1. Add a typed row — this is the important one.

Define structs for two or three corpora (twitter.json partial-tweets is the conventional one; citm_catalog and canada are the other two from the nativejson-benchmark set). Then measure the end-to-end task on both sides: bytes in, populated user-defined struct out.

For Glaze that's glz::read<partial>(t, buffer). For cjson it's whatever the equivalent is — parse plus the walk plus the per-field extraction into the same struct, including the linear key scans. That's a fair fight because both libraries are being asked for the same product, and it's the comparison a user choosing between the two actually faces. I'd expect cjson to do well on the pure scan and Glaze to do well on avoiding the intermediate entirely; either way the result would be informative, which the current DOM rows are not.

2. If you keep an untyped row, separate the bands and label it.

Either split the chart into "tape/index builders" (cjson, simdjson-dom, yyjson) and "owning tree builders" (glz::generic, nlohmann, boost.json, rapidjson), or retitle the row to something like "build an owning mutable tree" vs "build an index". A one-line note that glz::generic is Glaze's schema-less fallback and not its fast path would also do a lot, given the README currently presents it as simply "glaze".

3. Charge both sides for answering the query, not just for parsing.

If the tape's design tradeoff is deferring string materialization and key indexing to access time, then a benchmark that stops at parse only measures half of it. The extract-dom group already does the right thing here; extending that idea — parse then read N fields — would show where each representation's cost actually lands.

4. Serialization from the typed struct.

glz::write_json(t, out) with a retained out, against cjson::write_into with the warm wbuf. Both sides then reuse their buffer and the row is apples-to-apples. Serializing glz::generic is pointer-chasing a node tree; serializing a struct is a straight-line write.

Two harness items, independent of the above

Compile flags. scripts/vsbuild:94 builds the glaze shim at -std=c++23 -O3 -march=native. Line 121-124 builds the cjson TU at -std=c++26 -Ofast -mavx2 -mbmi -march=native -flto. Since the shim objects are compiled without -flto, the link-time -flto only benefits cjson, and cjson is additionally inlined into the timing lambda while every contender pays an opaque extern "C" call. Negligible on a 26 MB document; not negligible on the 64kb row or the lazy rows at ~600-1200 cyc/op. Same -O level and same LTO treatment for all shims — or a note documenting the difference — would close this.

Amortization parity. cjson-reuse borrows a warm scratch and simdjson-dom reuses a persistent parser (simdjson_shim.cpp:79-83), while glaze, boost.json, nlohmann and rapidjson construct and destroy their tree every rep. The README subtitle discloses this for cjson but not that the field is mixed. Marking each row warm/cold in the chart would make the headline comparison self-describing.

Reproducibility

The repo ships the rendered PNGs but no benches/results/*.txt, and .glaze-vendor isn't committed, so the Glaze commit behind the published charts isn't recorded anywhere. Committing the raw result text alongside the charts, with the vendored commit hashes in the header, would let people re-run and diff. (I couldn't reproduce locally at all — micron, GCC 16, Linux perf and x86_64 are all required — so everything above is from reading the harness rather than from measurement.)

[rfgplk]

Yea, you were right on a few of those. Already updated the benches to be fairer (especially the simdjson warm scratch reuse issue and asymmetric build flags). Regarding glazes lazy parsing, these parsers effectively belong in different categories/where completely engineered for different use cases. Architecturally cjson is closest to simd/yy, both of which being tape parsers while glaze/rapid et al are all tree based, and fundamentally suited for different workloads. cjson entirely doesn't have any lazy style stop at first parse function, mainly because I made it for whole json doc parsing use cases (ie load up a whole API response or a mid workflow schema -> parse / do something with it -> serialize a response out) and importantly for those use cases you sort of WANT to parse every field explicitly (meaning every field is meaningful). Benchmarking the two categories is difficult and would be unfair in either direction so for clarity I split the charts up into two separate categories (index based/tree based).

Regarding the warm row for glz::generic, I tried looking for a warm scratch equivalent (similar to the simdjsons realloc_if_needed = false parsing into an existing buffer) but I genuinely couldn't find one. If there is a public API let me know.

Regarding benches/results/.* I've pushed them to the git repo (for Zen+) plus I'm planning on uploading a wider range of benches of different hardware/arches (Graviton/Intels core) in the near future.

[stephenberry]

Yeah, for whole JSON parsing, Glaze parses into structs, which often beats simdjson and this library's approach in performance, especially if some keys are missing or out of sequence.