Skip to content

json: a build with serde_json/arbitrary_precision rejects every number outside the i64 and u64 integer ranges #482

Open

Description

@iainmcgin

buffa's JSON decoding rejects every number that is not an integer in the i64 or u64 range when serde_json's arbitrary_precision feature is enabled by any crate in the build. That is every number with a fraction or an exponent, and every integer above u64::MAX or below i64::MIN. Cargo unifies features across a build, so a dependency that enables the feature for its own types changes how buffa decodes.

Reproduce

[dependencies]
buffa-types = { version = "0.9", features = ["json"] }
serde_json = { version = "1", features = ["arbitrary_precision"] }
use buffa_types::google::protobuf::{DoubleValue, Struct};

// Err: invalid type: map, expected a double, "NaN", "Infinity", "-Infinity", or null at line 1 column 3
serde_json::from_str::<DoubleValue>("1.5");

// Err: invalid type: map, expected a double, ... — a `double` field, for an integer above u64::MAX
// {"doubleValue": 18446744073709551616}

// Err: invalid type: map, expected a u64 as a quoted decimal string, integer, float, or null — a `uint64` field
// {"positiveIntValue": 1e3}

// Ok, but the value of "a" is a struct with the one key "$serde_json::private::Number"
serde_json::from_str::<Struct>(r#"{"a": 1.5}"#);

Integers in the i64 and u64 ranges decode as they do without the feature.

Cause

With arbitrary_precision, serde_json's deserializer keeps the text of a number that does not parse as an i64 or a u64. It hands that number to a visitor through visit_map, as an object with the one key $serde_json::private::Number and the text as the value. serde_json's own Number and Value types read that key. buffa's visitors for float, double and integer fields implement no visit_map, so they reject the map. The visitor for google.protobuf.Value reads the map as a struct.

What a fix needs

Each numeric visitor would accept that map and parse the text. The visitors cannot read the key unconditionally: in a build without the feature, the same object in the JSON text is an ordinary object, and must not decode as a number. The feature is not visible to buffa at compile time, because another crate can enable it, so the visitors need a runtime check that the deserializer produces the key for a number.

Until then, arbitrary_precision is documented as unsupported.

Activity

  1. changed the title [-]json: a build with `serde_json/arbitrary_precision` rejects every number with a fraction or an exponent[/-] [+]json: a build with `serde_json/arbitrary_precision` rejects every number outside the `i64` and `u64` integer ranges[/+] on Sep 24, 2026
  2. v0ropaev commented on Sep 28, 2026

    @v0ropaev
    Contributor

    On the runtime check, there is a way to get it without reading anything private out of serde_json: deserialize the literal 1.5 with a probe visitor and see which method it lands in. visit_f64 means the feature is off, visit_map means it is on. Only the public Deserializer contract is involved, and the answer can sit in an AtomicU8 so it costs one parse per process.

    struct Probe(bool);
    struct ProbeVisitor;
    
    impl<'de> Visitor<'de> for ProbeVisitor {
        type Value = bool;
        fn expecting(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
            f.write_str("a number")
        }
        fn visit_f64<E: de::Error>(self, _: f64) -> Result<bool, E> {
            Ok(false)
        }
        fn visit_map<A: MapAccess<'de>>(self, mut m: A) -> Result<bool, A::Error> {
            while m.next_entry::<IgnoredAny, IgnoredAny>()?.is_some() {}
            Ok(true)
        }
    }
    
    impl<'de> Deserialize<'de> for Probe {
        fn deserialize<D: Deserializer<'de>>(d: D) -> Result<Self, D::Error> {
            d.deserialize_any(ProbeVisitor).map(Probe)
        }
    }

    serde_json::from_str::<Probe>("1.5") gives false on serde_json 1 as it is and true with arbitrary_precision on, checked both ways in a scratch crate.

    With that gate, each numeric visitor can take a visit_map arm, read the one key, and replay the text as the scalar visit the same input makes in a build without the feature: try u64, then i64, then f64. That keeps 1e3 on an integer field working, since the feature hands over the text 1e+3 and parsing that as f64 gives the same 1000.0 that visit_f64 receives today.

    Happy to send it if that is the shape you want. It looks like it has to land across json_helpers, value_ext and the reflect path together though, since a dev-dependency on the feature unifies across the workspace the way raw_value already does, so the google.protobuf.Value decode would go wrong in the same test run.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions