bug: serde_json/arbitrary_precision + PickFirst causes issues

[OliverNChalk]

I have a peculiar bug where when I enable serde_json's arbitrary_precision feature flag and deserialize and f64 using PickFirst it inexplicably fails. I have created a minimal reproduction here:

https://github.com/OliverNChalk/serde-with-repro/blob/master/src/main.rs

There are two branches:

• master - code is broken
• fixed - code works (i just turn the feature off)

Thought I would file here as I assume you guys would be most familiar with the internals of what this derive macro is doing and how this behavior could be surfacing.

[jonasbb]

This is not a problem of this crate, but rather how serde_json chooses to transmit large numbers with the arbitrary_precision feature set and serdes failure to support extended data models and that any form of buffering (such as untagged enums) break various features (serde-rs/serde#1183).
There are also issues for this in the serde_json repo: serde-rs/json#740 and serde-rs/json#559.

PickFirst uses an untagged enum internally. But due to serde-rs/serde#1183 this can break stuff. Here you see the same problem without PickFirst.
https://www.rustexplorer.com/b/p77mis

/*
[dependencies]
serde.version = "*"
serde.features = ["derive"]
serde_json.version = "*"
# Comment the next line and see how the code successfully deserializes both values
serde_json.features = ["arbitrary_precision"]
*/

#[derive(Debug, serde::Serialize, serde::Deserialize)]
#[serde(untagged)]
enum Foo {
    S(String),
    F(f64),
}

fn main() {
    let f: Foo = serde_json::from_str(r#"123"#).unwrap();
    dbg!(f);
    let f: Foo = serde_json::from_str(r#"8.0e-5"#).unwrap();
    dbg!(f);
}

But I do have a workaround for you. You can add a third variant to the PickFirst, which first buffers into a serde_json::Value. That type understands how arbitrary precision values are encoded and can then be used to deserialize a f64 from it. The f64 then no longer has the arbitrary precision.
https://www.rustexplorer.com/b/77w4yn

/*
[dependencies]
serde.version = "*"
serde.features = ["derive"]
serde_json.version = "*"
serde_json.features = ["arbitrary_precision"]
serde_with = "*"
*/

use serde::Deserialize;
use serde_with::{serde_as, DisplayFromStr, PickFirst};

serde_with::serde_conv! {
    IntermediaryJsonValue,
    f64,
    |f: &f64| *f,
    |v: serde_json::Value| f64::deserialize(v)
}

fn main() {
    #[serde_as]
    #[derive(Debug, Clone, PartialEq, Deserialize)]
    struct MaybeFloatMaybeString(
        #[serde_as(as = "PickFirst<(DisplayFromStr, _, IntermediaryJsonValue)>")] f64,
    );

    println!("{}", serde_json::from_str::<f64>(r#"8.0e-5"#).unwrap());
    println!(
        "{}",
        serde_json::from_str::<MaybeFloatMaybeString>(r#"8.0e-5"#)
            .unwrap()
            .0
    );
    println!(
        "{}",
        serde_json::from_str::<MaybeFloatMaybeString>(r#""8.0e-5""#)
            .unwrap()
            .0
    );
}

[OliverNChalk]

Thanks a bunch for your response & explanation, really above and beyond :)