Api proposal: Change JsonSerializerOptions default settings

New API:

public static class JsonSerializerOptions
{
    // Can only be set until the first (de)serialization occurs (like other properties here)
    public static Func<JsonSerializerOptions> Default { get; set; }
}

Newtonsoft \ JSON.NET does it this way:

public static class JsonConvert
{
    public static Func<JsonSerializerSettings> DefaultSettings { get; set;}
}

The upside is that you can have different settings for different situations, including different options per assembly, even though that would require the developer be aware of the potential pitfalls of that. The fact that it has also worked for many years in JSON.NET I think is a plus as well.

Performance could suffer a little bit, would have to test. Either way, one could always get over that by explicitly passing their options.

Original text:

Provide a way to change the default settings for the static JsonSerializer class. In Json.NET, you could do:

JsonConvert.DefaultSettings = () => new JsonSerializerSettings();

Currently you have to keep passing the options around and remembering to pass it to each call to [De]Serialize

Currently you have to keep passing the options around and remembering to pass it to each call to [De]Serialize

Can you expand on why that is a concern for you and what scenario is blocked by the current design (preferably with some sample code)?

Given the JsonSerializer is a static class containing purely static methods, it doesn't contain any state atm. Providing rationale for why this needs to change would help motivate it (other than the fact that Json.NET has this behavior) since it does introduce some complexity in the design.

Can you expand on why that is a concern for you

JSON properties are conventionally camelCase. .NET type's members, on the other hand, are conventionally PascalCase. I don't really want to stray away from any of these two conventions.

The solutions today, as far as I'm aware, all involve passing things around:

You can annotate every single property in your project with JsonPropertyNameAttribute. This is not nice, convenient or even desirable. [JsonPropertyName("foo")] int Foo { get; set; } adds unnecessary verbosity and feels and looks completely redundant. The only thing changing between the two lines is the first character casing. It is also error prone.
Pass options around everywhere. I think this one speaks for itself. If you're passing options around everywhere, you should consider instead doing it in one place only. This is the path we're following with our project, which I talk more about in the end.
Pass a serializer around everywhere as well. Although this one seems to make some sense at first (oh, you need JSON serialization? Ask the IoC for a serializer!), it does make you wonder: why have a static helper class in the first place then? As far as I'm aware, this requires writing code considerably more involved than simply calling JsonSerializer.Desearilize<T>("..."). So you won't be writing conventional code just because you need different options. And again, error prone.

Now, of course one could argue that the correct way is indeed to have case-sensitivity, since you can always have JSON like { "foo": 1, "Foo": 2}. However, being honest, this is not a common case -- and not only that, but I don't really see people writing this kind of JSON and expecting it to work 100% in the wild. Of course it's on the spec, but realistically it will simply break lots of APIs out there. Finally, who governs the names of accepted properties is the API, thus, our API obviously do not do such things. In fact, creating classes like that would go against the .NET design guidelines.

A better option, perhaps, would be to inform the serializer that by default all properties are to be assumed camelCase (so, they would remain case-sensitive, but by default be assumed JSON camelCase even though being written PascalCase in C#). However, the problem here is the "by default" bit, which again boils down to the same problem exposed in this issue.

All of our projects currently expect this behavior. Changing this wouldn't be a deal breaker for adopting System.Text.Json, but it is surely requiring some unexpected changes. The first change we did was to make our REST internal library now accept a JsonSerializerOptions in its constructor so that it can be set during DI registration. However, there are a considerable number of other places in the project where we use JSON serialization for other purposes. For those other scenarios, the only idea we currently have is to pass options around.

If there was an instance, non-static version of JsonSerializer we could also make extension methods. This unfortunately still wouldn't change the behavior of the entire running program, but would at least change the behavior for our assembly.

Why don't you declare the default setting in a static class and then just do

static class DefaultJsonSettings {
	public static readonly JsonSerializerSettings Settings = new JsonSerializerSettings {
		NullValueHandling = NullValueHandling.Ignore,
		// [...]
	};
}

 JsonConvert.SerializeObject(myObject, DefaultJsonSettings.Settings);

That's what I did to avoid passing settings around but making sure it's getting serialized the same everywhere in project. If your wish is to set the settings ONCE at start you already know how the settings should be so static is fine. Otherwise if you load the settings, so they are unknown before application start, it cannot be static itself but you could do the same but with a static reference to an instance:

public sealed class DefaultJsonSettings  {
	static DefaultJsonSettings instance;

	public static DefaultJsonSettings Instance => instance ??= new DefaultJsonSettings();

	public DefaultJsonSettings() {
		//Load JSON settings here
		Settings = new JsonSerializerSettings {
			NullValueHandling = NullValueHandling.Ignore,
			// [...]
		};
	}

	public static JsonSerializerSettings Settings;
}
 JsonConvert.SerializeObject(myObject, DefaultJsonSettings.Instance.Settings);

Sure, but that still kind of have the same issues as passing it around everywhere, except they're not a method parameter.

You still have to remember to do it, and thus it's still error prone, and it still feels quite redundant to be passing the "default" options to every single method call. If it's supposed to be the default, why do I need to specify it every single time? It's like having C# optional parameters, but still having to specify them in every call.

why do I need to specify it every single time?

Because of what ahsonkhan said. Its a static class with static methods for performance reasons. You want add performance decreases by making the method not static only to avoid remembering to add "DefaultJsonSettings.Settings" every time you use it. This seems like a really bad trade, especially for the great majority where the current way it works isn't a big deal.

If this is such a big deal for you, why don't you just write DefaultJsonSettings.Settings once and put that inside a static method and call this instead, problem solved.

static class Bla {
	public string SerializeObject(object myObject)
	{
		return JsonConvert.SerializeObject(myObject, DefaultJsonSettings.Settings);
	}
}

Bla.SerializeObject(myObject);

And please do not say you don't want this because you have to remember using Bla.SerializeObject instead of JsonConvert.SerializeObject

@ahsonkhan

And please do not say you don't want this because you have to remember using Bla.SerializeObject instead of JsonConvert.SerializeObject

How so "please do not say"? Am I not allowed to express my opinion? By saying so, you show that you do understand the issues I'm raising and you do understand that they're present in the ""solution"" you suggest. That's dishonest. Remember that different people have different needs, works in different projects and under different scenarios. Your opinion is not golden. It's just as valid as mine. If my suggestion doesn't prove to be much useful, don't worry, it simply won't get traction and won't get implemented. Don't think there's a need for you to harass anyone.

You want add performance decreases

I want no such thing. Never said it. I'm merely asking for a way to have default settings. The implementation is up for discussion.

only to avoid remembering to add "DefaultJsonSettings.Settings" every time you use it

It's not "only" to avoid remembering. When you argue like that, you can make any argument look pointless. As I said in previous replies, this is simply a point of great coding practices. Anyone who owns a codebase and see a thing such as having a method being called all around the solution every time with the same argument would do something about it. And again, this is because it is simply a point of great coding practices. It's easy to see the point.

This seems like a really bad trade

Only if it has to be that way. I too agree that if a non-insignificant performance impact is required in order to do this, it wouldn't pay itself, and I wouldn't support it.

@ahsonkhan Can't we just create a new overload that doesn't have the JsonSerializerSettings optional parameter and that just redirects the call, passing the DefaultSettings as parameter? Sure, that would introduce state in the class, but I don't understand why is that a problem per se. And AFAIK that wouldn't be a breaking change either if the DefaultSettings property is default initialized with null. I'm sorry if there's other complexity involved that I'm not aware, I haven't looked at the code yet for that class.

Just as a quick note, I think it's also worth remembering that another benefit of having a static property is that you're able to change the serializer behavior for all assemblies in the running program. I don't depend on this behavior currently though, so I can't say how much of a benefit that would be. I guess it could be useful in scenarios where you have code over which you have no control that uses JsonSerializer internally and doesn't offer an way to change the default options.

I really think this is a huge trade-off. It can be a huge pit-of-failure in libraries which interact with web APIs which expect a specific casing. Of course, they should be written in a way which is not vulnerable to this, but by making it the "default", it makes it a lot easier for bad code to be written.

Personally, I don't really think specifying JsonPropertyName for every property name is a negative - in fact, I feel that explicitly specifying property names is advantageous, even if it can seem like code duplication.

Of course, they should be written in a way which is not vulnerable to this, but by making it the "default", it makes it a lot easier for bad code to be written.

I understand your point. However, I'm not sure if you misunderstood me or if I'm misunderstanding you 😄 but I'm not advocating for that behavior to be the default; instead, only to give users the option to have a default, whatever that might be.

Having said that, please note then that 1) you're focusing on case-sensitivity, when that was merely the reasoning behind my need for this. This issue is not about case sensitivity, it is about offering a way to change the defaults only once. No one is asking here for case-insensitive to be the default behavior anywhere and of anything; and 2) your point boils down to "giving developers a choice means they will write bad code and thus let's not give them a choice". And although I can see an argument like that being applied elsewhere, I don't really think it applies here.

I'm still going to reply to each one of your points, but I think this (whether or not case-sensitivity should be the default) is mostly something not worth arguing over, as it is not the point of this issue.

Please note that this has existed in JSON.NET for many many years and I don't think a lot of "bad" code has been written because of it. In fact, in JSON.NET, that behavior (case-insensitive) is the default behavior (but again, I am not saying it should be, I'm merely bringing that fact to light).

It can be a huge pit-of-failure in libraries which interact with web APIs which expect a specific casing

First of all, notice that I'm more interested in the deserialization behavior than the serialization one. I don't really care how the payload is going to be serialized.

As said above, 1) I am not advocating for this to be the default behavior; and 2) this has existed for years and I don't really see this ever being a problem. And, in fact, as I said before, I actually think it's the other way around: .NET developers are more familiar with cC -> CC because of the clash between the .NET naming guidelines and JS naming conventions. And that's not really a problem at all, since, as mentioned before, people simply don't go around there sending "foo: 1, "Foo": 2 JSON payloads.

Also, please note that this is the default behavior for MVC (and still continues to be, even after System.Text.Json: dotnet/aspnetcore#10723) and, again, people are not writing "bad" code because of it. And, most importantly, MVC was engineered in a way where you can, in fact, change the default only once.

I really think this is a huge trade-off.

So I don't think there is really any trade-off here (in the context of what you said -- note that I do understand that there may be some trade-offs in terms of performance and/or API design!).

By having the option to specify the defaults in System.Text.Json, not only it would give us more flexibility, since - and this is important to note - there are other behaviors besides case-sensitivity that can be controlled through the options, but it would also pave the way for a more seamless migration between the two libraries.

Personally, I don't really think specifying JsonPropertyName for every property name is a negative - in fact, I feel that explicitly specifying property names is advantageous, even if it can seem like code duplication.

Right. I'm a big advocate of the idea of being explicit. However:

I do not like the idea of specifying behavior in that way. One thing is having a property with a name that is not a valid C# identifier and having to enable that scenario through the use of an attribute. Another thing entirely is peppering your entire code with JsonPropertyNameAttributes everywhere, specifying through the use of a string that the behavior you expect is "transformation" of camelCase to PascalCase. The behavior is encoded not in code, but in the developer's mind. [JsonPropertyName("foo")] string Foo { get; set; } doesn't really say what you're meaning. A future developer entering your project might not notice at first hand that that is a behavior expected by the entire project, instead of being just a few attributes here and there. On the other hand, something like Options.CaseInsensitive = true is explicit and conveys all the meaning needed. By only looking at a single line such as that, a future developer can understand how your project deals with JSON.
It is brittle. You can't use nameof. You can't specify behavior, as said above (like something as [JsonProperty(JsonSerializationOptions.CaseInsensitive)] would). And logic is smeared all over your code, instead of of one place. Which leads me again to the obvious next point, also addressing one of yours;
It is duplication. It not only looks like it. It is behavior what you're describing when writing that attribute and you're specifying it literally everywhere. Again, even though I do love the idea of being explicit, I just can't feel that the advantages there cover the disadvantages of having to use attributes everywhere, especially in a situation like this, with all these other mentioned disadvantages. We've seen this before many many times in the BCL and other MS frameworks (cough DataMemberAttribute cough). It is not by chance that they strayed away from this philosophy.

The feature ask is valid IMO and was discussed early on when designing JsonSerializerOptions.

Currently the "default" option is on a private static variable. This means it can only contain default settings.

Also, FWIW, we also discussed add assembly-level attributes that can configure things like the naming policy. This would support some of the scenarios without having to specify where the default option lives.

((JsonSerializerOptions)typeof(JsonSerializerOptions)
                .GetField("s_defaultOptions", System.Reflection.BindingFlags.Static | System.Reflection.BindingFlags.NonPublic).GetValue(null))
            .PropertyNameCaseInsensitive = true;

😈😈😈

The solution @andre-ss6 came up with above using the internal default settings demonstrates that this is a real issue.
Can we get some movement on this? It's a pain to have to deal with libraries using System.Text.Json that don't let me configure these options.

Then the options could vary depending on which assembly is calling into System.Json.Text

I think that might be worse than having a global setter. It absolutely has the potential of catching users by surprise.

@eiriktsarpalis

@eiriktsarpalis I totally get that and agree to some extent. I guess I am a little confused on what the sentiment is. Are we at a point where you are willing to accept a solution with a public static default options (mutable or immutable), or should the solution exhibit other characteristics/behavior, and if so which?

.NET 7 will expose an immutable default instance (via #61093). For reasons already stated in this thread, I don't believe we would ever consider making it settable.

One possible alternative might be supporting "JsonSerializer" instances: an object encapsulating JsonSerializerOptions and exposing instance method equivalents to the existing static API. That might be one way of avoiding the common pitfalls of forgetting to pass the right JsonSerializerOptions instance or creating a new options instance on each serialization op.

@IanKemp

Then I will reiterate @IanKemp's suggestion (#31094 (comment)):

public interface IJsonSerializer
{
    Task<T> DeserializeAsync<T>(Stream utf8Json, JsonSerializerOptions options, CancellationToken cancellationToken);

    // other methods elided for brevity
}

public class DefaultJsonSerializer : IJsonSerializer
{
    private JsonSerializerOptions _defaultOptions;

    public DefaultJsonSerializer(IOptions<JsonSerializerOptions> defaultOptions)
    {
        _defaultOptions = defaultOptions.Value;
    }

    public async Task<T> DeserializeAsync<T>(Stream utf8Json, JsonSerializerOptions options = default, CancellationToken cancellationToken = default)
        => await JsonSerializer.DeserializeAsync<T>(utf8Json, options ?? _defaultOptions, cancellationToken);
}

Is this closer to something you will consider @eiriktsarpalis?

Something like that, although probably without the interface and IOptions dependency. I also think that the instance methods should not accept an options parameter to emphasize encapsulation.

At risk of stating the obvious, such a class can easily be defined by users and would introduce duplication of serialization APIs. It's a fairly drastic intervention providing comparatively low benefit and as such we should not do it unless we are absolutely convinced it's the way forward.

I think as a stop gap, registering in DI a IJsonSerializer in the with the JsonOptions.JsonSerializerOptions set to whatever has been configured in AspNetCore would be a good starting point so that we can use it inside the app, or retrieve it from services in testing. Would also add an easier access / discoverability for it in WebApplicationFactory as more often then not you will be serializing/deserializing Json and most likely want to use the same options.

Most of my use cases have been either integration testing an AspNetCore app, or within the app itself where I want to do some logging of some sort, or persisting Json into external storage systems.

@krwq

Here's another reason why we might want to consider introducing a JsonSerializer instance API: all current serialization methods accepting JsonSerializerOptions have been marked RequiresUnreferencedCode due to them using reflection-based serialization by default. This is behavior that predates the introduction of source generators in .NET 6 and contract customization in .NET 7.

The virality of RequiresUnreferencedCode can be a nuisance when using otherwise perfectly linker-safe code in trimmed applications. Using instance serialization methods means that we can push RUC annotations to specific constructors/factories and have linker warnings only surface in the app's composition root.

@krwq @eerhardt @jeffhandley thoughts?

@vitek-karas

we might want to consider introducing a JsonSerializer instance API

I chatted a bit with @vitek-karas about this yesterday. Being able to put all the "JSON source gen information" (i.e. JsonTypeInfos / JsonSerializerContext, or an instance of JsonSerializer) into DI makes a lot of sense to me. This could be a nice way for ASP.NET to use the JSON source generated code when they serialize objects.

One scenario that might need thinking about is if you want to serialize the same Type in multiple ways - lets say PascalCase in one case, and camelCase in another (or any other option that the source gen respects - like WriteIndented).

Either way - I think having a "DependencyInjection-friendly" way of serializing objects makes sense and would be valuable. This could be to make JsonSerializer be instantiable and have normal instance methods, or some other design like where the DependencyInjection info (JsonTypeInfos / JsonSerializerContext) gets passed to the static JsonSerializer methods. I do like the idea of instance methods on JsonSerializer, and then having certain ways of creating the instance of the JsonSerializer be RequiresUnreferenced/DynamicCode. That seems clean.

One more issue is that you'd still want the default experience to work in ASP.NET, when JSON source generation isn't used at all. That means a "fallback" or "default" experience (i.e. Reflection) needs to still work for "normal" apps. However, this "fallback" or "default" code will raise trimming warnings. To get rid of those, we could introduce a feature switch that removes the Reflection experience and ONLY works with trim-safe serialization code. Then your app wouldn't use Reflection to do serialization at all.

cc @davidfowl - FYI since we've discussed this "how can ASP.NET use JSON source generation code" topic a lot.

I think another important capability which ASP.NET would probably make use of is the ability to "merge" several such instances. For example let's say my app uses a library FabulousObjects which contains source generated serialization code for its objects because it calls into ASP.NET on my app's behalf. But my app also contains other source generated code for app's objects, which again are serialized by ASP.NET. For this to work both the app and the FabulousObjects would need to register the source generated code with the DI - but the fact that there are 2 sources should be transparent to ASP.NET itself, thus the need to "merge" the two instances into one which can serialize objects from both the app the library.

Such design would probably work better if we DI the contexts which can be merged, but maybe there's a way to do this with serializer instances themselves.

Having an "instance" serializer makes a lot of sense to me. The interesting pivot point to me is the concept of having a "serializer" vs a "serialization context".

What I mean by that is that we could consider three separate levels of serialization state.

At the highest level we have a static Serialize method that takes a type and some options, and preferably consumes no global state at all. This is what we already have. Each invocation is entirely self-contained. There is no state preserved between invocations.

The next level is a kind of serialization context, which bakes in some set of options. This version would allow you to preserve options between invocations, but wouldn't preserve any information about the serialization process itself.

The last level is the actual serialization process. This is an inherently stateful operation where the output and input are held in intermediate state as we walk the type graph (or text in the case of deserialization). This level would fix the input and the output and would be essentially one-shot, not reusable across multiple serializations.

Right now in serde-dn I have support for the first and last abstraction levels, but not the middle one, i.e. this is the implementation of JsonSerializer.Serialize, which is missing a set of options:

        public static string Serialize<T>(T s) where T : ISerialize
        {
            using var bufferWriter = new PooledByteBufferWriter(16 * 1024);
            using var writer = new Utf8JsonWriter(bufferWriter);
            var serializer = new JsonSerializer(writer);
            s.Serialize(serializer);
            writer.Flush();
            return Encoding.UTF8.GetString(bufferWriter.WrittenMemory.Span);
        }

One way to fix this would be to introduce a JsonSerializerContext type which takes and stores options.

The tricky bit in my mind here is the source generation stuff. To me that doesn't necessarily fit in any of the abstraction levels, as IMHO the source generation context should be carried with the type, not the options.

Serde solves the association problem by directly providing the source generation into the user type by implementing an appropriate interface. I wrote up some details in the docs on how that works, and how it can generate wrappers for types which aren't under the user's control: https://commentout.com/serde-dn/generator.html

Want to add that this is especially a concern for me when writing WebApplicationFactory based integration tests (or related unit tests). I was looking for a way to make xUnit globally set the same "Web" defaults for deserializing data. A shame that it's not possible, and a real toss-up for me between the Reflection hack above or "having to remember" using my own convenience deserialization methods in all tests.

Being able to set a more sane default for PropertyNameCaseInsensitive globally should outweigh the desire to keep all static things not mutable.

But perhaps there's greater concerns that block making this issue from being resolved?

EDIT: Right now I contemplate doing something like this to support my xUnit API Integration tests:
private static readonly JsonSerializerOptions jsonSerializerOptions
    = new JsonSerializerOptions(JsonSerializerDefaults.Web);

static HttpClientExtensions()
{
    jsonSerializerOptions.Converters.Add(new JsonStringEnumConverter());
}
to use it along these lines:
public static async Task<T> GetAsync<T>(this HttpClient client, string url)
{
    return await client
        .GetAsync(url)
        .ReadAsDeserializedData<T>();
}

private static async Task<T> ReadAsDeserializedData<T>(this Task<HttpResponseMessage> responseTask)
{
    var response = await responseTask;
    response.EnsureSuccessStatusCode();
    var json = await response.Content.ReadAsStringAsync();
    var data = JsonSerializer.Deserialize<T>(json, jsonSerializerOptions);
    if (data == null) throw new Exception("Unexpectedly encountered null response body.");
    return data;
}
😢

Cleanest solution for now.

Still not great to have to pass it in every serialization. Id rather have the risk of not configuring it when needed than to add the options everytime....

Per previous discussions in this thread, we don't plan on making JsonSerializerOptions.Default mutable in future releases. Exposing global mutable state can have unintended consequences: within the context of one application, multiple components can depend on the default options object, including internal implementations of third-party components. As such, assumptions about the application author owning all serialization calls within the process are transient at best.

At the same time, the conversation has spun into a discussion about potentially exposing the serialization APIs as instance methods: I've opened a separate issue (#74492) to track this, feel free to contribute to the discussion there.

@eiriktsarpalis

@eiriktsarpalis Nobody would have wanted to change the default unless you guys chose some completely irrational value as default. With the current default, it is almost impossible to use the JsonSerializer without changing the current default values. Without any second thought following two should have been defaulted in the library:

PropertyNameCaseInsensitive = true,
PropertyNamingPolicy = JsonNamingPolicy.CamelCase,

If your default is needed to change every time, then your default is entirely wrong and useless.

You can ask the whole world (except dotNET Team), and almost everybody will say that JSON deserialization must be case insensitive and serialization should be in camelCase by default.

What's wrong with changing the above two defaults in the library as these would not break any existing thing?

Navigation Menu

Repository navigation

Api proposal: Change JsonSerializerOptions default settings #31094

69 remaining items

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Api proposal: Change JsonSerializerOptions default settings #31094

Description

Activity

ahsonkhan commented on Oct 7, 2019

artdedeco commented on Oct 8, 2019

Symbai commented on Oct 9, 2019

artdedeco commented on Oct 9, 2019

Symbai commented on Oct 9, 2019

artdedeco commented on Oct 9, 2019

artdedeco commented on Oct 9, 2019

FiniteReality commented on Oct 10, 2019

artdedeco commented on Oct 10, 2019

steveharter commented on Oct 10, 2019

artdedeco commented on Oct 17, 2019

Alexei-B commented on Nov 20, 2019

AndrewZenith commented on Nov 27, 2019

francescocristallo commented on Nov 30, 2019

ahsonkhan commented on Dec 3, 2019

69 remaining items

eiriktsarpalis commented on Jun 22, 2022

schoder-moreno commented on Jun 22, 2022

eiriktsarpalis commented on Jun 22, 2022

schoder-moreno commented on Jun 24, 2022

eiriktsarpalis commented on Jun 25, 2022

michal-ciechan commented on Jul 8, 2022

eiriktsarpalis commented on Jul 9, 2022

eerhardt commented on Jul 12, 2022

vitek-karas commented on Jul 12, 2022

agocke commented on Jul 12, 2022

wrkntwrkn commented on Jul 23, 2022

eiriktsarpalis commented on Sep 2, 2022

tanvirarjel commented on Sep 2, 2022

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions