System.Text.Json Serialization Regression in NET6 (Net5 worked)

[jogibear9988]

Description

I Serialize a Class with an IQueryable in it, looks like this:

public class WebsocketResponse {
public object Response {get;set;}
}

Reproduction Steps

Test Project:

use following 2 nugets:

<PackageReference Include="linq2db" Version="3.5.1" />
<PackageReference Include="Microsoft.Data.Sqlite.Core" Version="5.0.11" />

and following code:

      using LinqToDB.Data;
      using System.IO;
      using System.Text.Json;
      
      using (var dc = new DataConnection("Sqlite", "Data Source=:memory:"))
      using (var ms = new MemoryStream())
      {
          using (var writer = new Utf8JsonWriter(ms))
          {
              JsonSerializer.Serialize(writer, new Response { Result = dc.GetTable<DbTable>() });
          }
      }
      
      public class DbTable
      {
          public string A { get; set; }
      }
      
      public class Response
      {
          public object Result { get; set; }
      }

Expected behavior

Serialization should work as it did before.

Actual behavior

System.NotSupportedException: 'The type 'LinqToDB.Linq.Table`1[...]' can only be serialized using async serialization methods.'

Regression?

Yes, this did work in Net5

Known Workarounds

No response

Configuration

No response

Other information

No response

Tagging subscribers to this area: @dotnet/area-system-text-json
See info in area-owners.md if you want to be subscribed.

Issue Details

---

Description

I Serialize a Class with an IQueryable in it, looks like this:

public class WebsocketResponse {
public object Response {get;set;}
}

Reproduction Steps

Test Project:

use following 2 nugets:

<PackageReference Include="linq2db" Version="3.5.1" />
<PackageReference Include="Microsoft.Data.Sqlite.Core" Version="5.0.11" />

and following code:

    using LinqToDB.Data;
    using System.Text.Json;
    
    using (var dc = new DataConnection("Sqlite", "Data Source=:memory:;Version=3;New=True;"))
    using (var ms = new MemoryStream())
    {
        using (var writer = new Utf8JsonWriter(ms))
        {
            JsonSerializer.Serialize(writer, new Response { Result = dc.GetTable<DbTable>() });
        }
    }
    
    public class DbTable
    {
        public string A { get; set; }
    }
    
    public class Response
    {
        public object Result { get; set; }
    }

Expected behavior

Serialization should work as it did before.

Actual behavior

System.NotSupportedException: 'The type 'LinqToDB.Linq.Table`1[...]' can only be serialized using async serialization methods.'

Regression?

Yes, this did work in Net5

Known Workarounds

No response

Configuration

No response

Other information

No response

Author:	jogibear9988
Assignees:	-
Labels:	area-System.Text.Json, untriaged
Milestone:	-

[jogibear9988]

If you run the example code in net5 it will work. (it will still throw an exception, but you see that it trys to enumerate the IQueryable)

[jogibear9988]

A workaround I could use atm:

    public static IEnumerable AsUntypedEnumerable(this IQueryable query)
    {
        foreach(var o in query)
            yield return o;
    }

[huoyaoyuan]

The error message looks related to IAsyncEnumerable.

[eiriktsarpalis]

Agree with @huoyaoyuan's assessment. Presumably this happens because the ITable<T> interface inherits from IAsyncEnumerable<T>. I would say that this is by design. Out of curiosity, what was the expected serialization here?

[huoyaoyuan]

@eiriktsarpalis Some libraries like EF treats IAsyncEnumerable<T> as an improvement - the collection can be iterated either synchronise or asynchronise. Refuse the synchronise version if the asynchronise is available looks breaking.

[eiriktsarpalis]

Hmm. IAsyncEnumerable does take precedence over IEnumerable since it is assumed that types implementing both interfaces will block when iterating as IEnumerable<T>.

At the same time the IAsyncEnumerable converter statically has no visibility over whether IEnumerable<T> is implemented, as such it will (rightly) fail the serialization if attempting to serialize via the sync serialization APIs.

I think the best solution in the future might be to expose IEnumerable converter factory so that users are able to register that behavior for the types they need. A similar request has been filed for users needing to opt into the object converter for classes implementing IEnumerable.

[jogibear9988]

That all means the breaking change will stay as is in Net6?

But why does System.Text.Json use IAsyncEnumerable at all, in the synchronous serialization methods, when it can only serialize with async serialization methods?

[jogibear9988]

[jogibear9988]

I think System.Text.json should, when I use the synchronous methods, stop using the IAsyncEnumerable interface.
If I use SerializeAsync then it should, but then it also would not break.

[eiriktsarpalis]

I think System.Text.json should, when I use the synchronous methods, stop using the IAsyncEnumerable interface.

Unfortunately that is not technically possible with the current converter model. Each converted type/member is fixed to a specific converter instance (and from the converter's perspective it's serializing a TCollection : IAsyncEnumerable<T> with no visibility on whether TCollection is also IEnumerable<T>). I should also point out that dynamically falling back to IEnumerable<T> if using a sync method has the potential of producing inconsistent outputs depending on whether you're serializing synchronously or asynchronously (since there's no guarantee that the implementations produce identical elements).

The only real workaround is to force blocking enumeration of the IAsyncEnumerable<T> (e.g. via #60363), but again this is something we would like to avoid unless the user opted in to that. It also wouldn't work on the browser.

[eiriktsarpalis]

FWIW this is documented in a relevant breaking change document: https://docs.microsoft.com/en-us/dotnet/core/compatibility/serialization/6.0/iasyncenumerable-serialization#new-behavior

[jogibear9988]

in the breaking change you list that the async methods will enumerate the iasyncenumerable, not that the synchronous will fail.

The only real workaround is to force blocking enumeration of the IAsyncEnumerable, but again this is something we would like to avoid unless the user opted in to that.
==> how do I opt into this?