Repository navigation
Support polymorphic serialization through new option #29937
Description
EDIT see #29937 (comment) for the finalized API proposal.
Original proposal by @steveharter (click to view)
Current behavior
Consider a simple model:
abstract class Person {...}
class Customer : Person {...}The current behavior, by design, is to use the static type and not serialize polymorphically.
// This will serialize all properties on `Person` but not `Customer`:
Person person = new Customer();
string str = JsonSerializer.ToString<Person>(person);
// This does the same thing through generic type inference:
string str = JsonSerializer.ToString(person);However, if the type is declared to be System.Object then all properties will be serialized:
object person = new Customer();
// This will serialize all properties on `Customer`
string str = JsonSerializer.ToString(person);In addition, the Type parameter can be specified:
Person person = new Customer();
// This will serialize all properties on `Customer`
string str = JsonSerializer.ToString(person, person.GetType());All of these semantics are current as will remain as-is.
Proposed API
Add an opt-in setting that will serialize all types polymorphically, instead of just System.Object:
namespace System.Text.Json
{
public class JsonSerializerOptions // existing class
{
. . .
bool SerializePolymorphically { get; set; }
. . .
}
}Example of new behavior:
var options = new JsonSerializerOptions();
options.SerializePolymorphically = true;
// This will serialize all properties on `Customer`:
Person person = new Customer();
string str = JsonSerializer.ToString(person, options);This new behavior applies both to the root object being serialized (as shown above) plus any properties on POCOs including collections, such as List<Person>.
Note: this issue does not enable any additional support for polymorphic deserialize. Currently, a property or collection of type System.Object will result in a JsonElement being created. Any additional polymorphic deserialize support must be done with a custom converter.
Activity
steveharter commentedon Jun 20, 2019
Moving to future for the following reasons:
david-driscoll commentedon Jun 24, 2019
I just ran into this scenario when doing some testing with latest preview. Is this something that is documented elsewhere? If not this should be highlighted in the migration guide as a new "feature" for new applications switching to use only the new JSON serializer. Otherwise someone might not notice the subtle change in their applications for a while.
Symbai commentedon Jul 8, 2019
@steveharter The new custom converter feature cannot be used to handle polymorphic (de)serialization in some cases as, unless I'm missing something, because all properties have to be deserialized by hand / code. Which is not an option for maintainability when you have classes with more than two properties. See https://github.com/dotnet/corefx/issues/39031
The issue is that in the custom converter you only have a Utf8JsonReader which can read but not deserialize. Means you have to read one-by-one and set properties one-by-one.
If there however would be a way to deserialize an object, but only the object not the whole json, within the custom converter all at once, it becomes a feature that can be used.
steveharter commentedon Aug 8, 2019
@Symbai yes you are correct for deserialization -- there is not an easy way to support without manually authoring.
Here's a test\sample for using a type discriminator to do that:
Also here's a test\sample for polymorphic serialization. Currently the test\sample assumes the base class is abstract.
I'll work on adding real doc and samples shortly.
Symbai commentedon Aug 9, 2019
@steveharter Thanks but the samples cannot be used in real world scenarios because classes mostly don't have only 2 properties as shown in the sample, but ten to fifty. And now imagine we serialize something that itself has a class as property and this class also has another class and so on. So in the end, we have over hundreds of properties we, currently, have to write for EACH of them this here:
But if you have to write this more than 10 times already the code gets ugly and hardly maintainable.
Example
AlexanderJohnston commentedon Oct 2, 2019
This seriously needs to be documented somewhere, like on the System.Text.Json namespace, in highlighted text. I spent hours trying to track down why it was serializing empty strings while working in a Blazor project.
knom commentedon Oct 10, 2019
This is bad.. just ran into this as well.. after hours of debugging :-(
Is there a workaround using Newtonsoft.JSON?
steentottrup commentedon Oct 10, 2019
My own findings:
https://stackoverflow.com/questions/58305928/the-new-asp-net-core-3-0-json-serializer-is-leaving-out-data/58316304#58316304
And an article on how to "go back" to Newtonsoft.Json:
https://devblogs.microsoft.com/dotnet/try-the-new-system-text-json-apis/
Symbai commentedon Oct 10, 2019
On Newtonsoft it's super easy. For example just tell Newtonsoft all known Types and Newtonsoft does all the magic itself for you. No converter, no worries:
I really wish this would get more attention, at least set to 5.0 milestone instead of future. Currently it's just unusable for polymorphic situations and there is NO workaround (I consider the converter as no workaround in real world scenarios for the reason I posted earlier), so switching is not possible.
steveharter commentedon Oct 10, 2019
Moving to 5.0.
However this should also be tied to polymorphic deserialization support as well which is also a requested feature but much larger in scope. There are several requests for that and the implementation must be secure and opt-in through known types; we need to combine\triage these polymorphic issues -- here's one request: https://github.com/dotnet/corefx/issues/41347
bklooste commentedon Oct 21, 2019
Known types / magic polymorphism on the wire are bad patterns and I have seen people waste a ton of time on weird issues and bugs around this - the benefit aint worth it. Neither Json or XML are polymorphic by design and making them do it is a hack at best.
That said we should document explicit type creation better and possibly have some patterns around this eg
var typeName = json.GetProperty("TypeName").GetString(); var type = Type.GetType(typeName); return (T) JsonSerializer.Deserialize(utfJson , type);53 remaining items
qsdfplkj commentedon Jun 15, 2021
How about deserialzing and a tree of foo’s where nodes can be any subtype.
For top level polymorphic behavior as in your foo example you can easily use ‘serialize < object >‘ to get similar behavior?
besides I think you should be able to deserialize whatever you serialize. It would look really broken if my client serialize an object and the server can’t deserialize it.
eiriktsarpalis commentedon Jun 15, 2021
@qsdfplkj the particular feature has no provision for deserialization. This is a simple feature intended at extending the simple brand of polymorphism that is currently possible with
objectvalues. It should only be used in scenaria where you are only interested in dumping all properties of a type hierarchy on the wire or where the deserializing process is dynamically typed.Since it is not possible to roundtrip polymorphic values in nominal type systems without emitting some form of metadata on the wire, we are working on a separate feature that allows polymorphic deserialization using type discriminators (cf. #30083)
qsdfplkj commentedon Jun 15, 2021
@eiriktsarpalis So far it seems you can do all of it with
JsonSerializer.Serialize<object>(...). And forList<Foo>you either need to change it toList<object>or create aJsonConverter<List<T>>that callsJsonSerializer.Serialize(writer, value.Cast<object>().ToList());eiriktsarpalis commentedon Jun 17, 2021
This feature is orthogonal to #30083, which implements a different brand of polymorphism using type discriminators. Calling both features "polymorphism" is likely to confuse users who might expect the two API sets to be the same or compatible. They might reasonably expect that APIs serialized using this feature to be deserializable using APIs from #30083. We need to be careful about the names we pick so that they clearly illustrate that they are independent. Suggested names might include "Simple Polymorphism" or "Write-Only Polymorphism".
qsdfplkj commentedon Jun 17, 2021
I think its not needed because I can write such code already but SerializeRuntimeTypes vs SerializeDeclaredTypes is what this is about.
bartonjs commentedon Jun 17, 2021
Video
API Review Feedback:
public Dictionary<Type, IEnumerable<Type>> PolymorphicSerialization { get; }eiriktsarpalis commentedon Jun 17, 2021
Here is a sketch of a model that might accommodate both designs:
Here's how a hierarchy that doesn't emit any metadata can be defined:
And here's one that does:
A
JsonSerializerOptionsknob might have a shape equivalent toeiriktsarpalis commentedon Jun 17, 2021
Given the API review feedback I'm going to close this issue in favor of #30083, which will be revised to address both requirements.