cDAC data contract versioning

Background

Right now our data contracts are described with a name and a version number such "Thread" version 1 or "DebugInfo" version 2. Although version often implies that higher numbers are better or richer that isn't necessarily the case here. We could have Thread version=1 is the CoreCLR implementation and Thread version=2 is the NativeAOT implementation. These implementations would be different but neither of them is better. Right now version is best considered an opaque identifier, not an ordered one.

The problem:

As the runtime changes its likely we'll have two types of changes:

[Breaking] Changes that redesign internal data structures in ways that are incompatible with the existing data contract
[Non-breaking] Changes that add new functionality while continuing to satisfy all the existing contracts.

Our current versioning scheme appears to be a natural fit for the breaking changes, but we need to decide how to handle non-breaking changes.

Some Options

Within our existing scheme non-breaking changes could be described as the addition of a new contract, but using that approach will probably lead to a large number of very fine-grained contracts. For example cDAC - fix ReJIT IsEnabled #124441 adds a single flag to an existing data structure.
We could represent all changes as being breaking even when they don't need to be. This is simple to reason about for runtime developers but requires more frequent diagnostic tool upgrades. Once our initial implementation of the contracts is complete I'd speculate that non-breaking changes might occur 3-10 times per release while breaking changes may only occur once every 1-3 releases. Every breaking change requires updated versions of the tools are available and deployed by users. Even with coordination, synchronization lag times ranging from days to months seem likely during which users either need to avoid using a new runtime version or accept not having working tools. This isn't something we'll want to do frequently or it will add considerable friction to using daily/preview builds of the runtime.
We could mark certain parts of contracts as being conditionally supported depending on whether all the necessary fields/globals are present in the descriptor. This could allow discovering some non-breaking functionality but it wouldn't work for everything. For example .NET 11 might ship a global field where 3 of the bits are described and the remaining bits are classified as reserved. Then in .NET 12 if we wanted to commit to the behavior of an additional bit by default there would be no new field to detect. Alternate variations of this might introduce new global fields whose only purpose is to signal a non-breaking change occured elsewhere.
We could add minor versions explicitly in the datacontract descriptor. One concrete example of how we could do this:

Change the contract descriptor "version" field from 0 -> 1
Modify the contracts element to be a string -> string map rather than string -> number. So that we can encode "<major>.<minor>". We'd require minor to be a numeric value but optionally relax the requirement that major is a number.
Non-breaking changes would only increment the minor version number.

For example: "contracts": {"Thread": "1.0", "GCHandle": "1.7", "ThreadStore": "NAOT.4"}

Thoughts?

The strategies aren't mutually exclusive but I assume we'd prefer some degree on consistency. IMO option (1) is nice for large additive changes and option (4) is nice for smaller ones. Option (3) seems doable but I don't favor it because it will require additional probing for all the optional fields.

We should discuss and then update our docs based on the decision.

@jkotas

Link to the past open questions issue: #100162 (this is one of them)

feedback welcome @jkotas @dotnet/dotnet-diag

@steveisok

Tagging subscribers to this area: @steveisok, @tommcdon, @dotnet/dotnet-diag
See info in area-owners.md if you want to be subscribed.

I'd speculate that non-breaking changes might occur 3-10 times per release while breaking changes may only occur once every 1-3 releases.

I agree that it is what one would expect intuitively.

We may find something else in practice. Our experience from R2R versioning is that we are doing breaking changes more often than one would intuitively expect, and non-breaking changes are not that common.

add minor versions

IIRC, we discussed this when the original data contract spec was written. There was not a consensus at the time that the minor versions are worth the complications. (Personally, I would be fine with minor version to denote non-breaking changes.)

We may find something else in practice. Our experience from R2R versioning is that we are doing breaking changes more often than one would intuitively expect, and non-breaking changes are not that common.

I'd be fine adding the capability to do minor versioning even if it turned out we used it less than anticipated in the future. In terms of the parallel with R2R, my understanding is that R2R breaking changes go unnoticed by most users because the R2R images ship together with the runtime. For diagnostic tools I expect the consequences of a breaking change will be far more noticeable which should encourage us to apply a little more effort to design non-breaking changes when possible.

There was not a consensus at the time that the minor versions are worth the complications.

Do you recall which complications/concerns were raised? When I thought about it the extra complexity seemed minor (haha) and well worth it if it avoided a breaking change.

#99936 (comment) and other comments that mention "version"

@davidwrighton

Thanks Jan! Scanning through I didn't notice any concerns about having some way to represent a non-breaking change but there were opinions around the particular mechanisms to implement it. I didn't notice anything yet that changed my preference away from option (4), but adding @davidwrighton and @AaronRobinsonMSFT.

David/Aaron - any thoughts on this?

I don't see a particular reason to have a minor version. We're simply changing a "if field exists" to a "if minor >= X". The latter doesn't make anything clearer and in my opinion creates an illusion of reliability on the versioning scheme. Inevitably someone will write "if minor >= X and field exists", which is confusing at best and actively annoying when trying to find stable ground.

Versions should represent incompatibility states (breaking or algorithmic) and optional fields permit us to make non-breaking check in a way that helps us understand there is one pattern - if it is there, use it.

Aside, this is precisely why I wanted to avoid versioning both the data structures and algorithms. We are creating numbering schemes that will create multiple patterns that make understanding when to change them hard.

We would use the latest minor version of the contract that is less than or equal to that of the runtime. There would be no if check except for this logic in the contract cache initialization. Doing "if field exists" checks everywhere breaks down when you have >1 non-breaking change on a field.

Thanks Aaron!

We're simply changing a "if field exists" to a "if minor >= X"

Not every non-breaking change naturally comes with a new field. For example adding a new flag bit on a pre-existing flags field. Alternately some non-breaking changes may be associated with many new fields.

What is your preferred way to denote non-breaking changes when the natural implementation would use:

zero new fields
one new field (you've covered it)
>= 2 new fields

I think its important that we handle each of those cases. I'm not tied that minor version has to be the way we do it.

Aside, this is precisely why I wanted to avoid versioning both the data structures and algorithms

Sorry, I don't think I understood what you are getting at in this part.

zero new fields

I don't understand what this would mean. If the bit is set, do the action. We shouldn't be reading non-initialized data, that creates a load of confusion because it is "undefined". If we're already in this state then I agree it becomes harder to mentally address. Are we sure we are in that state?

>= 2 new fields

I don't see how this is different from the (1) new field. Wouldn't both fields then need to be optional? Furthermore if one is present and the other isn't, that seems like a bug we wouldn't ship.

Aside, this is precisely why I wanted to avoid versioning both the data structures and algorithms

Sorry, I don't think I understood what you are getting at in this part.

The point I'm making here is we're creating a very complex versioning scheme across multiple dimensions that is difficult to reason about and I'm mostly confident that it isn't needed in practice - (cross product scenario version [M.m] and data structure version [T.t]). Of course we can all come up with scenarios where it could be useful, but most (all?) of those should be addressed simply by having clear and robust policy about state (for example, we don't have "undefined state" flags). I'm happy to elaborate further offline, I realize the discussion at this point is largely unproductive.

I don't understand what this would mean. If the bit is set, do the action. We shouldn't be reading non-initialized data, that creates a load of confusion because it is "undefined". If we're already in this state then I agree it becomes harder to mentally address. Are we sure we are in that state?

Here is an example. MethodTable has a bunch of flags. Most of them are not currently exposed in the contract so cDAC should not be infering any meaning about them. Lets say in .NET 12 we find a use for the enum_flag_wflags2_unused_2 bit and now we do want to expose it to the debugger. We have to let cDAC know that bit is now safe to draw meaning about.

There are certainly ways to do it that don't involve a minor version. I'm hoping to have you select one of the alternatives you think is good. Then we document thats our strategy and off we go.

I'm happy to elaborate further offline

Sure, happy to chat. I'll try to find a time we are both free today or tomorrow. Thanks Aaron!

Lets say in .NET 12 we find a use for the enum_flag_wflags2_unused_2 bit and now we do want to expose it to the debugger. We have to let cDAC know that bit is now safe to draw meaning about.

The version check in that case is "If >= .NET 12". If it is during the .NET 12 timeframe, the state of all the bits is known and therefore if we need to add it in a .NET 12 servicing, adding a check for that is non-breaking and just works. If it is removed in .NET 13, then the check becomes "if == .NET 12".

@AaronRobinsonMSFT

Chatted some with @AaronRobinsonMSFT this afternoon. We landed on:

For non-breaking changes that add some canonical field, the presence of the field can be the signal of feature support
For non-breaking changes that don't have some canonical field (for example adding a bit), use a dedicated feature flag in a separate global field to indicate support.

We've already been using Feature flag checks in some parts of cDAC so this isn't a novel code pattern we'd be introducing. So far feature flags are mostly architecture dependent but the cDAC doesn't care why the runtime decided that some features are on when others are off.

Aaron and I didn't get a chance to get into the exact code pattern we'd see in the cDAC code but I am imagining something like this:

// this API could be public or an internal implementation detail depending on whether other
// parts of the API will require the caller to reason about it. If there is a good default result for
// APIs to return when the feature isn't supported then prefer not making it public
bool IsFeatureFooSupported()
{
    // Option A - field check
    // NOTE: The field could be cached in the contract constructor for faster performance.
    return _someType.Fields.ContainsKey("Foo");
    
    // Option B - feature flag check
    // NOTE: Current contracts tend to allocate one global per feature but we could be a little
    // more efficient and pack the bits. Minor optimization.
    return (_target.ReadGlobal<byte>(Constants.Globals.FeatureFlags) & FeatureFlag.Foo) != 0;
}

// This could be any API that exposes some information about the newly added feature. Here I used null
// to indicate the feature isn't present but we could use other values, TryGetFoo, or public IsSupported
// API + throw an exception. Thats an API design choice separate from the versioning.
public int? GetTheFoo()
{
    if(!IsFeatureFooSupported())
        return null;
    return target.Read<int>(address + (ulong)_fooField.Offset);
}

If folks are happy with this I'll update the docs. If you have any concerns please holler. Thanks all!
@rcj1 @jkotas @AaronRobinsonMSFT @davidwrighton @max-charlamb

@noahfalk

This looks good to me. Thanks.

A scenario that hadn't occurred to me that @noahfalk shared is below. I think it is helpful to share more broadly.

In preview 1 of .NET X+1, the runtime version is now X+1. No new non-breaking changes.
In preview 3 of .NET X+1, a new non-breaking change occurs and the cDAC responds with the "if >= .NET X+1" approach mentioned in cDAC data contract versioning #124448 (comment).
In preview 4 of .NET X+1, the runtime team gets a crashdump from a user on preview 2 of .NET X+1.

In the above scenario the cDAC being used (preview 4) wouldn't operate properly with the check added in (2) because in preview 2 the non-breaking change wasn't reflected in that version. It could be a benign action, but it does represent a release that may not work with the cDAC as intended. Using the feature flag approach allows us to handle this scenario.

Look fine to me.

We should clarify what we mean by non-breaking changes.

There are true non-breaking changes e.g. expose a new detail about the GC that does not interact with anything exposed by the existing contracts.

Then there are changes that modify the existing contracts, but the modification is relatively minor. #124441 is a change like that. For the purpose of this discussion, let's assume we would add both the profiler API flag and the matching contract at the same time - not as a bug fix for TODO. The old cDAC is not going to work 100% correctly against new runtime for changes like this. You need a new cDAC that is aware of this change to get 100% correct results. How are we going to ensure that you have a new cDAC in situations like this, and fail or warn otherwise?

We should clarify what we mean by non-breaking changes.

I'd say:

non-breaking - old tools work similar or better than before, non-blocking warnings are possible.
breaking - tools don't work or significantly degrade. Users see errors. cDAC APIs throw exceptions.

Its a good question because this is a different definition than we used in SOS. SOS treated breaking change as 'warn but proceed anyways'.

The old cDAC is not going to work 100% correctly against new runtime for changes like this.

I think there is room for judgement here, we'd have to decide are users better off with a degraded but operational diagnostic tool or are they better off with an error + forced upgrade. The "similar" in the definition above gives us wiggle room for discretion.

How are we going to ensure that you have a new cDAC in situations like this, and fail or warn otherwise?

We always have the breaking change option if we want it. Error + scenario blocked until upgrade.

We could build something that tried to do non-blocking notifications by having the cDAC compare versioning information it carries internally relative to what it observes in the runtime. For example cDAC could scan the descriptor to see if any field is present or feature flag bit is set that it doesn't recognize. That could trigger some cDAC IsUpToDate property to return false and the tool acts on that to print a notification to the user.

If we wanted to specifically warn on only a subset of descriptor changes we could define something explicit in the runtime data to signal it. For example we already have SOS_BREAKING_CHANGE_VERSION that plays this role for SOS specifically. We could expand its use to include cDAC too unless folks anticipate a need to signal cDAC and SOS warnings separately. (We could do a little renaming to bring the terminology into better alignment)

Thoughts? I'd be happy to expand the SOS_BREAKING_CHANGE thing as it leverage what we already have and would likely be useful. We could also add something finer grained if we want to indicate which specific contracts will be degraded rather than doing it globally. I don't know how much value that adds but it matches the effort we put in to have multiple contracts rather than a single mega-contract.

non-breaking - old tools work similar or better than before, non-blocking warnings are possible.
breaking - tools don't work or significantly degrade. Users see errors. cDAC APIs throw exceptions.

This sounds reasonable to me as a principle.

SOS treated breaking change as 'warn but proceed anyways'.

I am not really worried about SOS. SOS has very low bar for definition of working. We can continue doing what we have been doing for SOS and similar tools.

I am more worried about the experience for end-user debuggers like VS.

For example cDAC could scan the descriptor to see if any field is present or feature flag bit is set that it doesn't recognize.

This sounds expensive and fragile to me. I think we would want something that's very simple for debuggers like VS to check, like a top-level version.

I think we would want something that's very simple for debuggers like VS to check, like a top-level version.

I think expanding the use of SOS_BREAKING_CHANGE satisfies that goal if its VS specifically. I think my open question is how much do we care about doing something that operates per-contract rather than globally? We went through a fair bit of trouble to carve out a set of different contracts that are independently versioned so it feels odd to turn around and ignore that defining a single global version when we are issuing advisory warnings.

I think the reason for carving set of contracts is maintainability of one implementation that works against different runtime versions and variants over time. For something like VS, I think we should have a simple freshness check to save us from having to chase down all corner cases. It is fine if some tools want to do it differently.

expanding the use of SOS_BREAKING_CHANGE

I would leave SOS_BREAKING_CHANGE alone and introduce something new. SOS_BREAKING_CHANGE covers exact layout of several data structures that are not abstracted by the SOS DAC APIs. It is both too strict (field offset changes are not breaking with cDAC) and insufficient (covers only a few data structures). Also, it would be a misleading name.

It is both too strict

👍 Good call - I wasn't paying close enough attention to that.

Navigation Menu

Repository navigation

cDAC data contract versioning #124448

Background

The problem:

Some Options

Thoughts?

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

cDAC data contract versioning #124448

Description

Background

The problem:

Some Options

Thoughts?

Activity

noahfalk commented on Feb 16, 2026

dotnet-policy-service commented on Feb 16, 2026

jkotas commented on Feb 16, 2026

noahfalk commented on Feb 16, 2026

jkotas commented on Feb 16, 2026

noahfalk commented on Feb 17, 2026

AaronRobinsonMSFT commented on Feb 18, 2026

rcj1 commented on Feb 19, 2026

noahfalk commented on Feb 22, 2026

AaronRobinsonMSFT commented on Feb 22, 2026

noahfalk commented on Feb 23, 2026

AaronRobinsonMSFT commented on Feb 23, 2026

noahfalk commented on Feb 24, 2026

AaronRobinsonMSFT commented on Feb 24, 2026

jkotas commented on Feb 24, 2026

noahfalk commented on Feb 25, 2026

jkotas commented on Feb 25, 2026

noahfalk commented on Feb 25, 2026

jkotas commented on Feb 26, 2026

noahfalk commented on Feb 26, 2026

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions