Fix ManagedNtlm on big-endian architectures

[shreyarao4]

Fixes multiple endianness bugs in ManagedNtlm that caused incorrect behavior on big-endian platforms.
Validated via existing Ntlm test cases.

[shreyarao4]

@uweigand @giritrivedi @saitama951

[dotnet-policy-service]

Tagging subscribers to this area: @dotnet/ncl, @bartonjs, @vcsjones
See info in area-owners.md if you want to be subscribed.

[wfurt]

LGTM in general.

[rzikm]

LGTM, Thanks!

[gfoidl]

Nit: spaces for code-style.

[gfoidl]

Nit:

Suggested change

	readonly get =>BitConverter.IsLittleEndian? _payloadOffset: BinaryPrimitives.ReverseEndianness(_payloadOffset);
	set =>_payloadOffset = BitConverter.IsLittleEndian? value: BinaryPrimitives.ReverseEndianness(value);
	readonly get => BitConverter.IsLittleEndian? _payloadOffset: BinaryPrimitives.ReverseEndianness(_payloadOffset);
	set => _payloadOffset = BitConverter.IsLittleEndian? value: BinaryPrimitives.ReverseEndianness(value);

[gfoidl]

Suggested change

	readonly get =>BitConverter.IsLittleEndian? _productBuild: BinaryPrimitives.ReverseEndianness(_productBuild);
	set =>_productBuild = BitConverter.IsLittleEndian? value: BinaryPrimitives.ReverseEndianness(value);
	readonly get => BitConverter.IsLittleEndian? _productBuild: BinaryPrimitives.ReverseEndianness(_productBuild);
	set => _productBuild = BitConverter.IsLittleEndian? value: BinaryPrimitives.ReverseEndianness(value);

[gfoidl]

Suggested change

	readonly get =>BitConverter.IsLittleEndian? _flags: (Flags)BinaryPrimitives.ReverseEndianness((uint)_flags);
	set =>_flags = BitConverter.IsLittleEndian? value: (Flags)BinaryPrimitives.ReverseEndianness((uint)value);
	readonly get => BitConverter.IsLittleEndian? _flags: (Flags)BinaryPrimitives.ReverseEndianness((uint)_flags);
	set => _flags = BitConverter.IsLittleEndian? value: (Flags)BinaryPrimitives.ReverseEndianness((uint)value);

[shreyarao4]

@rzikm @wfurt Unfortunately, the getters and setters method doesn't resolve the endianness issues because of MemoryMarshal.AsRef. We would have to go with the previous commit 91fc6d6. What can be done?

[rzikm]

@rzikm @wfurt Unfortunately, the getters and setters method doesn't resolve the endianness issues because of MemoryMarshal.AsRef. We would have to go with the previous commit 91fc6d6. What can be done?

Can you be more specific, what about the MemoryMarshal.AsRef is making the changes broken?

[shreyarao4]

MemoryMarshal.AsRef is writing the fields in little-endian, not in the correct byte order overwriting the struct fields with little-endian data.

[rzikm]

MemoryMarshal.AsRef is not writing anything, if you cast Span<byte> to e.g. ref MessageField, the getters/setters added in the PR should take care of endianness conversion on read/write. Similarly, if you create MessageField variable on it's own and copy it to a ref MessageField variable, the backing data should already have the correct endianness. Same if you cast the source MessageField to Span<byte> and copy the raw bytes to the target buffer.

Can you point to the specific piece of code where you see the problem, or give a code example where the pattern does not work?

[shreyarao4]

https://github.com/dotnet/runtime/blob/main/src/libraries/System.Net.Security/src/System/Net/NegotiateAuthenticationPal.ManagedNtlm.cs#L582
https://github.com/dotnet/runtime/blob/main/src/libraries/System.Net.Security/src/System/Net/NegotiateAuthenticationPal.ManagedNtlm.cs#L593

Here, challengeMessage.TargetInfo.Length and challengeMessage.TargetInfo.MaximumLength are endian flipped. Can you help me with this?

[rzikm]

I believe that is because the Length and MaximumLength fields are not corrected for endianness when reading

runtime/src/libraries/System.Net.Security/src/System/Net/NegotiateAuthenticationPal.ManagedNtlm.cs

Lines 126 to 137 in 1fd4dda

	[StructLayout(LayoutKind.Sequential)]
	private unsafe struct MessageField
	{
	public ushort Length;
	public ushort MaximumLength;
	private int _payloadOffset;
	public int PayloadOffset
	{
	readonly get =>BitConverter.IsLittleEndian? _payloadOffset: BinaryPrimitives.ReverseEndianness(_payloadOffset);
	set =>_payloadOffset = BitConverter.IsLittleEndian? value: BinaryPrimitives.ReverseEndianness(value);
	}
	}

@rzikm I think
the assertion happens here (http://148.100.85.217:8080/job/daily-builds/211/consoleFull)

runtime/src/libraries/Common/tests/System/Net/Security/FakeNtlmServer.cs

Lines 306 to 307 in c30eec2

	Flags flags = (Flags)BinaryPrimitives.ReadUInt32LittleEndian(incomingBlob.AsSpan(60));
	Assert.Equal(_requiredFlags, (flags & _requiredFlags));

we don't write in little endian order here, since response is of type AuthenticateMessage I don't see a getter / setter for the endianess here.

runtime/src/libraries/System.Net.Security/src/System/Net/NegotiateAuthenticationPal.ManagedNtlm.cs

Lines 645 to 646 in 7356734

	response.Header.MessageType = MessageType.Authenticate;
	response.Flags = s_requiredFlags | (flags & Flags.NegotiateSeal);

https://github.com/dotnet/runtime/blob/main/src/libraries/System.Net.Security/src/System/Net/NegotiateAuthenticationPal.ManagedNtlm.cs#L206

 while the original commit did handle this (the case where the test passed)
91fc6d6#diff-07ef1e3bc82826f3e53cb6d9d9f4efef56b93b428ebef12c806ca209063ce74bR632
 
 The file already consists of various getters and setter, which take care of the endinanness, i.e GetField and SetField ,
 
 

runtime/src/libraries/System.Net.Security/src/System/Net/NegotiateAuthenticationPal.ManagedNtlm.cs

Lines 324 to 360 in 7356734

	private static unsafe int GetFieldLength(MessageField field)
	{
	ReadOnlySpan<byte> span = new ReadOnlySpan<byte>(&field, sizeof(MessageField));
	return BinaryPrimitives.ReadInt16LittleEndian(span);
	}
	private static unsafe int GetFieldOffset(MessageField field)
	{
	ReadOnlySpan<byte> span = new ReadOnlySpan<byte>(&field, sizeof(MessageField));
	return BinaryPrimitives.ReadInt16LittleEndian(span.Slice(4));
	}
	private static ReadOnlySpan<byte> GetField(MessageField field, ReadOnlySpan<byte> payload)
	{
	int offset = GetFieldOffset(field);
	int length = GetFieldLength(field);
	if (length == 0 || offset + length > payload.Length)
	{
	return ReadOnlySpan<byte>.Empty;
	}
	return payload.Slice(GetFieldOffset(field), GetFieldLength(field));
	}
	private static unsafe void SetField(ref MessageField field, int length, int offset)
	{
	if (length is < 0 or > short.MaxValue)
	{
	throw new Win32Exception(NTE_FAIL);
	}
	Span<byte> span = MemoryMarshal.AsBytes(new Span<MessageField>(ref field));
	BinaryPrimitives.WriteInt16LittleEndian(span, (short)length);
	BinaryPrimitives.WriteInt16LittleEndian(span.Slice(2), (short)length);
	BinaryPrimitives.WriteInt32LittleEndian(span.Slice(4), offset);
	}

 can't we reuse those?

#125039 should be addressing these, let's move conversation there if necessary.