[AndroidMessageHandler] ReadAsByteArrayAsync occasionally returning incorrect byte array #8740

Android application type

.NET Android (net8.0-android)

Affected platform version

34.0.79/8.0.100

Description

My app is syncing contents to and from users personal cloud storage (e.g. OneDrive, Google Drive, & Dropbox), using the respective official HTTP APIs. The files are encrypted and after downloading, my app decrypts them using Aes.DecryptCbc. Now, occasionally, this method throws an exception:

System.Security.Cryptography.CryptographicException: Cryptography_PartialBlock

System.Security.Cryptography.UniversalCryptoOneShot.OneShotDecrypt(ILiteSymmetricCipher , PaddingMode , ReadOnlySpan`1 , Span`1 , Int32& )
System.Security.Cryptography.AesImplementation.TryDecryptCbcCore(ReadOnlySpan`1 , ReadOnlySpan`1 , Span`1 , PaddingMode , Int32& )
System.Security.Cryptography.SymmetricAlgorithm.DecryptCbc(ReadOnlySpan`1 , ReadOnlySpan`1 , PaddingMode )

It appears that an incorrect byte array has been returned by HttpContent.ReadAsByteArrayAsync() and the decryption fails due to invalid padding.

The following reasons make me believe that this is a bug in AndroidMessageHandler rather than my apps code or the servers returning wrong data:

• when the CryptographicException mentioned above occurs, I am retrying the download of the same file & running the decryption again: it always works and doesn't throw an exception any more
• the issue only occurs for Android, but not on iOS, macOS or Windows (see how I configured the HttpClientHandlers below)
• the issue only occurs for requests to OneDrive and Google Drive, which do return gzip-compressed data when using Accept-Encoding: gzip,deflate in the http request. Dropbox on the other hand does not compress the http body when downloading files even when the Accept-Encoding header is present.

Therefore it appears to me that the problem is somehow connected to how the AndroidMessageHandler does decrypts the gzipped data before returning it to the application

Steps to Reproduce

I'd expect this problem very hard to reproduce. According to Google Console, my app does approx. a million requests to Google Drive & OneDrive every month and according to AppCenter, this issue occured ~100 times in the last 28 days. I did not yet observe the problem when downloading a file through the Dropbox API

As I am using the same HttpClient instance across the app and multiple requests are sent simultaneously, perhaps the issue is caused by some race condition.

public static AndroidMessageHandler HttpMessageHandler => new() { AutomaticDecompression = DecompressionMethods.Deflate | DecompressionMethods.GZip };
public static HttpClient HttpClient = new(HttpMessageHandler);

On iOS/macOS I use NSUrlSessionHandler. Note: I did not specify AutomaticDecompression as NSUrlSessionHandler is using gzip by default

public static NSUrlSessionHandler HttpMessageHandler => new()};

On Windows/UWP:

public static HttpClientHandler HttpMessageHandler => new() { AutomaticDecompression = DecompressionMethods.Deflate | DecompressionMethods.GZip };

And this is essentially how the request is sent

using var requestMessage = new HttpRequestMessage(HttpMethod.Get, url);
requestMessage.Headers.Authorization = new AuthenticationHeaderValue("Bearer", access_token);
var response = await HttpClient.SendAsync(requestMessage, cancelToken).ConfigureAwait(false);
var bytes = await response.Content.ReadAsByteArrayAsync().ConfigureAwait(false);

Did you find any workaround?

I will attempt to use another static HttpClient that is not configured with AutomaticDecompression that I will use for downloading binary data, so that the servers will not return gzipped data. As the data I am syncing is mostly .jpgs or .mp4s, gzip-compressing that data doesn't shrink the size any further, I might even save some CPU cycles and network bandwidth...