HDDS-15424. Support concurrent positional read by taklwu · Pull Request #11102 · apache/ozone

taklwu · 2026-08-24T18:10:04Z

introduce stateless pread for major input streams.
use cursor editor to assist this change.

What changes were proposed in this pull request?

Fix Non-Stream read failed positional read checksum intermittently during concurrently access by multi-threads.

Please describe your PR in detail:

Fixing the concurrent positional read by introducing stateless pread for major input streams (BlockInputStream, ChunkInputStream, MultiPartInputStream) as the default pread method, such that the read could be getting the range / chunkinfo via readChunk (additional RPC) and use copyRange without touching shared cursor / position and avoid race condition. any child class of ExtendedInputStream is now supported with stateless pread.

The problem without thread-safe inputstream, checksum may fail because the offset has moved from the current position.

note that hadoop's FSInputStream does have this synchronized block and make sure the input stream is thread-safe. Also we're learning from DFSInputStream's getBlockRange and have this logic as an additional RPC that used by readChunk and copyRange.

What is the link to the Apache JIRA

https://issues.apache.org/jira/browse/HDDS-15424

How was this patch tested?

provided unit tests
tested on a HBase's Regionserver with this change, and after Andrey suggestion, the stateless read has less impact on the performance and I cannot find any significant regression from the YCSB 100% random read case.

Copilot

Pull request overview

This PR introduces an opt-in mechanism to make positioned reads (pread) thread-safe in OzoneFS by serializing the seek-read-restore sequence inside OzoneFSInputStream, addressing intermittent checksum failures when multiple threads reuse the same input stream.

Changes:

Add ozone.fs.synchronize.positioned.reads.enabled to optionally synchronize positioned reads in OzoneFSInputStream.
Plumb the new flag through the OzoneFS implementations so the created FS input streams honor the configuration.
Add a unit test that exercises concurrent positioned reads with the flag enabled/disabled.

Reviewed changes

Copilot reviewed 10 out of 10 changed files in this pull request and generated 1 comment.

Show a summary per file

File	Description
hadoop-ozone/ozonefs/src/main/java/org/apache/hadoop/fs/ozone/RootedOzoneFileSystem.java	Passes the new positioned-read synchronization flag when constructing the FS input stream wrapper.
hadoop-ozone/ozonefs/src/main/java/org/apache/hadoop/fs/ozone/OzoneFileSystem.java	Passes the new positioned-read synchronization flag when constructing the FS input stream wrapper.
hadoop-ozone/ozonefs-hadoop3/src/main/java/org/apache/hadoop/fs/ozone/RootedOzoneFileSystem.java	Same flag plumbing for the Hadoop3 variant.
hadoop-ozone/ozonefs-hadoop3/src/main/java/org/apache/hadoop/fs/ozone/OzoneFileSystem.java	Same flag plumbing for the Hadoop3 variant.
hadoop-ozone/ozonefs-common/src/test/java/org/apache/hadoop/fs/ozone/TestOzoneFSInputStream.java	Adds concurrent positioned-read coverage to validate the opt-in synchronization behavior.
hadoop-ozone/ozonefs-common/src/main/java/org/apache/hadoop/fs/ozone/OzoneFSInputStream.java	Implements optional serialization of positioned reads via an internal lock.
hadoop-ozone/ozonefs-common/src/main/java/org/apache/hadoop/fs/ozone/CapableOzoneFSInputStream.java	Extends constructors to propagate the new positioned-read synchronization option.
hadoop-ozone/ozonefs-common/src/main/java/org/apache/hadoop/fs/ozone/BasicRootedOzoneFileSystem.java	Reads the new config key and uses it when creating `OzoneFSInputStream`.
hadoop-ozone/ozonefs-common/src/main/java/org/apache/hadoop/fs/ozone/BasicOzoneFileSystem.java	Reads the new config key and uses it when creating `OzoneFSInputStream`.
hadoop-hdds/common/src/main/java/org/apache/hadoop/ozone/OzoneConfigKeys.java	Defines the new configuration key and default value.

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Copilot

Pull request overview

Copilot reviewed 21 out of 21 changed files in this pull request and generated 4 comments.

taklwu · 2026-08-25T16:06:49Z

the om integration test TestKeyLifecycleService$Normal.testMoveToTrashAbortTaskWhenTrashRootPrepareFails failed but I checked it's not related to this pread change.

rich7420 · 2026-08-26T16:12:35Z

+      return doPositionedRead(chunkRelativePosition, dst);
+    }
+    // Local (short-circuit) reads share a FileChannel cursor; serialize them.
+    synchronized (this) {


Could we synchronize local positioned reads on the shared block FileChannel, or use positional FileChannel reads? Each LocalChunkInputStream locks its own instance, while all chunks in the block receive the same blockFileInputStream, so preads to different chunks can still interleave position(...).read(...) and use the wrong offset.

yup, you're right about the positional FileChannel, I made the change and please review again.

rich7420 · 2026-08-26T16:12:42Z

+      int idx = chunkIndexForPosition(pos, offsets);
+      ChunkInputStream chunk = streams.get(idx);
+      long chunkPos = pos - offsets[idx];
+      int n = chunk.readPositioned(chunkPos, dst);


Could we preserve BlockInputStream’s outer retry and refresh handling here? readChunk still tries the DNs in the current pipeline, but this native path cannot refresh an expired block token or fetch an updated pipeline after those attempts fail.

thanks, this is an important miss.

rich7420 · 2026-08-26T16:13:57Z

@taklwu thanks for the patch!

szetszwo

@taklwu , thanks for working on this! BlockInputStream is not designed to support concurrent access. Filed HDDS-16325 to refactor it.

szetszwo · 2026-08-28T17:56:39Z

+    final long[] offsets = chunkOffsets;
+    final BlockData currentBlockData = blockData;
+    final long blockLength = length;


They need to be synchronized.

sorry, may ask why it need to be synchronized? I thought chunkOffsets, blockData and length were being initialized once in the initialize() which is a synchronized function already, here the readPositioned implemented as stateless and is trying to getting the the snapshot of this read only information for further operation within readPositioned

maybe I missed something that these three data will be changed over time?

szetszwo · 2026-08-28T18:02:41Z

+    final int startPosition = dst.position();
+    int preadRetries = 0;
+    while (true) {
+      final ChunkInputStream chunkStream = createChunkInputStream(chunkInfo);


createChunkInputStream uses xceiverClientFactory, xceiverClientGrpc and xceiverClientShortCircuit, which need to be synchronized.

I rechecked with below, so it seems the synchronized should not be required in the body

gRPC path (blockFileInputStream == null):

The ephemeral ChunkInputStream gets its own xceiverClient via ChunkInputStream.acquireClient() (synchronized per instance).

That uses xceiverClientFactory.acquireClientForReadData(), which is thread-safe internally (XceiverClientManager synchronizes on its cache).

Block-level xceiverClientGrpc / xceiverClientShortCircuit are only used for getBlockData() during init, not for pread data reads.

Short-circuit path:

LocalChunkInputStream uses positional FileChannel.read(buffer, pos) so concurrent preads on the shared block channel are safe.

Its acquireClient() is a no-op; it does not use the block’s short-circuit client for reads

but I did checked that the real edge case is pread vs close()/unbuffer(), which we can be fixed with a brief synchronized checkOpen() + snapshot before createChunkInputStream, without synchronizing the whole pread

I will address it in next revision.

szetszwo · 2026-08-29T20:49:52Z

... BlockInputStream is not designed to support concurrent access. Filed HDDS-16325 to refactor it.

Tried to implement HDDS-16325 but the BlockInputStream related classes are far from being able to support concurrent access.

Filed also HDDS-16335. I guess this will need more subtasks.

yandrey321 · 2026-08-31T13:21:54Z

@taklwu what is the use case in Hbase for concurrent seek/read access to InputStream from multiple threads in Hbase, it seems like an error-prone implementation, lets assume that 2 threads are doing:
Thread 1

seek(1000) - t1.1
2 read(1000) - t1.2

Thread 2

seek(0) - t2.1
read(100) t2.2

if they perform both perform operations without synchronizations between threads it can lead to the following execution sequence:

seek(1000) - t1.1
seek(0) - t2.1
read(1000) - t1.2
read(100) t2.2

And both threads would read data at the wrong offset.

And its true for any client. So if client does internal synchronization between readers' treads, InputThread itself can remain non-thread safe.

yandrey321

Do we really need to support concurrent read/seek from multiple threads? and what would be the guarantees from the InputStream perspective?

taklwu

thanks @szetszwo , I'm proposing to fix it in readChunkAt with below , and that should address your concern

  private int readChunkAt(ChunkInfo chunkInfo, long chunkOffset, int numBytesToRead, ByteBuffer dst)
      throws IOException {
    final int startPosition = dst.position();
    int preadRetries = 0;
    while (true) {
      final ChunkInputStream chunkStream;
      synchronized (this) {
        checkOpen();
        chunkStream = createChunkInputStream(chunkInfo);
      }

taklwu · 2026-09-01T16:20:41Z

+    final long[] offsets = chunkOffsets;
+    final BlockData currentBlockData = blockData;
+    final long blockLength = length;


sorry, may ask why it need to be synchronized? I thought chunkOffsets, blockData and length were being initialized once in the initialize() which is a synchronized function already, here the readPositioned implemented as stateless and is trying to getting the the snapshot of this read only information for further operation within readPositioned

maybe I missed something that these three data will be changed over time?

taklwu · 2026-09-01T16:31:35Z

+    final int startPosition = dst.position();
+    int preadRetries = 0;
+    while (true) {
+      final ChunkInputStream chunkStream = createChunkInputStream(chunkInfo);


I rechecked with below, so it seems the synchronized should not be required in the body

gRPC path (blockFileInputStream == null):

The ephemeral ChunkInputStream gets its own xceiverClient via ChunkInputStream.acquireClient() (synchronized per instance).

That uses xceiverClientFactory.acquireClientForReadData(), which is thread-safe internally (XceiverClientManager synchronizes on its cache).

Block-level xceiverClientGrpc / xceiverClientShortCircuit are only used for getBlockData() during init, not for pread data reads.

Short-circuit path:

LocalChunkInputStream uses positional FileChannel.read(buffer, pos) so concurrent preads on the shared block channel are safe.

Its acquireClient() is a no-op; it does not use the block’s short-circuit client for reads

but I did checked that the real edge case is pread vs close()/unbuffer(), which we can be fixed with a brief synchronized checkOpen() + snapshot before createChunkInputStream, without synchronizing the whole pread

I will address it in next revision.

taklwu · 2026-09-01T16:40:51Z

Do we really need to support concurrent read/seek from multiple threads? and what would be the guarantees from the InputStream perspective?

@taklwu what is the use case in Hbase for concurrent seek/read access to InputStream from multiple threads in Hbase, it seems like an error-prone implementation, lets assume that 2 threads are doing: Thread 1
1. seek(1000) - t1.1
   2 read(1000) - t1.2
Thread 2
1. seek(0) - t2.1

2. read(100) t2.2
if they perform both perform operations without synchronizations between threads it can lead to the following execution sequence:
1. seek(1000) - t1.1

2. seek(0) - t2.1

3. read(1000) - t1.2

4. read(100) t2.2
And both threads would read data at the wrong offset.

And its true for any client. So if client does internal synchronization between readers' treads, InputThread itself can remain non-thread safe.

yeah, you found the problem of what HBase has been using Hadoop's DFSInputStream that implemented ByteBufferReadable and ByteBufferPositionedReadable that DFSInputStream and FSInputStream are thread-safe for concurrent-read by multiple threads. So, if we need to support HBase on Ozone especially the positional read of HFile that has been using by HBase, we will need this feature and also better to be stateless pread that does not come with much performance penalty.

Do we really need to support concurrent read/seek from multiple threads? and what would be the guarantees from the InputStream perspective?

I don't have good judgement on this, but HBase is not the only one ask for this feature, you can see use case in HDDS-15734 also want this thread-safe positional read. So it depends on how much we want to support the existing HDFS's DFSInputStream use cases.

yandrey321 · 2026-09-01T17:18:03Z

I don't have good judgement on this, but HBase is not the only one ask for this feature, you can see use case in HDDS-15734 also want this thread-safe positional read. So it depends on how much we want to support the existing HDFS's DFSInputStream use cases.

is it possible to keep seek and read non-thread safe and optimized for non-concurrent usage; and guarantee thread safety for positional read APIs for concurrent usage?

szetszwo · 2026-09-01T17:20:02Z

... may ask why it need to be synchronized? ...

@taklwu , We are copying chunkOffsets, blockData and length there but some of them may mutate during copying.

taklwu · 2026-09-01T22:17:28Z

is it possible to keep seek and read non-thread safe and optimized for non-concurrent usage; and guarantee thread safety for positional read APIs for concurrent usage?

I thought we're already doing that in this PR , e.g. for the positioned read APIs aka PositionedReadable and ByteBufferPositionedReadable

org.apache.hadoop.fs.PositionedReadable (byte-array flavour)

int read(long position, byte[] buffer, int offset, int length)
void readFully(long position, byte[] buffer, int offset, int length)
void readFully(long position, byte[] buffer)

org.apache.hadoop.fs.ByteBufferPositionedReadable (ByteBuffer flavour)

int read(long position, ByteBuffer buf)
void readFully(long position, ByteBuffer buf)

we didn't touch the seek (e.g. MultipartInputStream#seek ) and other read() functions.

Mainly we make the existing read(position, ByteBuffer) function underlying implementation from non-thread-safe to thread-safe (without adding any synchronized to the header, and no performance regression) . see below

public boolean readFully(long position, ByteBuffer buffer) that uses BlockInputStream#readPositioned as stateless positional read (and that's the APIs), here I will need next JIRA to align with ByteBufferPositionedReadable API but the implementation is more or less the same as what I have here.

szetszwo · 2026-09-02T16:49:01Z

@taklwu , Both XceiverClientGrpc and XceiverClientShortCircuit are not threadsafe. In this PR, xceiverClient is not synchronized. The following is the trace:

ChunkInputStream.doPositionedRead //new code in this PR
-> ChunkInputStream.readChunk
-> ContainerProtocolCalls.readChunk
-> tryEachDatanode
-> ContainerProtocolCalls.readChunk
-> xceiverClient.sendCommand // a race condition for concurrent position-read.

taklwu · 2026-09-02T22:18:28Z

@szetszwo I made a change to make sure the xceiverClient are synchronized by making acquireClient return the client, would that help ?

szetszwo · 2026-09-03T17:50:58Z

... the xceiverClient are synchronized by making acquireClient return the client, ...

This sounds like a good idea! Let's separate the change to a new JIRA for refactoring the code. It will be easier to review.

taklwu · 2026-09-09T17:06:55Z

. xceiverClient are synchronized by making acquireClient return the client
This sounds like a good idea! Let's separate the change to a new JIRA for refactoring the code. It will be easier to review.

@szetszwo sorry I didn't express the statement clearly. I actually made that change and included as part of the latest revision. Can you review and see if that address your concerns? if not , we may need to split to another JIRA.

additionally, I split this code with two more followup that learnt form Sergey's path2 patches.

HDDS-16395 Add lock-free StreamBlock pread and datanode small-read optimization
HDDS-16396 Adopt PositionedReadable APIs and add concurrent pread integration test

szetszwo

@taklwu , thanks for the update!

You are right that xceiverClient looks good now.
This change is quite big. Let's separate the OzoneCryptoInputStream and OzoneFSInputStream changes to a different JIRA. (I have not reviewed them.)

Please see the comments inlined and also https://issues.apache.org/jira/secure/attachment/13084411/11102_review.patch

szetszwo · 2026-09-10T17:49:38Z

+  private int partIndexForPosition(long pos) {
+    int idx = Arrays.binarySearch(partOffsets, pos);
+    if (idx < 0) {
+      // binarySearch returns -insertionPoint - 1; the containing part is
+      // insertionPoint - 1.
+      idx = -idx - 2;
+    }
+    return idx;
+  }


Let's make it static and reuse it in BlockInputStream:

static int binarySearchOffsetIndex(long[] offsets, long pos) { final int idx = Arrays.binarySearch(offsets, pos); if (idx > 0) { return idx; } // binarySearch returns n = -insertionPoint - 1; // insertionPoint is -n - 1 // the containing index is insertionPoint - 1. return -idx - 2; }

taklwu · 2026-09-10T23:35:24Z

* This change is quite big.  Let's separate the OzoneCryptoInputStream and OzoneFSInputStream changes to a different JIRA. (I have not reviewed them.)

I removed the changes of OzoneCryptoInputStream and OzoneFSInputStream as well as the new tests in TestOzoneFSInputStream , also created JIRA HDDS-16400 Make positional read thread-safe for non-ExtendedInputStream

-align with HDDS-15920

- use positional FileChannel reads for local read - fix retry with the same token when using block input stream - remove the opt-in flag, default to stateless pread for ExtendedInputStream and fallback to sync if it's all other input stream

…putStream. - Use ephemeral ChunkInputStreams with per-call retry counters for block pread so concurrent positioned reads do not share sequential retry state or chunk stream buffers. - Serialize cursor-moving and positioned reads on OzoneCryptoInputStream because CryptoInputStream is not thread-safe.

…FSInputStream

- we need the protected readChunk in ChunkInputStream that used by LocalChunkInputStream and DummyChunkInputStream, other we will see NPE during execution. - fix unit test after HDDS-16100 merged

taklwu · 2026-09-11T07:52:32Z

+   * LocalChunkInputStream reads from a local FileChannel; no xceiver client is needed.
+   */
+  @Override
+  protected synchronized XceiverClientSpi acquireClient() {
+    return null;
+  }


need this after HDDS-16100 #11117 merged

taklwu · 2026-09-11T07:53:19Z

+  @Override
+  protected ByteBuffer[] readChunk(ChunkInfo readChunkInfo, XceiverClientSpi client) throws IOException {
+    return readChunk(readChunkInfo);
+  }


this is needed otherwise the datanodeBlockID is emitting NPE

taklwu · 2026-09-11T16:08:19Z

all below failures are not related, I may trigger another execution.

org.apache.hadoop.hdds.scm.TestXceiverClientMetrics (NullPointerException: "source" is null) should not be related
org.apache.hadoop.ozone.recon.TestReconAndAdminContainerCLI (timeout)
org.apache.hadoop.ozone.om.TestOzoneManagerHAWithStoppedNodes (flaky, OM HA test that couldn't elect a leader after stopping nodes)

taklwu · 2026-09-11T17:57:21Z

@szetszwo it's ready for another look

szetszwo · 2026-09-12T01:12:25Z

the tests with DummyChunkInputStream and the actual use case of LocalChunkInputStream may fail with NPE for datanodeBlockID if we change this back to private

Overriding the acquireClient/readChunk makes code confusing; see below for my suggestion

unless we agreed that the shared xceiverClient will not be set to null during the readPositioned(chunkRelativePosition, dst) when the releaseClient is called by either sequential read's handleReadError or unbuffer. if so, we can revert this readChunk(ChunkInfo, XceiverClientSpi) and have a simple readChunk(ChunkInfo) like before this change.

You are right that sequential read can release the client. Then, even if the client is non-null in pos read, it is still incorrect to use it after release.
BTW, the XceiverClientSpi implementations are not thread safe. Using the same client for concurrent pos reads is also incorrect.

So, I suggest to acquire a client directly from the factory and release it after use:

  int readPositioned(long chunkRelativePosition, ByteBuffer dst) throws IOException {
    if (chunkRelativePosition < 0 || chunkRelativePosition >= length) {
      return EOF;
    }
    final int toRead = (int) Math.min(dst.remaining(), length - chunkRelativePosition);
    if (toRead == 0) {
      return 0;
    }

    final ChunkInfo readChunkInfo = getChunkInfo(chunkRelativePosition, toRead);
    final long adjustedOffset = readChunkInfo.getOffset() - chunkInfo.getOffset();

    final long skip = chunkRelativePosition - adjustedOffset;
    if (xceiverClientFactory == null) {
      return copyRange(readChunk(readChunkInfo), skip, toRead, dst);
    }

    final Pipeline pipeline = pipelineSupplier.get();
    final ContainerProtos.DatanodeBlockID bid = buildDatanodeBlockId(pipeline, blockID);
    final XceiverClientSpi client = xceiverClientFactory.acquireClientForReadData(pipeline);
    try {
      final ByteBuffer[] readBuffers = readChunk(client, readChunkInfo, bid, validators, tokenSupplier.get());
      return copyRange(readBuffers, skip, toRead, dst);
    } finally {
      xceiverClientFactory.releaseClientForReadData(client, false);
    }
  }

  private static ByteBuffer[] readChunk(
      XceiverClientSpi client, ChunkInfo chunk, DatanodeBlockID blockID,
      List<Validator> validators, Token<? extends TokenIdentifier> token) throws IOException {
    Objects.requireNonNull(client, "client");
    final ReadChunkResponseProto readChunkResponse = ContainerProtocolCalls.readChunk(
        client, chunk, blockID, validators, token);

    if (readChunkResponse.hasData()) {
      return readChunkResponse.getData().asReadOnlyByteBufferList()
          .toArray(new ByteBuffer[0]);
    } else if (readChunkResponse.hasDataBuffers()) {
      List<ByteString> buffersList = readChunkResponse.getDataBuffers()
          .getBuffersList();
      return BufferUtils.getReadOnlyByteBuffersArray(buffersList);
    } else {
      throw new IOException("Unexpected error while reading chunk data " +
          "from container. No data returned.");
    }
  }

Copilot AI lite review requested due to automatic review settings August 24, 2026 18:10

Copilot started reviewing on behalf of taklwu August 24, 2026 18:10 View session

Copilot AI reviewed Aug 24, 2026

View reviewed changes

Comment thread hadoop-hdds/common/src/main/java/org/apache/hadoop/ozone/OzoneConfigKeys.java Outdated

taklwu force-pushed the HDDS-15424 branch from e1a8bc2 to cb9460a Compare August 24, 2026 18:17

taklwu changed the title ~~HDDS-15424. Non-Stream read failed positional read checksum intermitt…~~ HDDS-15424. Fix Concurrent positional read Aug 24, 2026

taklwu force-pushed the HDDS-15424 branch 2 times, most recently from 0b724bf to ffb81d6 Compare August 24, 2026 21:17

taklwu requested a lite review from Copilot August 24, 2026 22:40

Copilot started reviewing on behalf of taklwu August 24, 2026 22:40 View session

Copilot AI reviewed Aug 24, 2026

View reviewed changes

peterxcli self-requested a review August 24, 2026 23:54

taklwu force-pushed the HDDS-15424 branch 3 times, most recently from 8293565 to d45d09d Compare August 25, 2026 03:51

chungen0126 mentioned this pull request Aug 25, 2026

HDDS-15734. Implementing position read in BlockInputStream #10765

Closed

rich7420 reviewed Aug 26, 2026

View reviewed changes

peterxcli reviewed Aug 26, 2026

View reviewed changes

Comment thread hadoop-hdds/common/src/main/java/org/apache/hadoop/ozone/OzoneConfigKeys.java Outdated

taklwu force-pushed the HDDS-15424 branch from 0f9b353 to ccd1699 Compare August 27, 2026 19:11

szetszwo reviewed Aug 28, 2026

View reviewed changes

yandrey321 suggested changes Aug 31, 2026

View reviewed changes

taklwu commented Sep 1, 2026

View reviewed changes

szetszwo reviewed Sep 10, 2026

View reviewed changes

taklwu changed the title ~~HDDS-15424. Fix Concurrent positional read~~ HDDS-15424. Support concurrent positional read Sep 10, 2026

taklwu added 10 commits September 10, 2026 22:54

HDDS-15424. Fix Concurrent positional read

65e908d

Introduce stateless read for pread

38d59d2

-align with HDDS-15920

address comments

62b1358

- use positional FileChannel reads for local read - fix retry with the same token when using block input stream - remove the opt-in flag, default to stateless pread for ExtendedInputStream and fallback to sync if it's all other input stream

address synchronized for close and unbuffer in BlockInputStream

ab8ed5f

address mutate during copying

a5ee57f

fix indentation and remove white spaces

d4c89d5

fix comment and make synchronized acquireClient() return xceiverClient

af8f6e3

address comments, revert changes for OzoneCryptoInputStream and Ozone…

4e7e36f

…FSInputStream

fix unit tests

3f55a12

- we need the protected readChunk in ChunkInputStream that used by LocalChunkInputStream and DummyChunkInputStream, other we will see NPE during execution. - fix unit test after HDDS-16100 merged

taklwu force-pushed the HDDS-15424 branch from d827962 to 3f55a12 Compare September 11, 2026 07:38

taklwu commented Sep 11, 2026

View reviewed changes

retrigger test

b7f002e

Uh oh!

Conversation

taklwu commented Aug 24, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What changes were proposed in this pull request?

Please describe your PR in detail:

What is the link to the Apache JIRA

How was this patch tested?

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

taklwu commented Aug 25, 2026

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

taklwu Aug 26, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

rich7420 commented Aug 26, 2026

Uh oh!

Uh oh!

szetszwo left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

szetszwo commented Aug 29, 2026

Uh oh!

yandrey321 commented Aug 31, 2026

Uh oh!

yandrey321 left a comment

Choose a reason for hiding this comment

Uh oh!

taklwu left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

taklwu commented Sep 1, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

yandrey321 commented Sep 1, 2026

Uh oh!

szetszwo commented Sep 1, 2026

Uh oh!

taklwu commented Sep 1, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

szetszwo commented Sep 2, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

taklwu commented Sep 2, 2026

Uh oh!

szetszwo commented Sep 3, 2026

Uh oh!

taklwu commented Aug 24, 2026 •

edited

Loading

taklwu Aug 26, 2026 •

edited

Loading

taklwu commented Sep 1, 2026 •

edited

Loading

taklwu commented Sep 1, 2026 •

edited

Loading

szetszwo commented Sep 2, 2026 •

edited

Loading

taklwu commented Sep 9, 2026 •

edited

Loading