Conversation
|
💔 -1 overall
This message was automatically generated. |
|
💔 -1 overall
This message was automatically generated. |
|
💔 -1 overall
This message was automatically generated. |
3d0177c to
412f3c5
Compare
|
Rebased on trunk. Adjusted Local tests: |
a6f6354 to
ef0809a
Compare
|
🎊 +1 overall
This message was automatically generated. |
|
CI is green on the current head, and this one still has not had a human review. @Hexiaoqiao @ZanderXu, when you have a moment, could you please take a look? Happy to make any follow-up changes. |
|
CI is still green on the current head, and this one is still waiting on human review. @hfutatzhanghb @Hexiaoqiao, when you have a moment, could you please take a look? Happy to make any follow-up changes. |
|
CI is still green on the current head, and this one is still waiting on HDFS review. @hfutatzhanghb @Hexiaoqiao, when you have a moment, could you please take a look? Happy to make any follow-up changes. |
1 similar comment
|
CI is still green on the current head, and this one is still waiting on HDFS review. @hfutatzhanghb @Hexiaoqiao, when you have a moment, could you please take a look? Happy to make any follow-up changes. |
Sorry for responding too late. Will reivew it quickly |
|
@deepujain Can you fix the checksyle warning? Others LGTM. |
|
Removed the redundant same-package import in |
|
Rebased this branch onto current |
5e6ae2c to
38cdba4
Compare
|
🎊 +1 overall
This message was automatically generated. |
|
No code changes here. I reran the Build workflow to refresh the red |
|
Pushed an empty |
|
💔 -1 overall
This message was automatically generated. |
|
Pushed another empty |
|
💔 -1 overall
This message was automatically generated. |
|
No code changes here. I reran the fork-side |
1a62b51 to
dacc840
Compare
|
No code changes here. I reran the current head again on July 12, 2026 to refresh the stale red GitHub Actions lane. The rerun still failed in the shared If the HDFS-side change itself looks good, @hfutatzhanghb @Hexiaoqiao, could you please take a look when you have a moment? Happy to make any follow-up changes. |
|
No code changes here. The HDFS-side patch and Yetus signal on the current head are unchanged, and this one is still waiting on human review as of July 20, 2026. @hfutatzhanghb @Hexiaoqiao, when you have a moment, could you please take a look? Happy to make any follow-up changes. |
|
No code changes here. I reran the existing fork-side |
|
No code changes here since the August 4, 2026 rebase onto current On the refreshed fork-side run, the failing GitHub Build lanes are stopping in the The earlier focused local validation on this branch remains the latest direct patch signal: |
|
No code changes in this update. I reran the fork-side GitHub Build workflow on the current head so the prior Build failure can get a fresh signal; the branch content itself is unchanged from the latest rebase and validation pass. |
|
No code changes in this update. The August 5 rerun progressed past the earlier image/container setup failure and most shards on the current head passed; the only remaining red GitHub Build lane is now |
|
No code changes here. I reran |
|
No code changes here. On the August 6, 2026 rerun, the fork-side Build progressed through image setup/build and most shards passed; the only red shard was |
|
No code changes here. I reran the existing fork-side |
|
Quick status refresh as of August 9, 2026: I reran the fork-side GitHub Build for the current head and that rerun is green. The focused local validation I used for this branch remains |
|
No code changes in this update. I pushed a no-code |
|
💔 -1 overall
This message was automatically generated. |
|
No code changes in this update. On the August 10, 2026 head, the fork-side GitHub |
8abc989 to
21afcbe
Compare
|
Rebased on current |
21afcbe to
1f232f4
Compare
|
Hi @slfan1989 @steveloughran @Hexiaoqiao, HDFS-17179 fixes CorruptMetaHeaderException reporting in DFSInputStream. Open since March, current on trunk, Yetus +1. Let me know if you can review or if I should ping someone else. |
|
Jenkins build 19 completed its test-result collection, then the worker exited with code 137. No patch-specific test failure appeared in the console. I pushed no-code commit |
e98d9ae to
a07e35d
Compare
|
Jenkins build 21 timed out after more than three hours and exited 143 while running the HDFS module tests, with no patch-specific failure in the console. I pushed a no-code CI refresh on e14bfde. Ready for the new CI run. |
|
🎊 +1 overall
This message was automatically generated. |
e14bfde to
ce487c2
Compare
|
Rebased cleanly onto current trunk at |
|
🎊 +1 overall
This message was automatically generated. |
|
I would ask on hdfs-dev list for reviewers, more human than just randomly asking people on prs who get pinged by so many others they get lost in the noise anyway |
ce487c2 to
ea07181
Compare
|
Rebased onto trunk |
|
🎊 +1 overall
This message was automatically generated. |
|
FYI, I have sent a review request to the Hadoop developer mailing list, following the recommendation here. Thanks for pointing me to that channel. |
4 participants
Summary
When a block read fails with
CorruptMetaHeaderException(e.g. corrupt or too-short block meta file on the DataNode), DFSInputStream did not add the block to the corrupted set or report it to the NameNode, unlikeChecksumException. Corrupt blocks were therefore not invalidated or re-replicated. This change treatsCorruptMetaHeaderExceptionthe same asChecksumExceptionin the client read path so the block is reported to the NameNode and can be re-replicated.Change
CorruptMetaHeaderException. InreadBuffer()(block read loop), add acatch (CorruptMetaHeaderException)that logs, adds the block tocorruptedBlocks, and setsretryCurrentNode = false, mirroring the existingChecksumExceptionhandling. InactualGetFromOneDataNode()(fetchBlockByteRange), add acatch (CorruptMetaHeaderException)that logs, adds the block tocorruptedBlocks, marks the datanode dead, and throws anIOException, mirroring theChecksumExceptionhandling.testReportCorruptMetaHeaderToNameNode(): create a file, corrupt the block meta file with an invalid 7-byte header, read from the client (expect IOException), then wait until the NameNode’s corrupt block count is 1 (HDFS-17179).JIRA
Fixes HDFS-17179
For code changes:
AI Tooling
Contains content generated by Codex.
How was this patch tested?
Earlier commands above are historical evidence, where present. Current rebase validation passed
git diff --check; fresh hosted CI is running. Focused Java validation is reported separately when complete.Validated repair head
ea07181e(2026-09-19)TestCorruptMetadataFile#testReportCorruptMetaHeaderToNameNodepassed: 1 test, 0 failures, 0 errors, 0 skipped. The selected reactor finished successfully in a Java 17/Maven 3.9.15 container with GNU stat and bounded memory (MAVEN_OPTS=-Xmx1024m, test heap 1536 MB).mvn -B -ntp -pl hadoop-hdfs-project/hadoop-hdfs -am -Dtest=TestCorruptMetadataFile#testReportCorruptMetaHeaderToNameNode -Dsurefire.failIfNoSpecifiedTests=false -Dmaven.test.failure.ignore=false "-Dmaven-surefire-plugin.argLine=-Xmx1536m -Xss4m" test