HDFS-16187. SnapshotDiff behaviour with Xattrs and Acls is not consistent across NN restarts with checkpointing #3340

bshashikant · 2021-08-26T15:17:21Z

Please check https://issues.apache.org/jira/browse/HDFS-16187

…tent across NN restarts with checkpointing.

szetszwo

Thanks for working on this. We may use java.util.Object.equals(a, b) to check null.

...project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/INodeDirectory.java

...doop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/INodeDirectoryAttributes.java

hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestEncryptionZones.java

bshashikant · 2021-09-08T06:19:40Z

The failures don't seem related. @szetszwo , @smengcl , can you please have a look?

szetszwo

+1 the change looks good.

cnauroth

@bshashikant , thank you for the patch.

I see that the patch works by converting logic from reference-equals to value-equals. In AclStorage, we maintain a reference-counted mapping of every immutable instance of every AclFeature currently in use by the namesystem. Inodes with identical ACL entries all point to the same AclFeature, so reference equals should work fine. This was an intentional choice of the ACL design to lower memory footprint and provide an inexpensive equality operation.

Xattrs do not use this same flyweight pattern though, so I can see why deep equals would be necessary there. I wonder if this bug really only happens for xattrs and not ACLs? If the bug happens for ACLs, then that might mean there is really something wrong with reference counting in AclStorage, causing duplicate instances for the same ACL entries.

Also, if value equals is necessary, then there is similar logic in INodeFile and INodeFileAttributes.

bshashikant · 2021-09-09T05:43:30Z

@bshashikant , thank you for the patch.

I see that the patch works by converting logic from reference-equals to value-equals. In AclStorage, we maintain a reference-counted mapping of every immutable instance of every AclFeature currently in use by the namesystem. Inodes with identical ACL entries all point to the same AclFeature, so reference equals should work fine. This was an intentional choice of the ACL design to lower memory footprint and provide an inexpensive equality operation.

Xattrs do not use this same flyweight pattern though, so I can see why deep equals would be necessary there. I wonder if this bug really only happens for xattrs and not ACLs? If the bug happens for ACLs, then that might mean there is really something wrong with reference counting in AclStorage, causing duplicate instances for the same ACL entries.

Also, if value equals is necessary, then there is similar logic in INodeFile and INodeFileAttributes.

Thanks @cnauroth for the explanation. I tested the acl feature and it seems the issue only seem to exist with Xattr feature not acl itself as you suggested. I have modified the patch accordingly. Thank you very much for the details.

hadoop-yetus · 2021-09-09T11:05:27Z

💔 -1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	0m 57s		Docker mode activated.
			_ Prechecks _
+1 💚	dupname	0m 0s		No case conflicting files found.
+0 🆗	codespell	0m 0s		codespell was not available.
+1 💚	@author	0m 0s		The patch does not contain any @author tags.
+1 💚	test4tests	0m 0s		The patch appears to include 3 new or modified test files.
			_ trunk Compile Tests _
+1 💚	mvninstall	31m 13s		trunk passed
+1 💚	compile	1m 23s		trunk passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04
+1 💚	compile	1m 15s		trunk passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
+1 💚	checkstyle	1m 2s		trunk passed
+1 💚	mvnsite	1m 24s		trunk passed
+1 💚	javadoc	0m 57s		trunk passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04
+1 💚	javadoc	1m 29s		trunk passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
+1 💚	spotbugs	3m 5s		trunk passed
+1 💚	shadedclient	16m 10s		branch has no errors when building and testing our client artifacts.
			_ Patch Compile Tests _
+1 💚	mvninstall	1m 13s		the patch passed
+1 💚	compile	1m 14s		the patch passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04
+1 💚	javac	1m 14s		the patch passed
+1 💚	compile	1m 9s		the patch passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
+1 💚	javac	1m 9s		the patch passed
+1 💚	blanks	0m 0s		The patch has no blanks issues.
+1 💚	checkstyle	0m 53s		hadoop-hdfs-project/hadoop-hdfs: The patch generated 0 new + 175 unchanged - 1 fixed = 175 total (was 176)
+1 💚	mvnsite	1m 14s		the patch passed
+1 💚	javadoc	0m 47s		the patch passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04
+1 💚	javadoc	1m 35s		the patch passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
+1 💚	spotbugs	3m 5s		the patch passed
+1 💚	shadedclient	16m 9s		patch has no errors when building and testing our client artifacts.
			_ Other Tests _
-1 ❌	unit	236m 16s	/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt	hadoop-hdfs in the patch passed.
+1 💚	asflicense	0m 47s		The patch does not generate ASF License warnings.
		321m 57s

Reason	Tests
Failed junit tests	hadoop.hdfs.TestRollingUpgrade

Subsystem	Report/Notes
Docker	ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3340/5/artifact/out/Dockerfile
GITHUB PR	#3340
Optional Tests	dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell
uname	Linux d5e8ef15c320 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/bin/hadoop.sh
git revision	trunk / `1f43713`
Default Java	Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
Multi-JDK versions	/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
Test Results	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3340/5/testReport/
Max. process+thread count	3547 (vs. ulimit of 5500)
modules	C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs
Console output	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3340/5/console
versions	git=2.25.1 maven=3.6.3 spotbugs=4.2.2
Powered by	Apache Yetus 0.14.0-SNAPSHOT https://yetus.apache.org

This message was automatically generated.

cnauroth

Thanks for the update, @bshashikant.

Do you also want to update the similar logic in INodeFile and INodeFileAttributes.SnapshotCopy?
Related to that, I wonder if the bug also exists for files. Right now, the unit test exercises a directory. Could you also write a test that covers snapshotting a directory without xattrs containing a file that has xattrs? If that test fails before your patch but passes afterwards, then we know it's a good test to keep.
For each of the metadataEquals methods that you are updating (4 total?), would you please add a brief comment stating that reference-equals is intentional for ACLs because of the reference counting in AclStorage? That might prevent confusion for future maintainers.

bshashikant · 2021-09-10T03:19:08Z

Thanks @cnauroth . The problem here seems to be very specific to snapshotRoot.
`
@test

public void testXattrOnFileWithSnapshotAndNNRestart() throws Exception {
FileSystem.mkdirs(hdfs, path, FsPermission.createImmutable((short) 0700));
Path file = new Path(path, "file1");
FileSystem.create(hdfs, file, FsPermission.createImmutable((short) 07777));
hdfs.setXAttr(file, name1, value1);
hdfs.allowSnapshot(path);
hdfs.createSnapshot(path, snapshotName);
SnapshotDiffReport report =
hdfs.getSnapshotDiffReport(path, snapshotName, "");
System.out.println(report);
Assert.assertEquals(0, report.getDiffList().size());
report =
hdfs.getSnapshotDiffReport(path, snapshotName, "");
System.out.println(report);
Assert.assertEquals(0, report.getDiffList().size());
hdfs.setSafeMode(HdfsConstants.SafeModeAction.SAFEMODE_ENTER);
hdfs.saveNamespace();
hdfs.setSafeMode(HdfsConstants.SafeModeAction.SAFEMODE_LEAVE);
cluster.restartNameNode(true);
report =
hdfs.getSnapshotDiffReport(path, snapshotName, "");
System.out.println(report);
Assert.assertEquals(0, report.getDiffList().size());
}`

The same test with xattrs set on file inside the snapshotRoot/SubDirecory will just work with/without the patch.

cnauroth · 2021-09-10T03:53:25Z

@bshashikant , thanks! In that case, let's please just add the comments I requested to INodeDirectory and INodeDirectoryAttributes. Then I'll approve and merge.

bshashikant · 2021-09-10T04:41:41Z

@bshashikant , thanks! In that case, let's please just add the comments I requested to INodeDirectory and INodeDirectoryAttributes. Then I'll approve and merge.

Thanks @cnauroth . As the problem is very specific to snapshot root, "metedataEquals()" function is now overriden in Snapshot.Root() and added the comments as per your suggestion

bshashikant · 2021-09-10T04:41:59Z

@bshashikant , thanks! In that case, let's please just add the comments I requested to INodeDirectory and INodeDirectoryAttributes. Then I'll approve and merge.

Thanks @cnauroth . As the problem is very specific to snapshot root, "metedataEquals()" function is now overriden in Snapshot.Root() and added the comments as per your suggestion

@szetszwo , can you have another look?

hadoop-yetus · 2021-09-10T10:22:39Z

💔 -1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	0m 50s		Docker mode activated.
			_ Prechecks _
+1 💚	dupname	0m 0s		No case conflicting files found.
+0 🆗	codespell	0m 0s		codespell was not available.
+1 💚	@author	0m 0s		The patch does not contain any @author tags.
+1 💚	test4tests	0m 0s		The patch appears to include 3 new or modified test files.
			_ trunk Compile Tests _
+1 💚	mvninstall	30m 52s		trunk passed
+1 💚	compile	1m 24s		trunk passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04
+1 💚	compile	1m 15s		trunk passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
+1 💚	checkstyle	1m 6s		trunk passed
+1 💚	mvnsite	1m 25s		trunk passed
+1 💚	javadoc	0m 57s		trunk passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04
+1 💚	javadoc	1m 32s		trunk passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
+1 💚	spotbugs	3m 14s		trunk passed
+1 💚	shadedclient	16m 29s		branch has no errors when building and testing our client artifacts.
			_ Patch Compile Tests _
+1 💚	mvninstall	1m 14s		the patch passed
+1 💚	compile	1m 22s		the patch passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04
+1 💚	javac	1m 22s		the patch passed
+1 💚	compile	1m 7s		the patch passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
+1 💚	javac	1m 7s		the patch passed
+1 💚	blanks	0m 0s		The patch has no blanks issues.
+1 💚	checkstyle	0m 53s		hadoop-hdfs-project/hadoop-hdfs: The patch generated 0 new + 184 unchanged - 1 fixed = 184 total (was 185)
+1 💚	mvnsite	1m 19s		the patch passed
+1 💚	javadoc	0m 50s		the patch passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04
+1 💚	javadoc	1m 27s		the patch passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
+1 💚	spotbugs	3m 12s		the patch passed
+1 💚	shadedclient	16m 3s		patch has no errors when building and testing our client artifacts.
			_ Other Tests _
-1 ❌	unit	249m 39s	/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt	hadoop-hdfs in the patch passed.
+1 💚	asflicense	0m 47s		The patch does not generate ASF License warnings.
		335m 27s

Reason	Tests
Failed junit tests	hadoop.hdfs.TestRollingUpgrade
	hadoop.hdfs.TestHDFSFileSystemContract

Subsystem	Report/Notes
Docker	ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3340/7/artifact/out/Dockerfile
GITHUB PR	#3340
Optional Tests	dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell
uname	Linux 727f2af57434 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/bin/hadoop.sh
git revision	trunk / `c273207`
Default Java	Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
Multi-JDK versions	/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
Test Results	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3340/7/testReport/
Max. process+thread count	3006 (vs. ulimit of 5500)
modules	C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs
Console output	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3340/7/console
versions	git=2.25.1 maven=3.6.3 spotbugs=4.2.2
Powered by	Apache Yetus 0.14.0-SNAPSHOT https://yetus.apache.org

This message was automatically generated.

hadoop-yetus · 2021-09-10T10:24:51Z

🎊 +1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	0m 41s		Docker mode activated.
			_ Prechecks _
+1 💚	dupname	0m 0s		No case conflicting files found.
+0 🆗	codespell	0m 1s		codespell was not available.
+1 💚	@author	0m 0s		The patch does not contain any @author tags.
+1 💚	test4tests	0m 0s		The patch appears to include 3 new or modified test files.
			_ trunk Compile Tests _
+1 💚	mvninstall	35m 47s		trunk passed
+1 💚	compile	1m 23s		trunk passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04
+1 💚	compile	1m 14s		trunk passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
+1 💚	checkstyle	1m 1s		trunk passed
+1 💚	mvnsite	1m 28s		trunk passed
+1 💚	javadoc	0m 57s		trunk passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04
+1 💚	javadoc	1m 27s		trunk passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
+1 💚	spotbugs	3m 16s		trunk passed
+1 💚	shadedclient	16m 13s		branch has no errors when building and testing our client artifacts.
			_ Patch Compile Tests _
+1 💚	mvninstall	1m 16s		the patch passed
+1 💚	compile	1m 21s		the patch passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04
+1 💚	javac	1m 21s		the patch passed
+1 💚	compile	1m 14s		the patch passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
+1 💚	javac	1m 14s		the patch passed
+1 💚	blanks	0m 0s		The patch has no blanks issues.
+1 💚	checkstyle	0m 57s		hadoop-hdfs-project/hadoop-hdfs: The patch generated 0 new + 184 unchanged - 1 fixed = 184 total (was 185)
+1 💚	mvnsite	1m 15s		the patch passed
+1 💚	javadoc	0m 49s		the patch passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04
+1 💚	javadoc	1m 23s		the patch passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
+1 💚	spotbugs	3m 22s		the patch passed
+1 💚	shadedclient	16m 25s		patch has no errors when building and testing our client artifacts.
			_ Other Tests _
+1 💚	unit	252m 54s		hadoop-hdfs in the patch passed.
+1 💚	asflicense	0m 46s		The patch does not generate ASF License warnings.
		342m 53s

Subsystem	Report/Notes
Docker	ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3340/6/artifact/out/Dockerfile
GITHUB PR	#3340
Optional Tests	dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell
uname	Linux e927483d6afb 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/bin/hadoop.sh
git revision	trunk / `cebf98d`
Default Java	Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
Multi-JDK versions	/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
Test Results	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3340/6/testReport/
Max. process+thread count	2866 (vs. ulimit of 5500)
modules	C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs
Console output	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3340/6/console
versions	git=2.25.1 maven=3.6.3 spotbugs=4.2.2
Powered by	Apache Yetus 0.14.0-SNAPSHOT https://yetus.apache.org

This message was automatically generated.

cnauroth

+1

Thanks for incorporating the feedback, @bshashikant . I will hold off committing in case you want to hear back from @szetszwo first.

szetszwo · 2021-09-11T02:44:05Z

@cnauroth , thanks a lot for helping out.

... Inodes with identical ACL entries all point to the same AclFeature, so reference equals should work fine. ...

In this case, it is a good idea to check == inside the equals(Object) method. Then, calling equals(Object) is equally cheap as checking ==.

+++ b/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/AclFeature.java
@@ -63,6 +63,9 @@ int getEntryAt(int pos) {
 
   @Override
   public boolean equals(Object o) {
+    if (this == o) {
+      return true;
+    }
     if (o == null) {
       return false;
     }

Let's merge this change as-is and do the further improvement separately.

+1

@bshashikant , it somehow got "All checks have failed". Could you trigger another build?

hadoop-yetus · 2021-09-12T14:48:04Z

💔 -1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	0m 58s		Docker mode activated.
			_ Prechecks _
+1 💚	dupname	0m 0s		No case conflicting files found.
+0 🆗	codespell	0m 0s		codespell was not available.
+1 💚	@author	0m 0s		The patch does not contain any @author tags.
+1 💚	test4tests	0m 0s		The patch appears to include 3 new or modified test files.
			_ trunk Compile Tests _
+1 💚	mvninstall	33m 55s		trunk passed
+1 💚	compile	1m 23s		trunk passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04
+1 💚	compile	1m 15s		trunk passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
+1 💚	checkstyle	1m 0s		trunk passed
+1 💚	mvnsite	1m 23s		trunk passed
+1 💚	javadoc	0m 55s		trunk passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04
+1 💚	javadoc	1m 25s		trunk passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
+1 💚	spotbugs	3m 19s		trunk passed
+1 💚	shadedclient	19m 22s		branch has no errors when building and testing our client artifacts.
			_ Patch Compile Tests _
+1 💚	mvninstall	1m 17s		the patch passed
+1 💚	compile	1m 21s		the patch passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04
+1 💚	javac	1m 21s		the patch passed
+1 💚	compile	1m 9s		the patch passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
+1 💚	javac	1m 9s		the patch passed
+1 💚	blanks	0m 0s		The patch has no blanks issues.
+1 💚	checkstyle	0m 54s		hadoop-hdfs-project/hadoop-hdfs: The patch generated 0 new + 184 unchanged - 1 fixed = 184 total (was 185)
+1 💚	mvnsite	1m 15s		the patch passed
+1 💚	javadoc	0m 48s		the patch passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04
+1 💚	javadoc	1m 21s		the patch passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
+1 💚	spotbugs	3m 19s		the patch passed
+1 💚	shadedclient	18m 50s		patch has no errors when building and testing our client artifacts.
			_ Other Tests _
-1 ❌	unit	316m 23s	/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt	hadoop-hdfs in the patch passed.
+1 💚	asflicense	0m 39s		The patch does not generate ASF License warnings.
		409m 35s

Reason	Tests
Failed junit tests	hadoop.hdfs.server.mover.TestMover
	hadoop.hdfs.server.datanode.TestDataNodeHotSwapVolumes

Subsystem	Report/Notes
Docker	ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3340/8/artifact/out/Dockerfile
GITHUB PR	#3340
Optional Tests	dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell
uname	Linux d0f51dac6e97 4.15.0-147-generic #151-Ubuntu SMP Fri Jun 18 19:21:19 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/bin/hadoop.sh
git revision	trunk / `19ac0d5`
Default Java	Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
Multi-JDK versions	/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
Test Results	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3340/8/testReport/
Max. process+thread count	2321 (vs. ulimit of 5500)
modules	C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs
Console output	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3340/8/console
versions	git=2.25.1 maven=3.6.3 spotbugs=4.2.2
Powered by	Apache Yetus 0.14.0-SNAPSHOT https://yetus.apache.org

This message was automatically generated.

bshashikant · 2021-09-13T04:53:06Z

Thanks @szetszwo . I have retriggered the CI and the unit test failures in the report are not related.

bshashikant · 2021-09-13T04:55:42Z

Thanks @cnauroth and @szetszwo for the review.

cnauroth · 2021-09-13T21:04:02Z

@bshashikant , thank you for the contribution and for incorporating the code review feedback.

…tent across NN restarts with checkpointing (apache#3340) (cherry picked from commit 356ebbb) Conflicts: hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestEncryptionZones.java Change-Id: If76fd0d77fafc90fe2f2c19ab1d0c43a58510f6b

…tent across NN restarts with checkpointing (#3340) (#3640) (cherry picked from commit 356ebbb) Conflicts: hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestEncryptionZones.java Change-Id: If76fd0d77fafc90fe2f2c19ab1d0c43a58510f6b Co-authored-by: bshashikant <shashikant@apache.org>

…tent across NN restarts with checkpointing (#3340) (#3640) (cherry picked from commit 356ebbb) Conflicts: hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestEncryptionZones.java Change-Id: If76fd0d77fafc90fe2f2c19ab1d0c43a58510f6b Co-authored-by: bshashikant <shashikant@apache.org> (cherry picked from commit 5333e87)

…tent across NN restarts with checkpointing (apache#3340)

… and Acls is not consistent across NN restarts with checkpointing (apache#3340) (cherry picked from commit 9b7be58afe41a75a85511b3d95c00519ca5071be) Change-Id: Ie1d00f7b029e3182c7a2c2d288599e3fd4088b9c

…few important for 7.1.7 SP1 : [1] Upstream JIRA ID: HDFS-16187. SnapshotDiff behaviour with Xattrs and Acls is not consistent across NN restarts with checkpointing (apache#3340) (cherry picked from commit 356ebbb) Change-Id: Ic971626aa30f49af0aaa653ca5f43c9d8078fc17 (cherry picked from commit be77d73)

HDFS-16187. SnapshotDiff behaviour with Xattrs and Acls is not consis…

69fb04d

…tent across NN restarts with checkpointing.

bshashikant requested a review from szetszwo August 26, 2021 15:17

This comment has been minimized.

Sign in to view

Fixed complilation issue.

e44b21f

This comment has been minimized.

Sign in to view

szetszwo reviewed Aug 28, 2021

View reviewed changes

jianghuazhu reviewed Aug 30, 2021

View reviewed changes

hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestEncryptionZones.java Outdated Show resolved Hide resolved

Addressed review comments.

dbb9608

This comment has been minimized.

Sign in to view

Addressed findbug and UT failures.

c7d160d

This comment has been minimized.

Sign in to view

szetszwo approved these changes Sep 8, 2021

View reviewed changes

cnauroth requested changes Sep 8, 2021

View reviewed changes

Addressed review comments.

1f43713

cnauroth requested changes Sep 9, 2021

View reviewed changes

Addressed review comments.

cebf98d

Remove inintended changes.

c273207

cnauroth approved these changes Sep 10, 2021

View reviewed changes

Trigger CI check

19ac0d5

bshashikant merged commit 356ebbb into apache:trunk Sep 13, 2021

kiran-maturi pushed a commit to kiran-maturi/hadoop that referenced this pull request Nov 24, 2021

HDFS-16187. SnapshotDiff behaviour with Xattrs and Acls is not consis…

69a28f4

…tent across NN restarts with checkpointing (apache#3340)

HDFS-16187. SnapshotDiff behaviour with Xattrs and Acls is not consistent across NN restarts with checkpointing #3340

HDFS-16187. SnapshotDiff behaviour with Xattrs and Acls is not consistent across NN restarts with checkpointing #3340

Uh oh!

Conversation

bshashikant commented Aug 26, 2021

Uh oh!

This comment has been minimized.

This comment has been minimized.

szetszwo left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

This comment has been minimized.

This comment has been minimized.

bshashikant commented Sep 8, 2021

Uh oh!

szetszwo left a comment

Choose a reason for hiding this comment

Uh oh!

cnauroth left a comment

Choose a reason for hiding this comment

Uh oh!

bshashikant commented Sep 9, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

hadoop-yetus commented Sep 9, 2021

Uh oh!

cnauroth left a comment

Choose a reason for hiding this comment

Uh oh!

bshashikant commented Sep 10, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

cnauroth commented Sep 10, 2021

Uh oh!

bshashikant commented Sep 10, 2021

Uh oh!

bshashikant commented Sep 10, 2021

Uh oh!

hadoop-yetus commented Sep 10, 2021

Uh oh!

hadoop-yetus commented Sep 10, 2021

Uh oh!

cnauroth left a comment

Choose a reason for hiding this comment

Uh oh!

szetszwo commented Sep 11, 2021

Uh oh!

hadoop-yetus commented Sep 12, 2021

Uh oh!

bshashikant commented Sep 13, 2021

Uh oh!

bshashikant commented Sep 13, 2021

Uh oh!

cnauroth commented Sep 13, 2021

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

bshashikant commented Sep 9, 2021 •

edited

Loading

bshashikant commented Sep 10, 2021 •

edited

Loading