NAS-116836 / Force BSD semantics for group ownership if NFSV4ACL (#78)

When a new file is created on FreeBSD it is given the group
of the directory which contains it. On Linux it is given
to either the effective GID of the process (System V semantices)
or the GID of the parent directory (BSD semantics).

Since there is no hard-and-fast rule about creation semantics
for NFSv4 ACLs on Linux, we should opt for what is least likely
to break users permissions on change from FreeBSD to Linux.

Avoid setting actually setting the SGID bit on dirs unless
it was explicitly set.

Signed-off-by: Andrew Walker <awalker@ixsystems.com>

module/os/linux/zfs/zfs_vnops_os.c

@@ -588,6 +588,22 @@ zfs_create(znode_t *dzp, char *name, vattr_t *vap, int excl,
 	os = zfsvfs->z_os;
 	zilog = zfsvfs->z_log;
 
+	/*
+	 * For compatibility purposes with data migrated from FreeBSD
+	 * (which will have NFSv4 ACL type), BSD file creation semantics
+	 * are forced rather than System V. Hence on new file creation
+	 * if NFSV4ACL we inherit GID from parent rather than take current
+	 * process GID. This makes S_ISGID on directories a de-facto
+	 * no-op, but we still honor setting / removing it and normal
+	 * inheritance of the bit on new directories in case user changes
+	 * the underlying ACL type.
+	 */
+	if ((vap->va_mask & ATTR_MODE) &&
+	    S_ISDIR(ZTOI(dzp)->i_mode) &&
+	    (zfsvfs->z_acl_type == ZFS_ACLTYPE_NFSV4)) {
+		vap->va_gid = KGID_TO_SGID(ZTOI(dzp)->i_gid);
+	}
+
 	if (zfsvfs->z_utf8 && u8_validate(name, strlen(name),
 	    NULL, U8_VALIDATE_ENTIRE, &error) < 0) {
 		zfs_exit(zfsvfs, FTAG);
@@ -1214,6 +1230,22 @@ zfs_mkdir(znode_t *dzp, char *dirname, vattr_t *vap, znode_t **zpp,
 		return (error);
 	zilog = zfsvfs->z_log;
 
+	/*
+	 * For compatibility purposes with data migrated from FreeBSD
+	 * (which will have NFSv4 ACL type), BSD file creation semantics
+	 * are forced rather than System V. Hence on new file creation
+	 * if NFSV4ACL we inherit GID from parent rather than take current
+	 * process GID. This makes S_ISGID on directories a de-facto
+	 * no-op, but we still honor setting / removing it and normal
+	 * inheritance of the bit on new directories in case user changes
+	 * the underlying ACL type.
+	 */
+	if ((vap->va_mask & ATTR_MODE) &&
+	    S_ISDIR(ZTOI(dzp)->i_mode) &&
+	    (zfsvfs->z_acl_type == ZFS_ACLTYPE_NFSV4)) {
+		vap->va_gid = KGID_TO_SGID(ZTOI(dzp)->i_gid);
+	}
+
 	if (dzp->z_pflags & ZFS_XATTR) {
 		zfs_exit(zfsvfs, FTAG);
 		return (SET_ERROR(EINVAL));